Machining feature and topological relationship recognition based on a multi-task graph neural network

Two-stage methods first train a single classifier for machining features and then decompose the multi-feature model based on geometric or topological prior knowledge. Finally, the decomposed sub-models are classified separately to obtain the final result.

The first deep learning method, FeatureNet [13], voxelizes 3D models, similar to the pixel concept of an image, using a 3D convolutional network to identify individual features. The 3D model is then decomposed using a watershed segmentation algorithm. MSVNet [27] is a multi-view approach that uses 2D images projected from the 3D model in different directions for single feature identification and then uses a selective search algorithm for feature localization. Wang et al. [28], [29] extracted face types and FAGs from B-Rep and then mixed GNNs and feature templates for single feature recognition, and then formulated depth-of-cut oriented rules for feature segmentation. Zhang et al. [30] used a two-stage approach to segment intersecting machining features based on attribute adjacency graphs of CAD models. By introducing deep reinforcement learning, this method overcomes the disadvantage that faces can only belong to a single machining feature in traditional deep learning methods. Still, its weak generalization ability and inability to achieve semantic classification of feature instances make it lack practical application value. Yao et al. [31] represented a 3D model with a single feature through a point cloud and utilized PointNet++ to classify the single feature. The point cloud's normal vector and edge concavity are subsequently designed for multi-feature 3D model decomposition. Mesh data is used in [32], where MeshCNN and faster regional convolutional neural networks are used for single feature recognition, and roughness annotations in the MBD model are used for feature localization.

In the above two-stage methods, the decomposition process of the multi-feature model is essentially an artificial rule formulation based on spatial continuity and topological connectivity, which makes it challenging to overcome the problems of spatial separation and topological destruction brought by feature interaction. In addition, there are time-consuming severe defects [12].

The drawbacks of the two-stage methods, such as the difficulty in handling highly interactive features, the severe time consumption, and the poor real-time performance, make it difficult to practically apply to real CAD models. Therefore, the researchers explored the one-stage methods for MFR to achieve real-time and effective feature recognition of feature-interacting parts by synchronizing the segmentation and classification of features.

With the development of 2D target detection, Shi et al. [26], [33] improved two single-stage target detectors, SSD and RefineDet, and successively proposed SsdNet and RDetNet methods in the object detection framework. These methods simultaneously predict the parameters of the 3D axis-aligned bracket box and the classes of machining features on multiple projection views of the 3D model. Finally, the proposals from numerous views are combined to achieve machining feature recognition.

Considering that STEP files hold all the information for 3D modeling, some researchers have analyzed STEP texts for MFR. Yeo et al. [34] extracted attributes of 3D models such as faces, edges, and rings by parsing STEP files and used a multilayer perceptron for semantic segmentation. In particular, Miles et al. [35], [36] introduced a natural language processing (NLP) approach for MFR. They analogously designed a STEP parser and successively performed single-feature and multi-feature recognition in an object detection framework based on the encoder-decoder model of the Long Short-Term Memory model (LSTM). However, the lack of explicit 3D model learning leads to a weak connection between geometry and topology.

B-Rep is a well-established, widely used, and clearly expressed representation in 3D solid modeling. It contains all the geometric and topological information necessary for 3D solid reconstruction, involving multi-level attributes such as vertices, edges, faces, rings, and bodies and their association relationships. B-Rep-based methods have been the most prominent methods for MFR and have shown the most competitive ability to achieve 100 % accuracy in single feature recognition [18]. Cao et al. [17] viewed MFR as a semantic segmentation problem and proposed CADNet. It extracts face adjacency graphs from B-Rep and utilizes the parameter vectors of each face as its node representation. However, the fact that the parameter vectors are only applicable to planar surfaces limits its application. Colligan et al. [18] proposed an improved method called Hierarchical CADNet. It discretizes faces into triangular slices and represents them hierarchically by the parameter vectors of these slices and their adjacencies. UV-Net [37] discretizes the faces and edges of the 3D model in its parameter space and learns face and edge embeddings based on 2D convolution and 1D convolution, respectively.

The instance segmentation framework for MFR has been studied relatively less at present. ASIN [12] utilizes face-level point cloud data to obtain geometric information and then uses two additional PointNet models for semantic segmentation and instance grouping. The final result of instance segmentation is obtained by encoding each face as a 64D vector using a distance matrix and a mean drift clustering algorithm. AAGNet [19] proposes a geometric attribute adjacency graph data structure to address the lost information caused by converting 3D models to intermediate representations such as point clouds and voxels. Instance segmentation of machining features is realized based on the graph data structure using the GNN encoder.

While the one-stage approaches described above accurately achieve machining feature recognition and localization at face-level, they lack consideration of extracting topological relationships between feature instances. This limitation makes it difficult for current MFR technology to support the realization of intelligent machining process planning.

To automate the generation of large-scale training data, the parametric modeling technique is used to create 3D CAD models using PythonOCC [38], a Python wrapper for Open CASCADE.

Specifically, the decision to add topological relationships and their types is made randomly after sketching the current feature. If topological relationships are to be added, the corresponding sketches are added using the appropriate operations. In order to prevent mutual interference between topological relations, this method employs certain rule constraints during the process of drawing groups of sketches with topological relations. For instance, with regard to the coplanar-based topological relationship (Coplanar, Circle Array, Line Array, and Mirror), it is necessary to ascertain whether the feature group with the aforementioned topological relationship already exists on the drawing face and its parallel faces. If this is the case, it is then necessary to select other faces for sketching. Furthermore, upon the addition of each feature, a systematic examination of the existing features on the sketched face and its parallel faces is conducted. If a coplanar-based topological relationship is identified between the newly added feature and the existing features, additional labels are generated. The relationships defined in Table 1 are mutually exclusive, i.e., there is one and only one relationship between a pair of features. Among them, the three relationships, Circle Array, Line Array, and Mirror, can be considered as special cases of Coplanar, i.e., if a set of features is defined as Coplanar, they do not have an Array or Mirror relationship with each other.

When the features on the bounding cube are applied, a Boolean Common operation is performed between the multi-layer part body and the bounding cube. Then, we obtain a multi-layer, multi-feature 3D model. Topological and feature integrity checks are performed on each 3D model. The topological integrity check [19] ensures the validity, closure, and manifoldness of the model. The feature integrity check ensures the model has not corrupted feature instances due to Boolean Common operations. Finally, the tested 3D models and their labels are saved. The distribution of feature types and topological relationship types in the MFTRCAD dataset is shown in Appendix A.

The semantic segmentation label marks each face in the 3D model as belonging to the class of its machining feature. For example, the three rectangular through slots in Fig. 3 include 9 faces, so the faces with IDs 18–26 are labeled as that machining feature class. The instance grouping label further classifies the constituent faces of each feature instance into a cluster. When the number of faces is $N_f$, the instance grouping label can form a 0–1 adjacency matrix of dimension $N_f\times N_f$. In the matrix, elements corresponding to faces belonging to the same feature instance are denoted as 1, whereas the remaining elements are denoted as 0. The topological relationship label treats a feature instance as a node forming a feature-level graph structure, labeling whether there is a topological relationship between nodes and the category of the relationship. As shown in Fig. 3, there is a Line Array relationship between the three rectangular through slots, so there are edges typed Line Array between the three feature nodes 4,5,6.

The high complexity and irregularity of the B-Rep data structure makes it difficult to be directly processed by machine learning methods [17], [18]. Transformation formats such as point clouds, meshes, voxels, etc., are not only limited by resolution but also challenging to capture the full geometric and topological information, which in turn affects the model's performance [37]. Therefore, this paper is based on geometric attribute adjacency graphs proposed in [19] to completely extract the geometric and topological information embedded in the B-Rep data structure.

Geometric information in B-Rep includes entities' size, position, and shape parameters. The faces and edges are defined through parametric equations, which are challenging to process directly by modern machine learning methods. Jayaraman et al. [37] propose a unified representation for B-rep data that utilizes U and V parameter domains of faces and edges to model geometry. In this paper, based on the same method, the faces and edges are discretized as UV grids with a fixed step size. The geometric information is characterized using each grid point's point coordinates, normal vectors, and tangent vectors. In addition, the geometric information extracted in this paper contains some attributes of faces and edges. The face attributes include face type, area, and center of mass coordinates; the edge attributes include edge type, length, and concavity.

The feature vector shaped $N_f\times 64D$ of each node is obtained after learning through multiple graph encoders. Further, the node feature vectors are passed into a readout layer to get a graph global information embedding shaped $1\times 64D$. The readout layer consists of a GAT layer [46], a pooling layer, and a linear layer. Finally, the graph global information embedding is concatenated with the feature vectors of each node to obtain the embedding shaped $N_f\times 128D$ that characterizes all the information of the graph.

Conventional multi-task learning frameworks tend to adopt the “Hard Parameter Sharing” paradigm, i.e., embedding the input information with a shared encoder and then using the embedding for several different tasks. However, the correlation patterns of the three tasks defined in this paper are complex and strongly dependent on the sample, which results in a model that improves some tasks but hinders the performance of others. This makes multi-task learning less effective than single-task models, which is known as the seesaw phenomenon [43]. Specifically, the semantic segmentation task is concerned with categorizing the different faces of a part according to their semantic categories; the instance grouping task requires identifying and distinguishing between different instances of the same semantic category; and the topological relationship prediction task aims to understand the spatial relationships between different instances. Although these three tasks are related in MFR, they differ in their goals and the type of information to be recognized. Therefore, in the case of shared resources, the performance improvement of one task may take up more model resources, thus affecting the performance of the other tasks. In addition, due to the uneven sample distribution of the three tasks in the dataset, the model may tend to optimize those tasks that are easier or more frequently occurring while ignoring the others.

In the topological information encoder, shared and task-specific knowledge are selectively fused through an information route constructed from a gating network. The gating network uses the input graph data as a selector and computes the weights of the knowledge vectors through a network consisting of a GCN layer [47], a Linear layer, and a SoftMax layer. The weighted sum shaped $N_f\times 128D$ of the outputs of the shared encoding module and the task-specific encoding module is computed based on the weight output from the gated network, thereby enabling the efficient fusion of shared and task-specific knowledge.

The embedding for task $k$ is expressed more precisely as follows:(3)$g^k\left(x\right)=w^k\left(x\right)S^k\left(x\right)$where $x$ is the input data, and $w^k(x)$ is the gating network used to calculate the weight vector of task $k$:(4)$w^k\left(x\right)=SoftMax\left(F\left(x\right)\right)$where $F$ denotes the transformation performed on the input by the GCN layer and the linear layer. $S^k(x)$ is a matrix composed of shared embeddings and task k’s specific embeddings:(5)$S^k\left(x\right)={\left[E_{\left(k,1\right)}^T,E_{\left(k,2\right)}^T,\cdots ,E_{\left(k,m\right)}^T,E_{\left(s,1\right)}^T,E_{\left(s,2\right)}^T,\cdots ,E_{\left(s,n\right)}^T\right]}^T$where $E_{(k,i)}$ represents the output of each GNN block in the task-specific encoder and $m$ is the total number. $E_{(s,i)}$ represents the output of each GNN block in the shared encoder and $n$ is the total number.

The proposed architecture effectively mitigates the harmful parameter interference between shared and task-specific knowledge by explicitly separating shared and task-specific knowledge. This allows different types of encoding modules to focus on learning different knowledge efficiently without interference. Combined with the advantage of the gating network's dynamic fusion of representations based on inputs, it achieves a more flexible balance between tasks and better handles task conflicts and sample-dependent correlations. Furthermore, the proposed architecture maintains the advantages of the multi-task learning paradigm, including strong generalization ability and data efficiency.

The semantic segmentation head consists of an MLP that classifies each node based on the input feature vectors, thus determining the machining feature type to which each face of the model belongs. The output is a probability vector shaped $N_f\times N_c$ indicating the confidence level of each face's machining feature type, where $N_c$ is the number of machining feature categories. Considering that there is a particular imbalance in the amount of individual machining features in the dataset proposed in this paper, Focal Loss [48] is chosen as the loss function in the training process, which is expressed as:(6)$L_{\mathit{Sem}}=-{\alpha }_t{\left(1-p_t\right)}^{\gamma }\log \left(p_t\right)$where $p_t$ is the probability vector of the semantic segmentation head output. ${\alpha }_t$ and $\gamma$ are the weight coefficients that can be adjusted. Compared with the cross-entropy loss, Focal Loss weights the classification scores of the model output into the loss function, which increases the loss weight of the difficult samples and is suitable for dealing with the classification problem with unbalanced samples. The cross-entropy loss function is used for the MFCAD++ and MFInstSeg datasets, which do not have category imbalance problems.

The instance grouping head consists of an MLP-based face encoder and an inner product decoder. The embedding of the input linear classification layer in the semantic segmentation head is fed into the instance grouping head through an MLP semantic transformation layer to establish a semantic link between the two heads. Thus, the instance grouping head input is a feature vector shaped $192D$ concatenated from the task-specific feature vector and the semantic segmentation head embedding. The face encoder converts the input feature vector into an embedding shaped $N_f\times 64D$. Then the embedding is inner product with itself in the decoder to obtain a symmetric matrix of $N_f\times N_f$, where each position of the matrix represents the probability that the corresponding two faces belong to the same feature instance. The above process can be expressed as follows:(7)$z=MLP\left(x\right)$(8)$S=\sigma \left(z^{T}z\right)$where $x$ is the input feature vector, $\sigma$ is the Sigmod function, and $S$ is the predicted link score. The loss function is the binary cross-entropy loss. It can be expressed as:(9)$L_{\mathit{Ins}}=\frac{1}{N}\sum _{i,j}-A_{\mathit{ij}}\log \left(S_{\mathit{ij}}\right)-\left(1-A_{\mathit{ij}}\right)\log \left(1-S_{\mathit{ij}}\right)$where $A_{\mathit{ij}}$ is 1 when nodes $i$ and $j$ belong to the same instance and 0 otherwise. $S_{\mathit{ij}}$ is the probability of predicting that nodes $i$ and $j$ belong to the same instance. $N$ is the total number of elements involved in the loss calculation. Because the adjacency matrix $A$ is sparse, the number of positive and negative samples is unbalanced which makes the model difficult to train. Therefore, the instance grouping head is trained using the technique of negative sampling. For each model, we sample the same number of negative labels as the number of positive labels for loss calculation and backpropagation to improve the network's training stability.

Based on probabilistic embedding [49], the relation prediction head encodes the constituent faces of each feature instance into a multivariate Gaussian distribution. Compared with encoding each face as an independent vector, the multivariate distribution can more accurately capture the relationship between feature instances and the internal structure to more naturally model the topological relationship between features. Its input comprises the task-specific feature vector and semantic embedding vectors from the previous two tasks. Two MLPs are used to learn the mean and standard deviation of the face distribution and encode them as $16D$ vectors, respectively. The hidden vector of the face embedding is obtained by sampling the encoded multivariate Gaussian distribution of each face. Then, through the inner product decoder and the linear classifier, whether there is a topological relationship between the feature instances and the type of relationship is obtained. The above process can be expressed as follows:(10)$\mu =MLP\left(x\right)$(11)$ϛ=MLP\left(x\right)$(12)$z=\mu +∊\times ϛ$(13)$S=\sigma \left(z^{T}z\right)$where $x$ is the input feature vector, $\sigma$ is the Softmax function, $∊$ is a random variable that conforms to the standard normal distribution, and $S$ is the predicted link score shaped $N_f\times N_f\times N_{\mathit{rel}}$. $N_{\mathit{rel}}$ is the number of topological relation types. $\mu$ and $ϛ$ are the mean and standard deviation of each face probability embedding, respectively, with dimension $N_f\times 16D$. The loss function of the relation prediction head consists of two parts: the cross-entropy loss function measures the deviation of the prediction result from the true label, and the KL divergence loss function measures how close the distribution encoded by each face is to the multivariate Gaussian distribution. It can be expressed as:(14)$L_{\mathit{Rel}}=CE+KL$(15)$\mathit{CE}=-\frac{1}{N_{\mathit{fe}}^2}\sum _{i,j}\sum _{c=1}^M{y}_{\mathit{ijc}}\log \left(p_{\mathit{ijc}}\right)$(16)$\mathit{KL}=-\frac{1}{2}Mean\left\{Sum\left[1+2\log \left(ϛ\right)-{\mu }^2-{ϛ}^2\right]\right\}$where $N_{\mathit{fe}}$ is the number of feature instances included in the model, $p_{\mathit{ijc}}$ is the predicted probability that the relationship category between instances $i$ and $j$ is $c$, and $M$ is the total number of relationship categories. $y_{\mathit{ijc}}$ is a sign function equal to 1 when the relationship category between instances $i$ and $j$ is $c$ and 0 for the rest. $\mathit{Sum}$ function in the KL loss computation sums the feature dimensions of the probabilistic embeddings, and $\mathit{Mean}$ function averages the embeddings over the $N_f$ faces.

The losses of the three task heads need to be fused and backpropagated to update and optimize the parameters of MFTReNet. The commonly used loss combination function is the arithmetic average. However, experiments found that the convergence speeds of the three task losses of MFTReNet differ and cause an imbalance in training, which in turn affects the overall performance of the model. Although this phenomenon can be mitigated to a certain extent by weighted averaging of the losses, the optimal weights are difficult to be obtained manually. Therefore, this paper uses Geometric Loss [44] to fuse the losses of the three tasks by using the geometric average instead of the arithmetic average to learn the three tasks equally. The total loss is formulated as follows:(17)$L_{\mathit{Total}}=\sqrt[3]{L_{\mathit{Seg}}\times L_{\mathit{Ins}}\times L_{\mathit{Rel}}}$