Manufacturing Process Classification Based on Distance Rotationally Invariant Convolutions

With the quick growth of web-based and cloud-based technologies, cyber manufacturing has developed as an area to improve management and promote productivity, especially for manufacturing centers that are geographically distributed [1]. Within cyber manufacturing, manufacturing process identification and manufacturing resource selection are two core problems. Because of the on-demand requirements, experts in manufacturing engineering make decisions for these two problems according to their experience and knowledge. However, the experience of engineers is limited, and we desire to automate the process. Thus, the automation of manufacturing process selection is desired, especially in the early stage of the design process before adding shape details to part models.

Plenty of previous research focused on manufacturing process selection and capability modeling; however, no general solutions have been achieved. There were mainly two directions for process selection. The first category of methods extracted features that were unique for manufacturing processes and chose several suitable processes based on these features [2]. The other category constructed an information model which aided software or humans in process selection [3]. The recent development of machine learning provided new methodologies for feature extraction, i.e., the first category of methods. Zhao et al. applied the decision tree to process selection, though their performance is limited by the choice of method and a small dataset [4]. In our previous work, a larger dataset was constructed and a convolution neural network (CNN) was applied to the heat kernel signature (HKS) shape descriptor extracted from triangle meshes [5]. The work achieved good accuracy and “rotation invariance,” i.e., rotating the model about any axis by any degree will not impact the classification result.

There are multiple methods for representations of computer-aided design (CAD) models. Many object recognition methods, as well as our previous process classification research, utilized triangle meshes for representation. Many other representations for CAD models have been used for object recognition and classification. Multiple-view images can represent a 3D model by projecting the model onto 2D planes at different angles. However, projection angles need to be chosen, and reconstruction techniques are required to build the 3D model from 2D projections. Another frequently utilized format was volumetric representation (i.e., voxels), but this approach can be computationally expensive. Signed distance field (SDF) is a frequent alternative representation to voxels [6]. Triangle mesh boundary representations can be regarded as a graph-based network that can be input to shape descriptors for feature extraction before being fed into neural networks [5,7,8]. In comparison, 3D point clouds have been used frequently in object recognition tasks and lead to low computation costs.

Deep learning has achieved great improvements in the past decade, especially in image recognition [9]. CNN technology played an important role in the breakthrough of deep learning in the image, video, and other domains where data are naturally represented as 2D or 3D matrices. First of all, CNNs had good performance in feature extraction from matrix representations, which would be essential for manufacturing process selection. Additionally, graphical processing unit (GPU) technology developed rapidly in the past decade, and it strongly accelerated parallel computation. Since parallel computation algorithms were available for CNN, its application was computationally efficient.

Recent developments on point clouds with deep learning provided an alternative way to select the manufacturing process. This direction will be our focus in the paper. Point clouds are a set of points sampled from the CAD model surface without mesh information. Changing the sequence of points in the point cloud does not change the model. That is, point clouds are “permutation invariant.” To achieve this property, a previous work applied a symmetric function like max-pooling within the neural network [10]. However, this method only extracted global information, while local information was omitted. To remedy this problem, convolution on the K-nearest neighborhood (KNN) was used to extract local information of a sub-point cloud to improve classification accuracy [11]. In our work, a similar technique was applied to extract global and local information. In addition, translation invariance, rotation invariance, and scale invariance were three important properties for manufacturing process classification. When the input to the neural network was the relative coordinate x_i − x_j rather than the absolute coordinate x_i, the goal of translation invariance was realized naturally. When some rotational-invariant features like distances between adjacent points within the point cloud were utilized as input, the trained neural network would become rotation invariant. If the dataset was scaled before training, the problem of scale invariance was resolved. Combining these techniques, a rotationally invariant convolution method was developed for object classification [11]. In this work, we developed a different choice of shape descriptors that we call the distance Rotation Invariant Convolution (dRIConv) and demonstrate that it achieves better accuracy than our previous research. The contributions of this paper include a distance-based feature extraction mechanism to extract rotational-invariant features, demonstration that these features and a CNN enable excellent manufacturing process classification capabilities, and visualization methods to understand internal information within the neural network.

In practice, manufacturing process classification, given a part design, involves the consideration of many types of information beyond part shape. Materials, part size, tolerances, production volume, etc., can be critical considerations. However, we choose to focus only on the part shape to investigate the capability of a machine learning model to learn shape features that characterize shapes producible by specific manufacturing processes. We realize that definitive process selection depends on non-geometric information, but our objective here is more modest, that of identifying processes that can in principle produce part shapes. Note that this work assumes a single manufacturing process for part fabrication and excludes the use of a manufacturing process chain.

The remainder of this paper is organized as follows: Sec. 2 will review related papers including feature identification in manufacturing and deep learning on point clouds. In Sec. 3, the dataset for manufacturing and the transformation to point cloud will be introduced. In Sec. 4, the detail of the neural network, i.e., dRIConv, will be explained. In Sec. 5, we will show the results of the manufacturing process classification. To help understand the internal structure of the neural network, some visualization tools are introduced. Finally, to test the robustness of the trained neural network, some ambiguous examples are included.

Much previous research focused on addressing manufacturing process selection and capability modeling. They included one instructional handbook [12] and several multi-attribute decision-making methods [13]. In this part, we review feature extraction for manufacturing process classification, deep learning on point clouds, and shape identification using CNNs.

The earliest work on feature identification for manufacturing was in the 1980s. Researchers desired to recognize manufacturing features from design models directly and utilize them to automate the process from design to manufacturing. Based on this goal, several categories of methods were created, including graph-based, volumetric-based, and hint-based methods [14]. Graph-based methods realized the transformation from boundary representation to a graph on some feature extraction techniques. Volumetric methods decomposed a solid model into simpler shapes, such that the simplest shapes could be interpreted as volumes that could be machined. Hint-based methods could handle interacting features, such as intersecting slots. Later research showed that a combination of these methods could achieve even better results [15]. In addition, material information was included in the feature extraction process by [16].

Another line of research focused on designing shape descriptors that characterized local or global part shapes and were invariant to translation, rotation, and scale. Within them, curvature was a simple yet useful example. Later, more complex shape descriptors were developed by researchers to facilitate the classification process. Some research [17] regarded the model signature as a distribution, and the matching process between features was transformed into a probability calculation process. With the development of techniques in machine learning, their application for feature extraction from local shapes was investigated. A clustering process on features was proposed, which was unsupervised learning realized by the KNN [18]. A support vector machine (SVM) was utilized to measure derived curvature, as the shape descriptor [19]. Zhao et al. [4] utilized the decision tree for feature extraction. Their dataset included 81 parts, and their test accuracy was 83%. In our previous work, a larger dataset was generated composed of 12,463 examples. We applied a CNN on HKS shape descriptors derived from triangle meshes, and the best test performance was 97% accuracy. However, we believed a better accuracy could be achieved by incorporating the latest development of CNN and point cloud-based shape descriptors.

The field of computer vision has made great breakthroughs in object recognition, and much research was based on CNNs. There were two main approaches for shape identification: 3D-based and image-based [20]. For the manufacturing process classification problem, 3D-based approaches should be more appropriate. Research started by developing a local shape descriptor to describe the local shape. Then, a global shape descriptor came into being which represented the distribution of the local shape descriptor over the part. Finally, CNNs were applied that used global and local shape descriptors for classification. In the literature, CNNs could recognize machining features, although this was limited to only simple features like pockets and holes from voxelized CAD models [21]. In the computer graphics field, Wu et al. [22] applied CNNs to segment point clouds, which was equivalent to constructing models as a composition of features, as one example of research progress.

There were multiple possible representations for 3D models using CNNs including volumetric representation, triangle mesh, and point cloud. [9,10,23,24]. Deep learning on volumetric representations could be straightforwardly transformed from convolution on 2D images. VoxelNet first voxelized 3D data into 3D grids and then applied a 3D CNN to the volumetric representation for shape classification [25]. Nevertheless, the huge computation cost limited the resolution of voxel size. Deep learning could not be applied to triangle mesh directly since the number of vertices differed in different models, but CNNs require consistent input sizes. Thus, a simplification technique (e.g., sampling or remeshing) was required to reduce the number of vertices to be the same value for all models [26]. In comparison, point cloud representations recently achieved great success in deep learning for their computation efficiency without any pretreatment. Thus, the point cloud seemed to be more suitable for our work.

A point cloud is a sequence of unordered points that is sampled from a part surface. Changing the sequence of points does not change the point cloud, which means that point clouds are permutation invariant, a desirable property. In early research on object classification, PointNet combined a Multi-Layer Perceptron (MLP) with symmetric aggregation functions like max-pooling [10]. However, PointNet only captured global information, while local information was omitted. Qi et al. [10] developed a hierarchical neural network (PointNet++) that applied PointNet recursively on a partitioning of the input point set, where the partition was defined as a neighborhood ball by KNN. As a result, both local and global information within the point cloud could be extracted.

Both PointNet and PointNet++ applied MLP for feature extraction, while some latest papers applied CNNs rather than MLPs to extract features. To maintain the property of permutation invariance, a CNN was applied to local points gathered by KNN. These methods could be roughly divided into two categories: continuous and discrete CNN methods. In continuous CNN, for a center point, the weights for neighboring points around the center point were related to the coordinate difference between neighboring points and the center point. For discrete CNN, the weights for neighboring points were related to the distances to the center point. Within continuous CNN methods, RSCNN was a pioneering work. It applied convolution to points within a neighborhood ball, and a channel-rising technique was used in the feature extraction process [27]. In DenseNet, compared with RSCNN, the convolution was simplified as a Single Layer Perceptron, while features from multiple layers were concatenated together as a larger input for classification [28]. In SpiderCNN, the authors designed a filter as a product of a simple step function and a Taylor polynomial within the CNN [29]. For the discrete CNN method, PointCNN applied convolution on features that were obtained by transforming point coordinates to a potentially canonical order [30]. RIConv achieved the goal of rotation invariance by utilizing some rotation invariant features like distances and angles to train the neural network [11]. Our work in this paper was based on a modification of RIConv, i.e., dRIConv.

The last category of methods is graph-based. A graph-based method considered each point in a point cloud as a vertex of a graph and generated directed edges for the graph based on the neighbors of each point. These methods could be performed either in spatial or in spectral domains. In the spatial domain, the procedure was quite close to the convolution method, while in the spectral domain, convolution was defined as spectral filtering, which was implemented as the multiplication of signals on the graph with eigenvectors of the graph Laplacian matrix. DGCNN was one representative work in the spatial domain [31]. It constructed the graph using KNN in the feature space. As a result, two points could be adjacent in feature space, even if they were far away from each other in Euclidean space. LDGCNN improved DGCNN by linking the hierarchical features from different layers [32]. LocalSpecGCN belonged to the spectral domain method and achieved similar performances as in the spatial domain [33].

In this section, the part dataset is introduced with a focus on the classification of different manufacturing methodologies. In addition, the transformation from the CAD model to the point cloud is discussed. Note that more details can be found in our earlier work [5].

Our previous research introduced a dataset for manufacturing, which included four manufacturing processes: laser-cutting, three-axis machining, turning, and injection molding. The dataset consisted of two components. The first component came from industrial parts, other real parts, and online part repositories. It contained 125 laser-cut, 155 three-axis machined, 114 turned, and 69 injection-molded parts. The other component contained synthetic models; 3000 parts for each of the four manufacturing processes were synthesized to complete the dataset. In total, there were 12,463 examples in the dataset.

For laser-cut parts, the stock sheet material is usually positioned horizontally, i.e., in the X–Y plane. To cut the part from the sheet, the laser impinges vertically along the z-axis. Two common features exist for laser-cut parts. The first feature of laser-cut parts is that the thickness of parts will be thin along the z-axis because it is tough for the laser to cut thick sheet materials. Another feature is that laser-cut models include only through holes because cut thickness cannot be controlled.

Three-axis machining can control the depth to remove materials. As a result, both through holes and partial holes could be manufactured. Hole shapes can be either circular or rectilinear such as slots and grooves. Additionally, three-axis machining parts in the dataset include material removal in lateral surfaces, in addition to the top surface. This is realized by multiple setups, i.e., the feedstock needs to be rotated and fixtured before removing material from lateral surfaces. However, the dataset assumes that at most two setups are available to manufacture a part, which is equivalent to at most one repositioning.

Axisymmetric parts are usually manufactured by a turning process on a lathe. In the dataset, the model of turning generates only axisymmetric part shapes and avoids parts that have little non-axisymmetric features like shafts with keyways.

Injection molding parts are manufactured by injecting molten materials into a predesigned mold. After the material cools down, the part can be ejected from the mold. As a result, parts typically consist of thin walls and features with thin cross sections. The dataset limits the parts to have at most two small undercuts.

Scripts were developed in matlab to generate stereolithography (STL) files according to the process characteristics mentioned above using randomly selected numbers of features and feature shapes, sizes, and orientations.

Point cloud representations were generated by uniformly sampling across the external part surfaces from their STL files. The number of points in each point cloud model was chosen to be 1024 to facilitate deep learning. Only the spatial coordinates were included for each point, i.e., information like the normal vector for each point was discarded. Figure 1 shows the point cloud sampling results for examples of parts from each manufacturing process.

In this section, the dRIConv shape descriptor and process classification method will be presented. Foundational technologies including CNNs and the PointNet method will be described first. PointNet, based on CNNs, successfully dealt with permutation invariance in point clouds but utilized only global information. To incorporate more local information and deal with rotation invariance, RIConv was developed. In this research, we propose dRIConv that explicitly combines local and global information and has some simplifications compared to RIConv.

The pooling layer is frequently utilized to reduce redundant information. In an image, a lot of redundant information usually exists. Max-pooling has been proposed to reduce that redundant information, which searches local maximum values as outputs. For example, a 2 × 2 max-pooling operation will search for the largest value from a 2 × 2 submatrix as output. As a result, 75% (3/4) of information will be discarded after a 2 × 2 max-pooling operation.

The combination of convolution layers and pooling layers is frequently utilized for feature extraction, while fully connected layers perform classification based on previously extracted features.

Combining convolutional, pooling, and fully connected layers, a modern CNN is constructed. The difference between predictions (CNN outputs) and labels that represent the ground truth is called the loss function. By minimizing the loss function through the gradient descent method, the neural network is “trained.” After the training process is finished, the neural network is frozen and used for testing. More details of CNN can be found in Ref. [5] and its references.

The success of CNNs relies heavily on the internal spatial structure of images, where each internal pixel is surrounded by four pixels. Thus, convolution, more specifically weight sharing, becomes possible. However, there does not exist such a structure within the point cloud, and as a result, directly applying convolution to a point cloud is not possible. Additionally, permutation invariance is desired.

To solve these two problems, the PointNet method proposed a combination of an MLP and a symmetric function: max-pooling. MLP was applied for feature extraction. In the process, information from different points would not be mixed. Max-pooling had the property of permutation invariance because after shuffling the point cloud, the maximum value remained the same: $max(x1,x2,⋯,xn)=max(p(x1,x2,⋯,xn))$ when p(·) was a function for shuffling. With the combination of MLP and max-pooling, the resulting neural network, shown in Fig. 2, would become permutation invariant.

PointNet had the drawback of only extracting global information in max-pooling while local information between adjacent points was omitted. To extract local information, the RIConv method combined farthest point sampling (FPS) with KNN. FPS could search for points that were most distinctive in Euclidean space [7]. Given a center point, KNN gathered K-nearest neighborhood points that were closest to the center point in the spatial domain.

A MLP was utilized to extract information from the rotation invariant features. However, these features, i.e., d₀, d₁, θ₀, θ₁, contain only local information as they are local distances and angles, respectively. A combination of binning and projection was utilized in RIConv for combining global information with local information, and we believe that this combination process was implicit and computationally expensive. To solve this problem, we proposed a different set of rotationally invariant features, a set of local and global distances, and call the new shape descriptor dRIConv. As a result, dRIConv includes both global and local distances and enables some simplifications compared to RIConv.

The structure of dRIConv is shown in Fig. 4. To begin with, FPS and KNN were applied as before. Then, local FPS, local mean, global FPS, and global mean could be obtained. The global mean refers to the mean of all the points within the point cloud, and the global FPS refers to the first point for the FPS, i.e., the farthest point to other points within the point cloud. In the first iteration of FPS, the global FPS is selected randomly as shown in a black circle in Step 1 of Fig. 4. The other FPS points are marked as red plus sign Fig. 4, and each of these points will be a local FPS. For each local FPS, K-nearest neighborhood will be found. For each local FPS, the local FPS and its corresponding KNN form a local point cloud. The mean of the local point cloud is called the local mean. For any point in KNN, seven distances were calculated, which included three local distances (Dist1–3) which connect points within the local point cloud, three global distances (Dist4, 6, 7) which connect points within the global point cloud, and one distance connecting the local region to the global region (Dist5) as shown in Step 2. The seven calculated distances formed the input to the neural network. The final step applied MLP to lift the number of channels and max-pooling to reduce dimensions and extract features, which were the same as in RIConv.

dRIConv, as a building block, was further utilized to form the whole neural network structure. Three layers of dRIConv were used to extract features, and three fully connected layers were applied for classification as shown in Fig. 5. Hyperparameters of each layer were also presented: K is the number of KNN, P is the number of points in FPS, and C is the number of channels after each dRIConv layer. FC(512, 256, 4) represents that the numbers of neurons in three fully connected layers are 512 and 256, respectively. The choice of parameters including K, P, C, and FC follows the previous paper of RIConv. Our tests show that when K, P, and C have reasonable values; i.e., not too small, good classification accuracy can be achieved.

In this section, we present results from three numerical studies to demonstrate the capabilities and benefits of dRIConv. The first study classifies four manufacturing processes. By comparing test accuracy, dRIConv excelled compared to all previous methods. The second study provides two visualization tools for intermediate layers of dRIConv. Last, we utilize some ambiguous examples to prove the robustness of dRIConv.

The first task focused on manufacturing process classification. We used the part dataset described in Sec. 3 for training and testing. We applied 80% of the dataset, i.e., 9970 parts, for training and 20%, i.e., 2293 parts, for testing.

Manufacturing process classification results are shown in Table 1 comparing our dRIConv method (last row), our previous method utilizing HKS (CNN + HKS), and four other object classification methods from the literature. For the column headings, Z refers to rotating part models around the z-axis by an arbitrary amount; SO3 refers to rotating around an arbitrary axis in 3D space. The notation Z/Z denotes that both training and testing utilized Z-axis rotations, while SO3/SO3 denotes training and testing with rotations about arbitrary axes.

For comparison, we incorporated some previous works to show the strength of our method. RSCNN, DensePoint, SpiderCNN, DGCNN, and dRIConv utilized point clouds as inputs, while CNN + HKS took triangle meshes as input. All the methods had good performances when training and testing utilized the same rotation approaches. However, only CNN + HKS, RIConv, and dRIConv achieved good performance when rotational invariance was tested as shown in the Z/SO3 column. The reasons for deterioration of previous works including RSCNN, DensePoint, SpiderCNN, and DGCNN in the test of Z/SO3 lie in that these previous neural networks are not designed for rotation invariant problems. As a result, though they can achieve good performance in the tests of Z/Z and SO3/SO3, they cannot handle Z/SO3 as well as CNN + HKS, RIConv, and dRIConv. In comparison, HKS + CNN and dRIConv utilize HKS and distances as inputs to neural networks, which are rotation invariant, and outperformed the other methods. Furthermore, the dRIConv method outperformed the CNN + HKS method from the perspective of classification accuracy. RIConv and dRIConv have similar performances on the dataset composed of industrial parts and generated parts. However, dRIConv outperformed RIConv by 1% when only training and testing on industrial parts as shown in Table 2. The authors believe that the generated data enrich the training dataset and make the classification task simpler. The outperformance of dRIConv over RIConv on the dataset that contains only industrial parts proves the advantages of dRIConv. dRIConv has a more clear structure for information extraction, is significantly more computationally efficient, and should be capable of handling more difficult classification problems compared with RIConv.

Two additional points of comparison will be made between the dRIConv and HKS + CNN methods. First, HKS is more computationally demanding than the shape descriptors (the seven distances) required for dRIConv. Second, CNN + HKS requires a high-quality triangle mesh so that the obtained HKS is accurate. Obtaining a high-quality mesh is more difficult than obtaining a high-quality point cloud. The time for generating a high-quality mesh from the original mesh is around 5 min, and it takes around 10 s for computing HKS from the triangle mesh. In comparison, generating point clouds and calculating seven distances can be accomplished in 10 ms. But the training and testing time for HKS + CNN and dRIConv are quite short when input data are available. Thus, the dRIConv method appears to be the best found so far from both accuracy and computational perspectives.

Next, we tested the performance of the neural network if the input contained less global or local information, i.e., fewer distances as input. Table 3 shows the mean classification accuracy of using different distances as input on only industrial parts. All these different combinations achieved good classification accuracy in the end; i.e., only a 0.2% difference between the neural network trained with three distances and with seven distances. However, that does not mean that distances 6 and 7 do not contain valuable information. Instead, there were some differences in the training process. When training with three distances or five distances, more fluctuations were evident compared with training or testing with seven distances as shown in Fig. 6. Though more distances as input would not improve accuracy, it would stabilize the training process; i.e., the loss function would decrease gradually. This was the core reason why we chose the neural network with seven distances as input.

The second study desired to visualize internal information learned within dRIConv. For simplification, the number of dRIConv layers was reduced to two, where P = 128, C = 256, and K = 32 in the first layer and P = 64, C = 512, and K = 16 in the second layer (see Sec. 4.3 for parameter definitions). In layer 1, the initial number of points was 1024, and 128 of them were chosen using FPS. For each of these points, KNN was applied (K = 32). The information of these 128 FPS and corresponding KNN points were extracted through MLP and max-pooling. For layer 2, the initial number of points was 128, and 64 of them were chosen using FPS, and for each of the 64 points, KNN was applied (K = 16). The information of these 64 FPS and corresponding KNN points were again extracted through MLP and max-pooling. Some visualization tools have been proposed by previous researchers [34]. Two visualization methods are provided in this section that are extensions of those from the literature. The first visualization method was modified from PointNet on critical points visualization [34], and the other critical point visualization method in 3D domain was proposed by the authors. The first method shows a comparison between points selected or not selected by FPS as shown in Fig. 7. In comparison, the second visualization tool shows the most representative points within FPS for different channels in Fig. 9.

The first visualization method mimicked the procedure of PointNet. In PointNet, critical points were defined as points whose information existed after max-pooling. In PointNet, MLP was utilized for feature extraction. As a result, information for one point would not be entangled with other points. In comparison, dRIConv applied CNN for feature extraction, and the information of a point will be entangled with its KNN. To solve this problem, in dRIConv, points that existed after max-pooling or FPS were considered critical points. We named points after FPS, i.e., 128 for the first layer and 64 for the second layer, as “L1 FPS” and “L2 FPS” separately. Following the same rule, points that existed after max-pooling in layers 1 and 2 were defined as “L1 max-pooling” and “L2 max-pooling,” respectively. The importance of points was ranked from high to low as “L2 FPS,” “L2 max-pooling,” “L1 FPS,” “L1 max-pooling,” and “All points.” This was because “L2 FPS” ⊆ “L2 max-pooling” ⊆ “L1 FPS” ⊆ “L1 max-pooling” ⊆ “All points.” Examples of different manufacturing methods are shown in Fig. 7 with such a visualization tool. As a result, from these figures, points that were significant for classification were identified. The result shows that more critical points, i.e., “L2 FPS” and “L2 max-pooling,” form the coarse shape of a part. These important points are uniformly distributed. In comparison, the fine shape is composed of less critical points like “All points.” Separate figures for these sets of points are provided in Fig. 8 for a three-axis machining example. In this separation process, “L2 max-pooling” is empty as there is no point that belongs to this set. From “L2 FPS” and “L1 FPS,” the rough shape of the three-axis machining example can be shown. In comparison, more detailed shapes can be shown in “L1 max-pooling.”

Another type of visualization combined point coordinates with point features. Previous visualization methods directly visualize features in each channel as each internal feature has a matrix shape. However, it is difficult to plot features of point clouds in the 2D domain directly as the features correspond to different points and the points cannot be plotted in the 2D domain. To solve this problem, this technique combined point coordinates for a 3D scatter plot with features for the colors of points. The first type of visualization focuses on identifying the importance of different points. In contrast, the second visualization desires to investigate what features each channel within the MLP has learned in dRIConv. After simplification, the numbers of channels were f = 256 in the first layer and f = 512 in the second layer. Thus, 256 different figures from the first layer and 512 different figures from the second layer could be plotted. Four representative figures were then selected from the 256 figures in layer 1 for visualization as shown in Fig. 9. The yellow points represent points with larger (important) features, while points in purple are less informative points. The importance of different points depends on the value of each point. After max-pooling, within each channel, i.e., each figure, there exists a value for each point within the point cloud. The points with larger values are considered important points. In each channel, by normalizing the largest value within the channel, all values lie in the interval [0, 1]. Using “colormap” and “colorbar” in Matlab, the most important points, whose values are close to 1, are plotted in yellow, while less important points, whose values are close to 0, are plotted in blue. In the four figures, different regions (top, middle, and bottom) are stressed separately. For example, in subfigure (a), all points with large features (in yellow) lay in the middle region. This means that this channel focuses on learning features from the middle region of the point cloud. Additionally, useful information is embedded in the local structure. In the turning part, points, which were symmetrical around the center axis, were usually stressed together. This phenomenon indicates that the neural network judges whether this part can be manufactured by turning through searching for symmetric points. This is consistent with human experience to classify manufacturing processes. In a similar idea, Figs. 9(b)–(d) stress top, middle, and bottom regions of the turning part. In the end, by combining useful information across different layers and across different regions of a part, the constructed neural network can achieve accurate results in the classification problem.

The last task tested different ambiguous examples to show the robustness of our trained neural network. First, we tested examples that lie in the boundary between two manufacturing methods: laser-cut and three-axis machining (Fig. 10). Next, two tests were performed on parts that could not be manufactured by any of the four manufacturing methods. In the first of these, we tested parts that could be machined but did not satisfy our limitation to two setups for three-axis machining. In addition, a test on hybrid manufacturing was conducted that required a combination of three-axis machining and turning. Lastly, we tested examples with closely spaced features that could be manufactured by more than one of our chosen processes.

In this section, we desired to test the performance of dRIConv on some examples if the boundary of two manufacturing methods, laser-cutting and three-axis machining. In the left subfigure, the example is composed of a thin wall and two through holes. We believed dRIConv would identify it as laser-cut. In the right subfigures, partial regions on the top surface were lifted to form blocks on the thin wall. In our opinion, the model would be identified as three-axis machining by dRIConv. However, the middle example might bring more problems for dRIConv since the lift on the top surface was very small.

Models with different lifted height ratios were created, where the ratio was determined by dividing lifted height by base height. dRIConv returned the probabilities of being laser-cut or three-axis machined, shown in Table 4. The result was consistent with our assumptions. When the height lifted was small, the neural network identified the model as laser-cut. However, as the height grew, the probability of three-axis machining finally overshadowed that of laser-cut. In particular, there was a sudden change when the lifted height ratio changed from 2.5 to 5. Thus, we believed that dRIConv had a threshold for lifted height ratio to separate laser-cut and three-axis machining.

In the constructed dataset, we only included examples that could be manufactured by three-axis machining with at most two setups. Here, we desired to experiment with parts requiring more than two setups. An example is shown on the left side of Fig. 11, and it could be manufactured by three-axis machining with two setups (one repositioning). This part was supposed to be identified as three-axis machining by dRIConv. On the right side of Fig. 11, another example was shown that required three setups. We assumed that it would also be identified as three-axis machining by dRIConv, but at a lower level of confidence.

We designed examples that required different numbers of setups for three-axis machining. The number of resettings varied from one to four (i.e., the number of setups was 2–5). The performance of dRIConv on these examples is reported in Table 5. All of these examples were identified as three-axis machining. This was also consistent with our expectation: these models, though they could not be manufactured by three-axis machining by one setup, were more similar to three-axis machining compared with the other manufacturing methods (laser-cut, turning, and injection molding). We were surprised that the level of confidence remained very high as the number of resettings increased.

Some parts can only be manufactured by a combination of different manufacturing methods, i.e., hybrid manufacturing. We desired to test the performance of dRIConv on these types of parts. Two examples are shown in Fig. 12, and both were manufactured by a combination of three-axis machining and turning. The left model was dominated by features of three-axis machining, and we believed that it would be identified as three-axis machining. In comparison, the right example had more features of turning, and it was assumed to be recognized as turning.

As shown in Table 6, dRIConv identified the left example as three-axis machining, while the right example was identified as turning. This was consistent with our expectations. Last but not least, dRIConv recognized the right example with a 22% probability of injection molding. After a careful check, we realized that the two examples shown in Fig. 12 could be manufactured by injection molding. This further proves the robustness of dRIConv.

In dRIConv, KNN was utilized to extract local structure information. However, the Euclidean distance between two points might be small even if they belonged to different features. For example, points from the top and bottom surfaces of a laser-cut model had short Euclidean distances, and they were utilized for local information extraction. However, misleading shape characteristics might be derived from these cases where clustered points come from multiple surfaces and features, which may negatively impact classification accuracy.

To test this assumption, some examples with closely spaced features were created as shown in Fig. 13. We utilized two different profiles: rectangle (top) and circle (bottom). For features, we included slots and ribs with different height ratios. The ratio was defined as the height of the rib divided by the width of the profile.

On the left two subfigures of Fig. 13, the models had closely spaced slots, and these models were most suitable for laser-cutting, although injection molding is also possible. The middle two figures and the right two figures all had closely spaced ribs, and these models were most suitable for either three-axis machining or injection molding. In comparison, the right two figures had longer ribs than that of the middle two figures. The naming convention Rib(#) indicates that rib features were extruded using a height ratio of #. For example, Rib(3) represents a part with ribs where rib height divided by its base was 3. In Fig. 13, the middle two examples belonged to Rib(1) while the right two examples belonged to Rib(5).

Classification results are shown in Table 7. These results had similar trends as that of “Examples on the boundary of two manufacturing methods” in Table 4. When the height ratio was small, the part was more likely to be classified as laser-cut, while when the rib height grew, the probability increased for three-axis machining or injection molding. From the results shown in Table 7, closely spaced features did not seem to have important effects on the classification results of dRIConv. This meant that the efficacy of the dRIConv shape descriptor and classification method was not sensitive to whether or not local point neighborhoods came from multiple surfaces or features.

Two possible explanations can be provided. The first one is that dRIConv might have the capability to get rid of the mixed information from two features. For example, max-pooling eliminated a lot of redundant information. The second explanation is that mixed information was unimportant. Even if information from points from two surfaces was combined into one local neighborhood, this would not change the required manufacturing process. For example, if the neural network misidentified two adjacent slots as one larger slot, it would still be suitable for laser-cutting.

One last interesting phenomenon was that Rib(5) was identified as three-axis machining for the rectangle profile, while it was determined to be injection molding for the circle profile. These Rib(5) models could be manufactured by either process. This phenomenon was probably caused by the part shapes generated in the training dataset. Rectangular shapes may be more predominant in the machined parts, while injection-molded examples may have contained more round shapes than the machined examples.

In Sec. 5, three different experiments were conducted. The first experiment compared manufacturing process classification accuracy between five different methods using point clouds and one method using triangle meshes. An ablation study on the input information was also conducted. For the second experiment, two different visualization tools were provided to interpret what the neural network has learned in the intermediate layers. The last experiment tested ambiguous examples that might cause problems for dRIConv.

The first experiment proved the strength of dRIConv on rotation invariance and accuracy. Existing methods on point clouds did not consider the problem of rotation invariance. Their performance on Z/SO3 tests seriously deteriorated, compared to the cases that used the same types of rotations for training and testing (i.e., Z/Z and SO3/SO3). The HKS + CNN method utilized the heat kernel signature shape descriptors, which was rotation invariant. However, pre-calculation of the HKS was needed, and it was computationally expensive. In addition, its classification accuracy was also somewhat lower than that of dRIConv. The ablation study compared the performance of dRIConv with the different input information. The result showed that more local and global information smoothed the training process; i.e., convergence of test accuracy was smooth, though this did not improve classification accuracy significantly.

In the second experiment, two different visualization tools were designed to interpret the mechanism of the neural network. From these interpretations, we believed that the trained neural network has successfully learned useful feature extraction methods. From the first interpretation plot, some critical points for classification were discovered. These critical points were usually uniformly distributed; i.e., information at different regions was considered for classification. However, in some local regions where the point density was relatively high, most points would be identified as unimportant while only one or two points would be recognized as important. This procedure was believed to remove redundant information. From the second visualization tool, we discovered that partial regions, such as top, middle, or bottom, of a model, were usually stressed within different channels. There were multiple channels within a layer, and when information from different channels was combined, accurate classification results could be accomplished. This procedure was interpreted as resembling human reasoning, i.e., learning local features of a model before being combined as global features.

The third experiment tested ambiguous examples that were on the boundary of two manufacturing processes, could not be manufactured by any one of our chosen processes, or had closely spaced features. When testing these ambiguous examples, the performance of the dRIConv method deteriorated in an expected manner. Results showed that the dRIConv method was classified based on dominant features, and these features seemed to influence the final process classification. In the first test, dRIConv seemed to apply a threshold for section height; after the height reached a certain value, the part was classified as three-axis machining, while below the threshold the classification was laser-cut. In the second test, the confidence of three-axis machining was maintained as the number of setups increased. In the third test, dRIConv seemed to determine the manufacturing process based on dominant features, either three-axis machining or turning, of the model. Finally, the closely spaced features test reinforced results from the previous tests. The dominant geometric features in parts determine the manufacturing process classification, as expected. Also, the dRIConv shape descriptor is robust to the sources of local point neighborhoods, meaning that these points can come from multiple surfaces and features.

With the rapid development of technologies related to cyber manufacturing, manufacturing process classification automation becomes more and more important. In this paper, we focused on identifying suitable manufacturing processes given a part design. Our previous research generated a large dataset for manufacturing process classification and achieved accurate rotation invariant classification results through a combination of HKS and CNN. However, our previous work relied on the triangle mesh, and thus, mesh quality would impact classification results. Besides, the calculation of HKS was required in advance, which was time-consuming. In this paper, another method based on CNN and point clouds was proposed, called dRIConv, that realized rotation invariance by extracting rotation invariant features like distances and utilizing them as input to train the neural network. There were several benefits of the new method. First, it achieved higher accuracy in the classification test and maintained rotation invariance. Then, point clouds for dRIConv did not require pretreatment like HKS calculations, which was much more efficient. Third, a uniform point cloud was easier to obtain in comparison with a high-quality triangle mesh.

In this paper, two visualization tools were designed to find critical points for classification and to interpret intermediate layers of the neural network. Critical points were usually uniformly distributed. In comparison, noncritical points usually formed fine-structured shapes. Additionally, we discovered that the neural network learned to classify in a similar workflow as humans. Local features were first extracted before being combined for classification.

Additionally, some ambiguous data were tested, and the performance of dRIConv deteriorated in an expected manner. Results showed that the dRIConv method classified based on dominant features: these features were extracted, combined, and applied for classification. The test on ambiguous examples showed the robustness of dRIConv.

Based on this research, the following conclusions can be drawn:

• A deep neural network is a promising tool for manufacturing process classification.

• The proper choice of distance yields rotation invariant features that are sufficient for manufacturing process classification.

• The neural network’s learning process is accomplished by integrating features from different channels, where different regions of the model are stressed.

Future research should proceed in several directions. More subdivided datasets are required to accurately classify parts that lie on the boundary between two manufacturing processes. For parts that require multiple manufacturing processes, a segmentation algorithm may be more suitable to identify different regions which are preferred for different manufacturing processes. Finally, the choices of data representation and the neural network are still open questions.

The authors gratefully acknowledge support from the National Science Foundation (Grant No. CMMI-2113672). The authors are grateful for the usage of the PACE high-performance computing cluster at the Georgia Institute of Technology. Any opinions, findings, and conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.

There are no conflicts of interest.

The data sets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.