GMAW welding procedure expert system based on machine learning

1. INTRODUCTION

With the development of the industrial internet of things, big data, artificial intelligence, machine learning and other technologies, information technology has been popularized, and intelligent manufacturing has also developed rapidly. Digitalization and intelligence have become the basis of manufacturing production. Welding is the basic processing method in the manufacturing industry. The combination of them promotes the automation and intelligence of welding process. With the urgent demand for data sharing and professional knowledge in welding enterprises, the welding expert system plays an increasingly important role in intelligent welding systems^[1].

An expert system is a computer program that combines the experience and knowledge of experts in a specific field. As early as the mid-1960s, a large number of researchers began to study the expert system. Welding technology is the product of multi-disciplines, ranging from weldability analysis before welding^[2,3] and preparation of welding process documents^[4] to the selection of process parameters in the welding process^[5-8] and real-time control of welding^[9], defect diagnosis and treatment after welding^[10], welding quality assessment^[11,12] and calculation of welding material consumption^[13]. Each stage involves a large amount of data, knowledge and models, which is an ideal field for expert system application^[14,15].

GMAW is a welding method that uses the arc heat generated between the welding wire and the weldment to melt the welding wire and the base material continuously fed by the welding torch under shielding gas to achieve the welding purpose. This method has the advantages of low cost, strong adaptability, easy operation and easy automation. The main advantage of GMAW lies in its metal deposition per hour, which greatly reduces labor costs. In addition, this technology is a clean process because it does not use flux, and it is easy for operators to observe the situation of the arc and molten pool and make adjustments at any time. The advantages of GMAW make it widely used in many industrial applications^[16]. Metal transfer modes are the way to transfer the molten electrode droplet to the molten pool. The modes are generally classified into three forms, which are often used for welding quality control and defect detection. Different metal transfer modes correspond to different welding parameters, especially current parameters. In this expert system, the current is mainly used as the output parameter, so the metal transfer modes are not considered as the input parameter^[17].

Welding procedure design expert system has always been a hot topic in the research of the welding expert system. A complete welding process involves many parameters. If developers can collect key information such as welding joint, groove, size and position through expert systems, process the data with the help of expert experience and knowledge, study relevant welding standards in the industry, and combine data mining and machine learning technology to generate specific welding process parameter schemes, which could avoid plenty of repetitive work. The use of the expert system improves the storage mechanism of welding data. By constantly updating the expert system with new technologies in this field, knowledge beyond expert experience will be found, which plays an important role in promoting the design of welding technology. In this case, research on the welding expert system is very necessary.

From the traditional expert system, which can only answer the results already in the system, to the expert system combining machine learning algorithms, software and hardware technology^[18-22]. Due to the rapid development of new computer technology, researchers in the world have made many attempts. The expert system can help users to make decisions by simulating expert thinking and predict answers to questions that are not involved in the knowledge base.

The design of the knowledge base, database and inference engine is the key part of an expert system design. The sustainability and extensibility of development are the difficulties in designing each module. As for the knowledge base design, Zhang^[23] uses the traditional design pattern and divides the design into knowledge representation and knowledge acquisition. Liu et al. use knowledge graph technology to build an expert system^[24]. The system crawls the data on the Internet, performs format conversion, entity recognition and other processing on the data, which accumulates frequently asked questions and their solutions to form the knowledge base, and provides a new idea for the knowledge acquisition of the expert system. Considering that the filling and modification of knowledge in the future may completely depend on the developer if only the database is used, which is not intelligent, and only applying crawler technology does not meet the parameter design requirements of accuracy and reliability. Therefore, the knowledge base design of this system combines the two methods, which employ the algorithm to improve knowledge acquisition and knowledge representation using the database. Regarding the design of the database and inference engine, Xiao et al. uses a transfer learning model^[25]. Firstly, the system learns the sample dataset to form the training model, and then migrates it to the system. The daily sampled data are further learned by the deep learning algorithm to continuously optimize the model. Similarly, this system designs a self-learning module based on the transfer learning model. It overcomes the problem that there is no accumulated data when the software is just used, and the number of reasonable solutions is insufficient in complex welding conditions, which leads to the inaccuracy of recommended solutions. Some machine learning models provide black-box prediction, and the prediction accuracy is satisfactory, but the model explanation is insufficient, which leads to users’ lack of confidence in the reliability of the results. Ahmed et al. puts forward a framework to work together with the established machine learning model and expert knowledge^[26]. Based on the generated model, the framework extracts decision rules and adopts expert suggestions, makes the dataset self-sufficient and constantly generates new rules by checking data sets. This system also takes into account the expert suggestions to improve the prediction accuracy of the system and interfaces some key parameters, which allows experts to adjust parameters according to experience and the effect of parameters in actual welding, so as to achieve the desired output.

The welding process is complicated, the final output parameters of the expert system are only a part of the welding process, and many parameters affect the welding process, so it is usually fundamental to visually monitor the welding process. Human welders can adjust the parameters in real time by observing the welding process to obtain the desired output. In addition to using sensing technology to simulate human welders to monitor the welding process, the welding procedure design expert system combined with the machine learning algorithm only needs to give the initial information and expected output under the actual welding conditions, and can simulate the intermediate process by itself based on the data without the user's understanding of the welding process, and obtain the expected output. The training data set often comes from the combination of parameters with a good welding effect, and with more data accumulated in the system, the algorithm learns again and optimizes the existing model parameters, which is also the correction of ideal output and actual welding results, and realizes the intelligent adaptation of the welding process to the environment to some extent^[27].

In recent years, XGBoost has gained great popularity in various data science competitions. The gradient boosting principle behind XGBoost is a representative algorithm of the boosting method in the ensemble algorithm, and XGBoost can be applied to both classification and regression problems. As verified by Guan^[28], compared with BP neural network, ARMA model and KNN algorithm, XGBoost model has higher accuracy and faster operation efficiency. In this system, XGBoost, with the best prediction performance, is selected as the machine learning model. With the adoption of different algorithms, expert systems have been widely studied and applied in various fields of science, engineering, agriculture and medicine, providing valuable experience for the research and development of welding expert systems in this paper^[29-36].

The overall architecture design of the GMAW welding procedure expert system, and the main functional modules will be summarized first. The second section will describe the knowledge base and the methods to supplement and update it. The database technology will be introduced in the third section, describe the main components and show some sample tables. The fourth section will introduce the reasoning logic of this system, and expound on the algorithm principle of welding parameter intelligent recommendation module. The fifth section will apply the expert system to practice, introduce the system workflow, and display the key output parts, as well as the concrete effect drawings obtained by actual welding. The last part is the summary and prospect of this paper.

2. OVERALL DESIGN OF GMAW EXPERT SYSTEM

The composition of the expert system usually includes the human-computer interaction interface, knowledge base, inference engine, and interpretation engine part. Faced with questions posed by users, computer programs simulate the thinking of experts to solve problems. Through the knowledge acquisition module, the human-computer interface is the carrier of getting user input information, calling the system’s storage of knowledge and rules, relying on the inference engine for the logical reasoning calculations, with the aid of the interpretation engine all the knowledge and logic used to conclude can be viewed, makes the whole operation process is determined.

The overall structure of the expert system is shown in Figure 1. Considering the welding process expert system involves many parameters, the human-computer interaction interface, knowledge base database, and inference engine’s realization are the main parts of the system design.

Figure 1. Overall system design.

Human-computer interaction interface is the dialogue window between the user and the program. The system involves many parameters of input and calculation, the interface will be an important part of the system design, and considering the actual welding environment is not clear, the interface design and function realization is mainly on the Qt Creator platform, and packaged to generate software that can be used offline, can cope with the complex working environment.

The knowledge base and database are the core part of the expert system. The completeness of system knowledge determines the reliability and accuracy of an expert system decision, and the sustainability of knowledge base development determines the service life of the expert system. This system adopts the mode of knowledge base and database work together. The software has its own basic knowledge base and database at first, and users then use the system to produce new data and timely feedback to the system database. When accumulating a certain amount of new data, the expert system uses the machine learning algorithm to learn new knowledge from new data, and optimize the existing systems. The knowledge base and database can be updated constantly to achieve the goal of more accuracy and more consistency with the actual production scene.

To evaluate the correctness of an expert system’s decision, in addition to the need for a good knowledge base design, it is more important to make the most of the existing professional knowledge. The inference engine is the whole knowledge and control strategy used to deduce the conclusion, and it is the key to integrating knowledge into the program. There are many welding parameters in a complete welding process, and the coupling degree of parameters is high. Changing one parameter will lead to the recalculation of a series of parameters. Therefore, the reasoning mechanism of this system combines accuracy and fuzzy. When the initial information is sufficient, the system calls the knowledge base and database to use forward reasoning to achieve accurate reasoning. When the initial information is insufficient to reason, the system uses the database data and combines the machine learning model to make a fuzzy prediction of the result.

Based on the overall architecture design of the system, the specific functional modules of the system mainly include the following parts:

(1) User management module

This module mainly divides the identity of users into ordinary user, engineer, and administrator roles. Ordinary users can use basic functional modules. On the basis of ordinary users, engineers can modify the knowledge base data on the interface, and generate more customized parameter schemes for the user's working environment. In addition, the administrator can manage user information, enhance user’ rights, and carry out daily system maintenance.

(2) Document management module

The module includes consulting the industry welding standards and weldability documents of welding materials, managing previous welding schemes and statistical information on weld cost corresponding to product functions, and providing help for users' pre-welding preparation.

(3) Knowledge base module

This module is responsible for the management of the rule base and knowledge database. Users modify relevant parameters according to their actual needs, and the system calculates the modified parameters in real time to generate a more guiding scheme for the work site environment. In addition, users can save the expert system parameters with expected welding effects in the actual welding site to the database.

(4) Procedure parameters design module

The welding procedure design module is the key part of this expert system, which also includes sub-functional modules such as calculation of weld cross-sectional area, planning of multi-layer and multi-pass welding, calculation of welding parameters and intelligent recommendation of parameter schemes. The weld cross-sectional area calculation module prepares data for groove stratified separation, where the stratified separation scheme under different welding parameters is various, the parameters of the recommended scheme based on different modes, are also combined with the cost calculation module to meet the user’s need.

(5) Material consumption calculation module

This module mainly includes the input and preservation of weld information, calculation of weld cost, and generation of cost files. For a specific product, users input the relevant information according to the system, where the cost can be divided into two parts, artificial operation cost and material cost. Save input information into the corresponding cost file, and summarize the total length and mass, welding wire consumption and other information of all welds under the current product. The module also provides two working modes: single weld and weld cost comparison mode. The calculation method also has different evaluation indicators to meet different needs. The flow chart of the module is shown in Figure 2.

Figure 2. Flow chart of weld cost calculation.

In the system, users input relevant information under the prompt, and the system integrates the information and displays it in a user-friendly manner. The output interface provides a set of appropriate welding procedure design schemes, which supports the user to modify the data and real-time calculation, but also can directly export the WPS for direct use. The extension module of the consumable cost calculation provides more reference indexes to meet the demand of actual welding conditions. With the use of this expert system, the formulation of the welding parameter scheme is not limited to experienced engineers, but enables new technical personnel to complete reasonable and effective welding through the welding scheme recommended by the system, avoiding the dependence on experienced personnel before actual welding.

3. KNOWLEDGE BASE DESIGN

An expert system integrates the knowledge of multiple experts in related fields. The accuracy of an expert system depends mainly on the completeness of the knowledge base and the ability of programmers to convert expert knowledge into codes. The content of the knowledge base can be divided into explicit knowledge and tacit knowledge. Explicit knowledge refers to that formulaic, regular and easy-to-express knowledge, while tacit knowledge refers to empirical, heuristic and difficult-to-convert knowledge. Tacit knowledge in this system mainly comes from experts’ experience and the enlightening knowledge obtained by reading welding documents. The explicit knowledge in this system mainly includes the part shown in Figure 3.

Figure 3. Part of knowledge base display.

In terms of designing the knowledge base, considering the wide variety of welding technologies and welding equipment, different instruments, operators and environmental conditions will lead to differences in the welding process. Even if the same parameters are set, the welding effect is different under different environments. In the realization of the knowledge part of the system, the tacit knowledge is incorporated into the program, the explicit knowledge is usually stored in the knowledge base, and key parameters and formulas that may need to be modified are made into interfaces to wait for the adjustment of experts, so as to meet the desired output under the actual welding conditions and make the whole system more flexible.

The knowledge acquisition part is the expansion of the knowledge base. The knowledge acquisition of this system mainly comes from the following aspects:

1. Basic rule base and database after downloading the software;

2. New welding data is added to the database after users use this expert system;

3. New knowledge learned by machine learning algorithm on new welding data;

4. Update data on the server at irregular intervals.

4. DATABASE DESIGN

The database part of the system combines MySQL database and SQLite database, which includes three major contents: a user login information table, welding information database, and cost calculation database. Considering the unclear welding environment, the basic functions can be realized by using the local SQLite database provided by the software when there is no network. When the network is available, users connect to MySQL remote database and automatically update the local database and upload user data to complete the update of the knowledge base and database on the client and server. In the design of the expert system database, different databases are selected, so as to improve the adaptability of the system to different welding environments. The database contents of some recommended parameter settings are shown in Figure 4.

Figure 4. Part of database display.

Among the tables as the main content of the database, the welding information database includes a series of tables, such as the groove information table, welding position graph table, layered lane division strategy table, welding setting reference table and so on. The cost calculation database stores specific information and corresponding cost statistics of each weld under different products. Table 1 shows only part of the database.

Table 1

Sketches of common welded joint

Joint type	Groove type	Position	Weld sketch	Dimension parameter
T-joint	I	2F		h: weld leg size w: weld leg size a: groove angle
T-joint	Single-V	2F		a: groove angle d: groove depth r: root face w: weld leg size h: weld leg size
Lap-joint	I	2F		h: weld leg size w: weld leg size
Butt-joint	V	1G		t: thickness a1: groove angle a2: groove angle r: root face g: root gap

5. INFERENCE ENGINE DESIGN

The reasoning logic of this system is mainly shown in Figure 5. According to the initial information of joints, welding wires and positions input by users, combined with the corresponding rule base and database, the system carries out multi-layer and multi-pass welding planning for the joint groove of specific welding positions, and obtains detailed data information of each weld. The key parameters set are loaded and calculated in real time to obtain the important parameters guiding robot welding. Users can modify the data information in each weld to generate welding parameters that meet the user’s expectations.

Figure 5. Expert system inference logic.

The design of the inference engine uses the principle of accurate and fuzzy reasoning. The concrete realization is that when the user inputs enough initial information, the inference engine calls rules and data, carries on the forward inference and obtains the accurate output. When the user inputs only part of the data, the system invokes the machine learning model built on the sample dataset and predicts the output. When more data instances are accumulated, the expert system will use the self-learning module integrated into the system to re-learn the model on the new data on the basis of the existing model, adjust the parameters of the model, and continuously improve the prediction accuracy.

5.2 Intelligent parameter recommendation

A complete set of the machine learning process includes data preprocessing, data exploration, feature engineering, machine learning modeling, parameter tuning, and test set validation. In this paper, for the intelligent prediction module of parameters, since this is a supervised regression problem, five regression prediction models containing linear regression, decision tree regression, random forest, polynomial regression and XGBoost are used to compare the model accuracy and operation speed on the same dataset. The dataset includes wire type, wire diameter, distance from conducting nozzle to the workpiece, wire feeding speed, and current, where current is used as the prediction column in this regression. There are 492 samples in the dataset.

(1) Modeling steps

Firstly, considering that the dataset comes from the reasonable parameter scheme of the welding site, there are no outliers or missing values. After that, the data were mapped in MATLAB to make a study of the data relationship. The features were selected in combination with the relevant knowledge and experience of parameter calculation provided by experts. The linear regression, decision tree, random forest, polynomial regression and XGBoost model, which are widely used in the current regression machine learning algorithms, were used to fit the data. In the model-building process, grid search and learning curve drawing were used to optimize the parameters.

(2) Model parameter optimization

The self-learning module of the system mainly realizes that when more instance data is accumulated in the system, the system calls the internally packaged function to optimize and reset the parameters of the machine learning model. XGBoost model has many parameters. Table 2 contains parameters mainly used for the process of parameter adjustment in this expert system.

Table 2

XGBoost model parameter list

Model parameters	Instructions
n_estimators	Number of weak evaluators in the model
learning_rate	To control the iteration rate of the tree
subsample	The proportion of sampling from samples
max_depth	Maximum tree depth of the weak evaluators
objective	The objective function
gamma	A decrease in objective function to minimize branching
reg_alpha	L1 regularization coefficient in the objective function
reg_lambda	L2 regularization coefficient in the objective function
n_jobs	Number of parallel threads used to run model
random_state	Random seed
colsample_bytree	The proportion of features selected each time

As shown in Table 2, the XGBoost model has many parameters, different model has different parameters, and parameter adjustment is also different. In the design process of this expert system, grid search and learning curve drawing methods are combined to find the best parameters.

In Figure 6, the GridSearchCV library in the sklearn module is called to arrange all parameters to find the combination that makes the scoring index perform best. The drawing learning curve method used in Figure 7 mainly relies on the programmer’s judgment of the effect. Lines in the figure represent the effect of XGBoost under default parameters, grid search recommended parameters, and learning curve manual parameter adjustment.

Figure 6. Grid-search codes and results.

Figure 7. Learning curve search parameters. The X-axis represents the number of weak estimators used in the current model, and the Y-axis shows the mean square error of the model.

From the trend of the lines in Figure 7, XGBoost has a good model effect only with the default parameters, which shows the power of the model design. The parameters determined by drawing the learning curve significantly reduced the degree of the overfitting phenomenon and performed best. The parameters by grid search reduce the error on the training set compared with the learning curve parameters. Grid search is the permutation and combination of all parameters, which is more time-consuming on medium and large datasets, but the advantage is that it is convenient to program, and users do not need to adjust parameters manually. By default, this expert system uses the grid search style to update machine learning models. Due to the time-consuming, the system will update the new data in the background when the data reaches the set threshold. After updating, the system will use the pickle library to save the new model for the next direct invocation.

(1) Model evaluation index

In the whole process, the ratio of 70% training set and 30% test set was used to divide the dataset. The same training set and test set for preliminary debugging are used to get the current parameter combination of the model, and the obtained parameter is used to train each model. After training, the mean absolute error (MAE), mean percentage of absolute error (MAPE), mean square error (MSE), and R² (a common indicator of regression models in machine learning) were used to evaluate the accuracy of the model.

The calculation formulas of the model evaluation index in Table 3 are as follows:

(6) $$ \begin{equation} M A E=\frac{\sum_{i=1}^{n}\left|y_{i}-\hat{y}_{i}\right|}{n} \end{equation} $$

(7) $$ \begin{equation} \text { MAPE }=\frac{1}{n} \sum_{i=1}^{n}\left|\frac{y_{i}-\hat{y}_{i}}{y_{i}}\right| \times 100 \% \end{equation} $$

(8) $$ \begin{equation} M S E=\frac{\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}}{n} \end{equation} $$

(9) $$ \begin{equation} R^{2}=1-\frac{\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}}{\sum_{i=1}^{n}\left(y_{i}-\bar{y}\right)^{2}} \end{equation} $$

Table 3

Each model’s performance on the test set

	Evaluation index	MAE	MAPE	MSE	R2
Test set
Linear regression		22.1175	11.6555%	870.8522	0.9148
Decision tree regression		2.8114	1.4815%	43.3126	0.9958
Random forest		4.5157	2.3797%	57.0616	0.9944
Polynomial regression		2.9628	1.5614%	14.4916	0.9986
XGBoost		2.4408	1.2863%	12.3635	0.9988

(2) Model effect analogy

In Figure 8, the linear regression model has a large variance and low stability after fitting. The fitting effect of the regression tree and random forest regressor is similar, which is more stable than that predicted by the linear regression model, but some data are still not fitted. Obviously, the dataset performs better on polynomial regression and XGBoost, and the model prediction has higher accuracy. Combined with the accuracy in Table 3, the XGBoost model is superior in all aspects. The model is integrated by multiple weak estimators and has a more stable prediction accuracy regardless of large or small datasets. Based on this, XGBoost is adopted as the parameter prediction model in this system, and the actual welding verification shows that the model has a good prediction effect.

Figure 8. Model fitting data renderings. The X-axis represents the id of the data sample, and the Y-axis represents the value. A: linear regression model; B: respectively are the fitting effects of decision tree regression and random forest, both of which are based on binary trees; C: the model of polynomial regression and XGBoost, both have high prediction accuracies.

6. SYSTEM TEST AND APPLICATION

6.1 System workflow

The software is mainly divided into two functional modules: welding parameter design and weld cost calculation. The parameter design module includes three interfaces, and the user can input information on the first two interfaces according to the actual welding conditions. The system workflow is shown in Figure 9, and the final output is shown in Figure 10. The button on the interface supports the integration of all parameter information into WPS for direct use [Figure 11].

Figure 9. Work flow chart.

Figure 10. Output parameter interface.

Figure 11. The generated WPS.

The interface of the cost calculation module is shown in Figure 12. The user completes the input of weld, procedure and cost related parameters on the interface, and the total cost is calculated and displayed in real time. Save, export and import buttons support the function of saving the cost information entered by the user into the database, generating the weld cost file, and reading the unfinished weld information to the work area to continue writing. The statistics button supports statistics of all parameters under the current product. Single mode displays the cost information of a single weld, while the comparison mode can display the cost comparison of two welds.

Figure 12. Cost calculation interface.

6.2 Actual application effect

The components of the robot system used are shown in Figure 13. After users input information, the welding effect diagram of the WPS recommended by the expert system in the actual scene is shown in Figure 14, and the welding parameters used are the output parameters in Figure 10.

Figure 13. Robot system.

Figure 14. Welding renderings.

It is easy to see that the welds are well formed with good consistency and less spatter. The weld has no external defects such as edge bite, pore and weld tumor. The measured size of the leg length is 16 × 16, which is consistent with the setting of the expert system. As shown in the macro section photos of the weld in Figure 15 below, the internal fusion of the weld is good, and there are no cracks, pores and other internal defects in the weld. Therefore, the research and development of the expert system are quite necessary.

Figure 15. Weld macro section figures.

7. CONCLUSION

After the machine learning model is trained on the initial training set, the model migrates to the expert system, and users use the system to generate new data instances later. When the number of data instances accumulates to a certain extent, the software optimizes and updates the model parameters in the background, continuously improves the prediction accuracy, and avoids waiting a long time for model updates. After downloading, the software carries the initial rule base and database, which can realize all functions of the expert system. With the self-learning module, the rule base and database can be dynamically updated to cope with different welding scenes.

Compared with other procedure design expert systems, this system improves the expansion mechanism of the knowledge base and database, realizes the dynamic update by using machine learning, and can adapt to different situations by selecting different databases. This system will be easy to modify knowledge has been made into an interface; users do not need to operate the database by themselves, but the rest of the knowledge representation, modification of the latter mainly relies on programmers to achieve, and it is hoped that in the future the formation of a welding knowledge representation system for the welding field can be considered. To realize the full automation of knowledge base representation and acquisition and avoid the repeated development of enterprise-oriented and data-oriented expert systems.