Research on Weld Tracking Based on Machine Vision

Haoru Pan

School of Mechatronics and Vehicle Engineering, Chongqing Jiaotong University, ChongQing, 402260, China

Keywords: Machine Vision, Seam Tracking, Novel Algorithm.

Abstract: The era of Industry 4.0 has promoted the development of a large number of industrial industries, and it

advocates and emphasizes the overall trend of most industries in terms of intelligence. Welding has become

an indispensable part of industrial manufacturing, and improving the efficiency and accuracy of welding can

significantly improve product quality, and machine welding is currently the mainstream new welding method.

Through the research and development of a weld tracking system based on machine vision, the autonomy and

intelligence of welding robots can be improved, and the problems existing in traditional welding methods can

be solved. At present, there are still many problems in this field of welding robots, and how to solve these

problems to improve the performance of welding robots is the main research direction at present. This paper

mainly studies the application of machine vision in the field of weld tracking and verifies its feasibility and

necessity..

1 INTRODUCTION

Welding, as a common material joining method, has

been widely used in machinery manufacturing,

aerospace and navigation, automobile and other

manufacturing fields in the modern manufacturing

industry (Xi, 2011). In the manufacturing industry,

the welding process has become an indispensable

means of processing. With the advent of Industry 4.0,

all industries need to make improvements to meet the

requirements of the new era. As a representative

industrial technology, the development of automation

and intelligence is an inevitable trend in the new

industrial era.

Traditional welding methods often rely on manual

operation. However, manual operation relies on the

experience of the welder, is subjective, is labor-

intensive, and can also produce deviations due to the

influence of smoke and arc light (Jin et al, 2023). At

present, the welding robot mainly used in welding

operations solves the problems of manual welding to

a certain extent. However, the current welding robots

are still mainly teach-in robots. That is, repeated

welding of a single welding path is achieved through

the teach-in operation before welding (Dong et al,

2022). This does not completely solve the above

problems. At the same time, due to the special

working environment and special operation

requirements of welding work, the teaching robot also

has the problem of weak robustness and low welding

accuracy. With the development of technology, new

welding robots are gradually becoming popular. The

new robot generally has high autonomy and

robustness and a larger advantage range than the

teach-in robot.

As the key in welding operations, the quality of

the weld directly affects the strength and tightness of

the weldment. Therefore, for the new welding robot,

weld tracking is an important step in the process of

welding operations, and machine vision is its main

technical support. In order to solve the problems of

teach-in robots and improve the automation degree of

new welding robots, weld tracking technology based

on machine vision has received extensive attention

and application. As mentioned above, there are many

difficulties in welding operations, such as welding

seam tracking: the strong light and heat generated

during the welding operation cause the workpiece to

be thermally deformed, the strong light or spatter

covers the vision sensor, and the error generated by

each clamping workpiece, although the designers

have taken a series of measures in terms of hardware,

such as installing a baffle on the vision sensor to block

splashes and smoke and installing a filter at the front

end of the vision sensor to filter out arc light (Bing et

al, 2020). However, including but not limited to the

above-mentioned difficulties, it will still interfere

206

Pan, H.

Research on Weld Tracking Based on Machine Vision.

DOI: 10.5220/0012867000004547

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st International Conference on Data Science and Engineering (ICDSE 2024), pages 206-210

ISBN: 978-989-758-690-3

with the welding robot's seam tracking process, and if

the welding robot cannot make corresponding

adjustments in time, the welding gun will deviate

from the center of the weld, resulting in a decrease in

welding quality (Lin et al, 2015). Therefore, research

on welding robot seam tracking based on machine

vision can improve the accuracy of welding

operations and work efficiency. There is a need to

enhance the robustness of welding robots so that they

can adapt to more working environments. Promoting

the development of automation and intelligence of

welding robots, this trend has become inevitable, and

efficient welding robots will be widely used in

advanced manufacturing (Muhammad et al, 2017).

The purpose of this paper is to explore the

enhancement effect of machine vision on the

performance of weld tracking systems. This paper

summarizes the current mainstream research

direction by analyzing the structure, principle, and

innovation of the new welding seam tracking system.

At the same time, this paper points out the advantages

and problems of the improved method of welding

seam tracking system based on machine vision and

gives relevant suggestions. This research has a

positive effect on the realization of automatic welding.

2 PRINCIPLE AND WIDE

APPLICATION OF MACHINE

VISION

Since the concept of machine vision was first

proposed in the 1960s, after research and

development at home and abroad, it has gradually

matured. Machine vision is a comprehensive

technology that includes image processing,

mechanical engineering techniques, aspects of

control, electrical lighting, optical imaging, sensors,

analog and digital video technology, computer

hardware and software technology (image

enhancement and analysis algorithms, image cards,

I/O cards, etc.). A typical machine vision application

system includes image capture, a light source system,

an image digitization module, a digital image

processing module, an intelligent judgment and

decision module, and a mechanical control execution

module. By analyzing targets acquired by charge-

coupled devices (CCD ) or complementary metal

oxide semiconductors (CMOS), the object is

converted into an image signal, transmitted to the

image processing system, and finally converted into a

digital signal. In this process, the machine vision

system will extract the current various features to

achieve automatic recognition.

Machine vision has great potential in agriculture,

railway equipment, road traffic equipment, aircraft

manufacturing, medicine and other fields. However,

compared at home and abroad, the popularity of

machine vision products in China is still not high, and

there is still a certain gap with the developed countries

represented by the United States, Japan and Europe.

Especially in the international market, this aspect is

still dominated by Japanese and American enterprises,

which are superior to domestic enterprises in terms of

talent, products and technology. At present, China is

also trying to reduce the technical generation gap and

has become the world's third largest machine vision

market.

Specifically, machine vision technology

combines the efficiency and replicability of

computers with the high intelligence and recognition

ability of human vision, so it can solve many highly

repetitive and highly intelligent tasks. At the same

time, with the progress of artificial intelligence,

computer algorithm models are becoming more and

more mature, and the two are usually combined and

applied in many fields.

In industrial inspection, machine vision, with its

non-contact, strong anti-interference ability and other

advantages, can reduce the risks caused by manual

inspection and complete the work that is difficult to

complete with artificial vision. At the same time, in

some high-volume production operations, machine

vision can also greatly improve production efficiency.

In the consumer electronics industry, due to the high

quality standards of components and small size, the

use of traditional manual visual inspection methods

has many drawbacks, and in this field, based on

machine vision and representative printed circuit

board (PCB) defect detection technology, can give

the operator timely operational feedback and

information processing results, greatly reducing the

circuit component repair or waste. In terms of image

recognition, there is often a wide variety of objects

that need to be identified at present, and machine

vision has a powerful ability to process, analyze and

understand images. The introduction of machine

vision is of great help to the efficiency and accuracy

of image recognition.

Research on Weld Tracking Based on Machine Vision

207

3 RESEARCH STATUS OF WELD

TRACKING

Weld tracking technology obtains weld images

through industrial cameras, sensors, etc., and after

further processing, obtains features such as the shape

and position of the weld, and controls the movement

of the welding gun to adjust to the correct position

(Hui et al, 2022). In the field of weld tracking, there

is also a certain gap at home and abroad. Foreign

companies such as Meta in the United Kingdom,

Scansoic in Germany, Worthington Industries in the

United States, Fanuc in Japan, and General Electric in

Sweden are all enterprises with deep technical

accumulation in weld tracking. After a series of

studies in recent years, certain results have been

achieved. The image processing algorithm has been

improved and perfected, which makes the image

information obtained by the final processing more

accurate, and at the same time, drives the

development of weld tracking technology and

improves the real-time performance of the system. Up

to now, the future development trend of weld seam

tracking in China mainly includes overcoming the

shortcomings of single signal acquisition of sensors

so as to improve the effectiveness and accuracy of

information tracking, innovative image processing

algorithms focusing on the research of multi-sensor

deposition tracking system, and summarizing

welding tracking algorithms suitable for different

jobs according to different working characteristics.

According to the above-summarized trends,

domestic and foreign scholars have carried out some

research and development innovations. In terms of

the tracking problem of diagonal welds, Wang SW

(Wei, 2019) designed a weld tracking system based

on laser vision. It collects image information through

laser vision sensors and adopts image filtering, image

enhancement, adaptive threshold segmentation, linear

complement and other technologies and means to

achieve real-time automatic and high-precision weld

tracking. The initial phase of the system involves

calibrating the vision system. This entails

establishing the conversion relationship between

various coordinate systems, including the world,

camera, image, and imaging plane coordinates. High-

quality images of weld tracking are then collected and

utilized for hand-eye calibration, resulting in the

derivation of the conversion matrix between the

camera coordinate system and the robot end

coordinate system. In the subsequent phase, noise in

the image is effectively reduced using a Gaussian

filter. Furthermore, histogram equalization

techniques are applied to enhance the laser stripe

information within the image. Employing local

adaptive threshold segmentation facilitates the

segmentation of the weld image, generating a binary

image. This process effectively separates the laser

fringe information from the background information.

In the third step, the improved upper and lower

average method and Hough transform are combined

to preliminarily extract the center line of the fringe,

and the straight line correction based on the least

squares method is carried out to obtain the accurate

center line equation, and the feature points of the weld

are obtained by simultaneous solution. Through the

analysis of experimental data, the system and its

supporting image processing algorithm have good

real-time, high recognition accuracy, and high

stability. Its detection accuracy is within 1.2mm, and

the average time is less than 22 ms.

In order to realize the automatic height adjustment

function of the industrial camera and the welding gun,

Han D (Han, 2022) designed a weld positioning and

tracking system with an initial height guidance

module. The initial altitude guidance module of the

system adopts the No-Reference Visual System Index

(NRVSI), a defocus image clarity evaluation method

based on the human eye vision system, and the

evaluation results of this method are in good

agreement with the subjective evaluation results of

the human eye and the relationship between image

acquisition height and clarity can be accurately

established. At the same time, the weld tracking

module of the system uses a deep learning method to

segment the weld image, and after analyzing the weld

image, the attention mechanism is introduced on the

basis of the ENet network and the loss function is

adjusted to form a weld image segmentation network.

It can better cope with the problem of uneven positive

and negative categories in the weld image, and its

segmentation accuracy is high and its anti-

interference ability is strong.

Xi T (Xi, 2022) aimed at the problem of poor

adaptability of welding robots to many unfavorable

factors, such as the complex shape of the workpiece,

machining error, clamping error and welding thermal

deformation, and carried out research on the key

technologies of weld detection, three-dimensional

positioning, and weld trajectory and attitude tracking

based on deep learning and laser binocular vision.

This paper proposes an improved CenterNet network

method for the detection of the starting vector of the

weld and the solution of the starting point position of

the weld, which can still accurately and stably extract

the characteristics of the initial vector of the weld

under the conditions of complex background and

variable posture and type of weld, which not only

ICDSE 2024 - International Conference on Data Science and Engineering

208

realizes the detection of the starting position of the

weld but also realizes the solution of the attitude of

the workpiece and the welding gun. At the same time,

in order to reduce the tracking drift error of the

ordinary target tracking algorithm under the

background of strong noise, the Kernel Correlation

Filter (KCF) target tracking algorithm was combined

with the image segmentation algorithm based on deep

learning to integrate a weld feature point detection

algorithm, which can still maintain high detection

accuracy and accuracy under the condition of severe

noise. A real-time weld tracking system was designed

for the synchronization of weld detection, point cloud

creation, welding path and attitude online planning,

which can maintain high adaptability in the

environment of different degrees of welding noise

and meet the requirements of welding production for

tracking accuracy, stability and real-time.

4 SUGGESTION

Although domestic and foreign scholars have made a

lot of research and development improvements, in the

current context of the Industry 4.0 era, there are still

some problems that need to be solved in the welding

seam tracking technology based on machine vision.

With the continuous development of industry, the

operating environment and requirements in various

fields gradually show special differentiation. The

corresponding welds have different shapes, and there

may be deformation and wrong weld texture in actual

welding, which increases the difficulty of machine

vision sensors to identify welds. At the same time, in

the welding process, welding speed, welding Angle,

welding wire diameter and other parameters will

show different trends with the shape of the workpiece,

material, and thickness, which is a huge challenge for

the adaptability of the weld tracking system. At

present, in actual production, it is still necessary to

detect and control the welding process in real time to

find welding defects and correct them, but some

systems still have a certain delay, which cannot meet

the requirements of high real-time.

In view of the existing problems, more algorithm

models based on deep learning and neural network

technology are introduced to identify various welds

through some training. In the aspect of feature

extraction and classification of welds, a convolutional

neural network is used to improve the robustness of

the system. They combine with other sensor

technologies to obtain more comprehensive and

accurate welding process data. Carry out a large

number of welding experiments, enrich the welding

database set, and improve the system's generalization

ability. The adaptive control algorithm is introduced

to detect the changes in conditions and parameters in

the welding process in real time and realize the

adaptive tracking. Using sectional computing and

edge computing technology reduces dependence on

the central server and improves real-time

performance.

In recent years, due to the rapid development of

the artificial intelligence industry, the future welding

seam tracking technology will pay more attention to

integration with the artificial intelligence industry.

The weld tracking system can be adapted to a variety

of different welding conditions and different weld

shapes by introducing more flexible and complex

neural network structures. At the same time,

reinforcement learning and other technologies can

enable the weld tracking system to continuously

optimize its own performance and improve stability

and accuracy. With the continuous improvement of

computer power through the optimization algorithm,

hardware equipment upgrade, and other channels, the

real-time detection and instant feedback of the

welding process can be gradually realized so as to

improve production efficiency and reduce the risk of

error. In the future, weld tracking technology will not

be limited to visual sensing but will be more inclined

to multi-modal fusion technology. By combining

multiple sensing information such as vision, sound,

and temperature, the system is able to perceive all

aspects of the welding process more comprehensively.

5 CONCLUSION

In order to solve the problems of low efficiency, poor

accuracy and high cost of traditional manual welding

and the still common teach-repeat model welding

robots, this paper analyzes a weld tracking scheme

based on machine vision.

At present, there is still a certain gap at home and

abroad in the field of machine vision and the field of

weld tracking supported by its technology. There is a

large amount of demand in the foreign market, and

well-known foreign enterprises have deep technical

accumulation, occupying most of the economic

market. China started late, and the application of

machine vision and weld tracking is still in a

relatively basic stage. The same is true in terms of

market demand because the development of products

Research on Weld Tracking Based on Machine Vision

209

in this field is still insufficient. New products can not

completely replace traditional products, resulting in a

relatively small demand for machine vision and weld

tracking products in the domestic market. Although

China started late, in recent years, the attention to this

field has made China develop rapidly, and it has now

become the third largest machine vision market in the

world.

The machine vision system will use all kinds of

optical systems, analysis systems and control systems

to realize the function of automatic identification and

can realize automatic and intelligent weld tracking in

welding. It mainly converts the image signal obtained

by the sensor into a digital signal and analyzes it,

outputs the result, and controls the rest of the

components to work. In this process, multiple

technologies work together. Image filtering, adaptive

threshold segmentation and other technologies can

eliminate the noise information in the original image

and enhance the laser strip information to ensure the

quality of the collected weld tracking image. The

computer algorithm model based on deep learning

can train various systems through massive amounts of

data to improve their adaptability and efficiency. The

algorithm represented by KCF target tracking can

help the weld tracking system accurately extract the

weld feature points and improve the system's overall

accuracy and stability.

By adopting the weld tracking system based on

machine vision, the performance of the welding robot

has been greatly improved, but there are still

problems, such as difficulty in identifying welds with

special properties, lack of adaptability, and lack of

real-time performance. Increasing the training

amount of deep learning models, using volume neural

networks combined with other sensing technologies,

and introducing adaptive algorithms and edge

computing technologies can solve the above

problems to a certain extent. At the same time, it is

actively combined with artificial intelligence

technology to realize the autonomy of the welding

process and comprehensive perception to meet higher

technical requirements.

REFERENCES

L.H.Xi, “Research on key technologies of autonomous

teaching of welding robots based on visual feedback,”

South China University of Technology, (2011).

G.Jin.Guo, R.De.Lin, Z.Zheng, etc., Journal of Welding

Science, (06), 77-80+88-5 (2023).

Y.Dong, W.Xin, M.Y.Zhou, Laser Magazine, 43(11), 6-10

(2022).

G.H.Bing, Electronic Technology and Software

Engineering,(4), 115-117 (2020).

L.Lin, L.B.Qiang, Z.Y.Biao, Chinese lasers, 42(05), 34-41

(2015).

J.Muhammad , H.Altun, E.Abo-Serie, International Journal

of Advanced Manufacturing Technology,88(14), 127-

145 (2017).

Y.J.Hui, S.X.Yu, Z.K.Hao, Electronic production, 30(06),

29-31 (2022).

W.S.Wei, “Research and development of weld seam

tracking system based on laser vision,” Harbin

University of Science and Technology, (2019).

D. Han, “Research and implementation of weld positioning

and tracking system based on machine vision,”

Southeastern University, (2022).

T.Xi, “Spatial weld detection and tracking based on deep

learning and line laser binocular vision ,” South China

University of Technology, (2022).

ICDSE 2024 - International Conference on Data Science and Engineering

210