Research on Autonomous Driving Based on Multi-Sensor

Information Fusion Technology

Xuanyu Lin

School of International Education, Wuhan University of Technology, Wuhan, 430000, China

Keywords: Autonomous Driving, Multi-Sensor, Information Fusion, Intelligent Vehicle.

Abstract: With the continuous development of artificial intelligence, deep learning and sensor technology, autonomous

driving based on multi-sensor information fusion technology has become the vital research direction of the

automobile industry. This paper analyzes the multi-sensor information fusion technology and introduces the

visual sensors and radar sensors commonly used in autonomous driving in detail. This paper concludes that

multi-sensor technology has the characteristics and advantages of obtaining information more quickly, high

real-time performance and high system robustness. Moreover, this technology effectively improves the

shortcomings of traditional single sensors, such as single information acquisition, low accuracy and poor real-

time performance, and makes a good foundation for the development of autonomous driving technology. At

the same time, many studies have shown that multi-sensor information fusion technology has important

significance and far-reaching influence in the three fields of road information perception, automatic parking

technology and vehicle safety systems in autonomous driving. However, multi-sensor information fusion

technology is a multi-domain, multi-theoretical and interdisciplinary technology, so it is still facing various

challenges in applying it to autonomous driving functions.

1 INTRODUCTION

With the rapid development of intelligent vehicles

and 5G technology, autonomous driving technology

has become the focus of today's automobile industry.

Autonomous driving is an advanced auxiliary driving

system that can assist or even replace human beings

to complete a series of driving behaviors. It includes

artificial intelligence, machine vision, automatic

control systems and other parts. All parts work

together to provide drivers with a more convenient,

comfortable and intelligent driving experience. To

realize the function of automatic driving, cars often

use infrared, cameras and other visual sensors for

two-dimensional road information like traffic lights

and street pedestrians. Additionally, radar sensors and

GPS positioning technology provide information on

vehicle speed, distance, and other location

information. After processing various types of

information, the cars can realize corresponding

control. However, the information obtained by a

single sensor has the disadvantages of one-sidedness,

singleness, inaccuracy, and susceptibility to external

interference. For example, radar sensors cannot

perceive the color and characteristics of objects such

as traffic lights. The camera can distinguish road

signs well, but it cannot accurately determine the

distance and speed of vehicles, and the camera is

susceptible to extreme weather such as heavy fog and

lighting (Hao et al. 2022). Thus, multi-sensor

information fusion technology has become the key to

the research of autonomous driving. By fusing and

collaboratively processing the data of different

sensors, the drawbacks of a single sensor can be

solved and the reliability and accuracy of the

autonomous driving system can be promoted

significantly.

The development of autonomous driving based on

multi-sensor information fusion technology brings

many benefits to today's society. For example, self-

driving cars use sensors and various algorithms to

perceive road condition information, avoid obstacles

in time, reduce traffic accidents, and improve driving

safety. By sensing and predicting traffic conditions,

vehicles can effectively avoid congested roads and

optimize traffic efficiency. In addition, vehicles make

autonomous decision-making and precise control

through real-time perception of the environment,

providing more travel choices for vulnerable groups,

such as the elderly and the weak. Hu et al. (2021)

196

Lin, X.

Research on Autonomous Driving Based on Multi-Sensor Information Fusion Technology.

DOI: 10.5220/0012849500004547

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st International Conference on Data Science and Engineering (ICDSE 2024), pages 196-200

ISBN: 978-989-758-690-3

improved the accuracy of the system by integrating

radar and visual sensors, while Yang et al. (2019)

used multi-sensor information fusion technology to

optimize the original lane change warning system. It

is evident that the advancement of multi-sensor

information fusion technology has a significant effect

on autonomous driving.

This paper classifies and introduces the multi-

sensor information fusion technology, and reviews

several literature, which reflects the research status

and significance of multi-sensor information fusion

technology in autonomous driving. At last, this paper

also puts forward the challenges and development

direction of this technology in autonomous driving.

2 CLASSIFICATION AND

CHARACTERISTICS OF

MULTI-SENSOR

INFORMATION FUSION

TECHNOLOGY

2.1 Classification of Multi-Sensor

Information Fusion Techniques

Multi-sensor information fusion technology fuses the

independent observation data of multiple sensors

through a series of computer algorithms. It uses

multiple sensors to work together to obtain more

effective and comprehensive information. Therefore,

the system can eliminate the limitation that a single

sensor can only obtain part of the object information,

and improve the accuracy and intelligence of the

whole sensor system (Shuo et al. 2018).

According to the different data processing flows,

the information fusion processing structure is divided

into three types: distributed, centralized and

integrated (Su 2018). The distributed fusion structure

means that each sensor uses its own independent data

processing system. The central processor receives the

outputs of data processing and uses them for fusion

processing. All types of initial data gathered by each

sensor are transferred directly to the central

processing system for fusion processing, which

enables real-time processing, according to the

centralized fusion structure. The integrated fusion

structure integrates the advantages of distributed

fusion structure and centralized fusion structure. The

integrated fusion structure not only fuses the original

data, but also fuses the decisions of each sensor,

which enhances the accuracy of the system to some

extent. However, the amount of calculation is too

large, and the system data transmission capability has

higher requirements (Su 2018).

Table 1 shows the performance comparison of the

three fusion structures. After comprehensive

comparison, it is found that the integrated structure

can achieve high reliability of the system while

ensuring a certain accuracy, and its fusion control is

simpler than that of the distributed structure.”

Based on the abstract degree of information

processing, multi-sensor information fusion

technology is divided into data-level fusion, feature-

level fusion, and decision-level fusion (Chen 2016).

Data-level fusion is also called pixel-level fusion

(Shuo et al. 2018). This method directly fuses the

information collected by the sensor, maintains the

characteristics and properties of the data to the to the

maximum degree, and reduces the data loss, but the

data calculation is too large and the real-time

performance is poor. The feature level fusion extracts

the features of the information provided by each

sensor, then fuses these features into a specific feature

quantity, and then analyzes and processes them to

obtain useful information for the system. Although

this method can process the signal quickly, there are

problems such as information loss and large

information errors (Chen 2016). Decision-level

fusion is the highest level of information fusion. After

feature-level fusion, it jointly judges and processes

the extracted feature quantities. This method has the

strengths of high fault tolerance, strong robustness,

small calculation workload and high accuracy.

Table 1: The comparison of three structural performance (Shi & Yang 2022).

Structure Loss of

information

accuracy Communication

andwidth

reliability Computation

spee

Fusion

rocessing

Distributed

structure

high low small high fast easy

Centralized

structure

low high big low slow difficult

integrated

structure

medium medium medium high medium medium

Research on Autonomous Driving Based on Multi-Sensor Information Fusion Technology

197

2.2 Advantages of Multi-Sensor

Information Fusion Technology

Compared with a single sensor, multi-sensor

information fusion technology can identify targets in

more complex environments. By fully acquiring and

fusing the data information of the target, this

technology can reduce the amount of information,

thereby significantly improving the efficiency and

accuracy of target recognition (Chen 2019). Multi-

sensor information fusion technology generally

benefits from the following aspects: Firstly, multi-

sensors work together to obtain multi-source

information, solve the problem of single, one-sided

and high uncertainty of information, and greatly

increase the credibility of the target system. Secondly,

multiple sensors process and analysis the data, so that

the system can enhance the system's resolution, fault

tolerance and reliability (Shi & Yang 2022). Thirdly,

multi-sensor fusion technology uses a series of

computer technologies to automatically analyze,

optimize and synthesize the collected information in

time and space, and obtain a considerable description

of the research objectives. What’s more it is a data

processing method based on multi-sensor, multi-

source information is utilized as the processing

object, and the core of automatic optimization

analysis is achieved through coordinated

optimization and comprehensive processing.

Ultimately, applying this technology to the

automobile system enhances real-time performance,

enabling quick and accurate responses to various

conditions, thereby improving driving smoothness

and operational stability.

3 APPLICATION OF MULTI-

SENSOR INFORMATION

FUSION TECHNOLOGY IN

AUTOMATIC DRIVING

3.1 Commonly Used Sensors for

Autonomous Driving

In automatic driving, visual sensors and radar sensors

are often fused to obtain accurate body state and road

information. This paper will introduce common

visual sensors and radar sensors in automatic driving.

Common visual sensors include camera sensors and

infrared sensors. Among them, the data information

generated by the camera is 2D data, and the

perception accuracy of the shape and category of the

object is high. The disadvantage is that it is greatly

affected by external illumination conditions, and it is

difficult to apply to all weather conditions. The

infrared sensor does not directly contact with the

measured object during measurement, so it has the

advantages of no friction and fast response. Its

disadvantages are insufficient sensitivity, ease to be

interference, and difficulty in penetrating the object.

Radar sensors include laser radar, ultrasonic

radar, and millimeter wave radar. To use lidar, the

process involves transmitting the detection signal to

the target and comparing it with the received signal

reflected from the target. The target's relevant

information can be obtained after proper processing.

Ultrasonic radar and millimeter wave radar use the

propagation and reflection of ultrasonic waves in the

air to obtain information such as examples,

characteristics, and speed of the cars. Among them,

millimeter wave radar works in the millimeter wave

band and can measure farther distances and have

stronger anti-interference ability than ultrasonic

radar.

3.2 Research and Application of Multi-

Sensor Information Fusion

Technology in Automatic Driving

Nowadays, with the proliferation of artificial

intelligence, deep learning, sensors and other

technologies, autonomous driving has been made

possible by the use of multi-sensor information fusion

technology., and this technology is becoming more

and more mature. For example, vehicles rely on

multi-sensor information fusion technology to more

accurately complete the road information perception

function. It can also help cars accurately and quickly

complete parking identification and improve

automatic parking technology. And in the vehicle

safety system, the technology also has a vital role.

The following will introduce the application of multi-

sensor information fusion technology in these three

aspects.

3.2.1 Road Information Perception

Vehicles will encounter various unexpected

situations during driving, and excellent road

perception ability can help vehicles accurately

perceive all kinds of information and detect road

environment, then help vehicles realize data analysis,

decision control and other functions. To a certain

extent, it ensures the efficient operation and real-time

performance of the system, improves the optimization

ICDSE 2024 - International Conference on Data Science and Engineering

198

efficiency of road traffic, and reduces the occurrence

of traffic accidents.

Hu et al. (2021) proposed a vehicle tracking

method on the strength of information fusion of

millimeter wave radar and visual sensor, reducing

radar position error. The final results indicate that this

tracking method can effectively track the vehicle

position information and improve the system’s

accuracy. Wang et al. (2023) constructed a multi-

sensor fusion perception system of millimeter wave

radar and camera, and proposed a two-level

information fusion perception strategy of target

decision, then carried out experiments in urban tunnel

roads. The experimental results show that the multi-

sensor perception information target-decision two-

level fusion strategy can meet the reliable perception

requirements of unmanned vehicles in the special

environment of the tunnel, improve the system

accuracy, and make up for the shortcomings of

insufficient perception of a single sensor in the tunnel.

Wang (2020) fused the millimeter-wave radar with

the Leopard Imaging visual sensor to remove the

abnormal signal in the data and designed an

interpolation time fusion scheme to realize the fusion

of radar and visual sensor data in space and time.

Finally, the vehicle detection method is used for

experimental verification. The results show that the

multi-sensor detection algorithm can effectively

improve the vehicle detection rate. In conclusion,

sensor information fusion technology significantly

contributes to road information perception, enhancing

the accuracy and reliability of vehicle systems.

3.2.2 Automatic Parking Positioning

Function

The automatic parking positioning function in

autonomous vehicles generally involves the

comprehensive utilization of sensors like cameras and

various radars. By using this method, the accurate

positioning of obstacles can be realized, and the

information perceived by each sensor is combined to

solve the problem that a single sensor can only detect

the parking space line of the parking space or can only

detect the empty parking space formed by vehicles on

both sides (Zhang et al. 2023). For example, Yang et

al. (2023) used an ultrasonic radar on the side of the

car body, and at the same time used a fisheye camera.

The radar and the camera worked together to

accurately determine and identify the location of

obstacles in the parking space. The experimental

results show that the multi-sensor information fusion

technology can increase the anti-interference of the

system and achieve the expected accuracy. Zeng

(2020) proposed a new multi-sensor fusion method.

This method first fuses the data obtained by ultrasonic

sensors, image sensors, and wheel speed sensors to

determine the type of parking space to achieve path

planning. At the same time, the research team

established a fuzzy rule base and used MATLAB /

Simulink software to simulate, which proved the

feasibility of the method.

3.2.3 Vehicle Safety System

The autonomous vehicle can also combine various

sensors that detect the running state of the vehicle

body to fuse and correct the data obtained by multiple

sensors, which can send out an early warning in time

before the vehicle failure and reduce the occurrence

of road accidents. Guan jointly calibrated the camera

and lidar and fused the two sensor data based on the

BP neural network of cumulative error (Guan 2021).

The team also improved the observation accuracy of

different sensors and realized that the system still has

high-quality information perception and decision-

making ability in the scene with fewer obstacles, so

that the vehicle can complete the lane keeping and

obstacle avoidance function in a particular scene.

Zhang et al. (2023) proposed a multi-target vehicle

tracking algorithm and longitudinal collision

avoidance warning strategy based on multi-source

sensor data fusion. They combine radar and camera

sensors to achieve safe following of multi-target

vehicles in dense cluttered environments. Yang et al.

(2019) proposed a lane change warning model based

on the fusion of lidar, camera, long-range millimeter-

wave radar, lateral millimeter-wave radar, differential

GPS, and IMU. The team fully considered the

decisive factors such as vehicle speed and relative

distance in the determination of the acceleration of

the model, optimized the original model, and made

the multi-sensor fusion lane change warning model

more sensitive and efficient.

3.3 Challenges Faced by Multi-Sensor

Information Fusion Technology in

Autonomous Driving

Although multi-sensor information fusion technology

has shown many advantages and has been widely

used in autonomous vehicles, there are also some

challenges and problems that need to be solved. Here

are some of the key issues:

(1) At present, most of the automatic driving

technology based on multi-sensor information fusion

technology is used in good road conditions, and

Research on Autonomous Driving Based on Multi-Sensor Information Fusion Technology

199

extreme conditions and harsh environments are also a

huge challenge for automatic driving.

(2) At present, most autonomous driving uses the

fusion of two sensors, which can appropriately

increase the type and number of sensors to obtain

more comprehensive information and improve the

accuracy and stability of the system.

(3) The use of multi-sensor information fusion

technology will obtain a large amount of data, but the

on-board memory is less, and it is difficult to meet the

storage and processing calculation of a large amount

of data in some specific cases. The data should be

optimized as much as possible or the on-board

memory should be expanded to meet the technical

requirements to obtain more excellent performance of

autonomous vehicles.

4 CONCLUSION

With the advance and proliferation of the automobile

industry and the era of data, multi-information fusion

technology can be applied to various scenarios of

autonomous driving, thereby improving the

performance of vehicles in all aspects. It can also help

the car to obtain more comprehensive, accurate, and

rapid information on road conditions and vehicle

conditions, and provide a better driving experience

for the driver. Especially in the three aspects of road

information perception, automatic parking

positioning function and vehicle safety system, multi-

sensor information fusion technology optimizes the

original model and makes the vehicle automatic

driving function perform better. However, the theory

and method of realizing excellent automatic driving

functions are constantly changing, and the automatic

driving vehicle relies on the accuracy and speed of

obtaining data information to optimize and realize

each target instruction. Therefore, appropriately

increasing the number of sensors, real-time fusion of

acquired data, ensuring the high accuracy of the

system and low data loss rate have become the

research direction of automatic driving technology in

the future. Ensuring optimal performance of

intelligent vehicles in harsh environments and

enhancing vehicle memory capacity are ongoing

challenges for autonomous driving technology.

REFERENCES

F.F.Hao et al., Shanxi Electronic Technology, (03): 93-6

(2022).

Y.P.Hu et al., China Mechanical Engineering, 32(18):

2181-8(2021).

M.L.Yang et al., Automotive Engineering, 41(10): 1197-

203+227(2019).

S.Shuo et al., Auto Electric Parts, (09): 41-3(2018).

J.P.Su, " Research on the key technology of the multi-

sensor information fusion,"MA.Eng thesis, Xidian

University, 2018.

X.D.Shi and S.K.Yang, Communication & Information

Technology, (06): 34-41 (2022).

Y.C.Chen, The Automotive Semi-active Suspension State

Monitoring System,MA.Eng thesis, Xi'an

Technological University, 2016.

Z.H.Chen, International Journal of New Developments in

Engineering and Society, 3(5) (2019).

M.S.Wang et al., China Mechanical Engineering 1-13

(2023).

Y.B.Wang, Research on Road Vehicle Recognition Based

on Multi-sensor Data Fusion, MA.Eng thesis, Nanjing

University of Aeronautics and Astronautics, 2020.

C.T.Zhang et al., Chinese Journal of Automotive

Engineering, 13(05): 603-14 (2023).

Y.F.Yang et al., Proceedings of the Institution of

Mechanical Engineers, 237(5): 1021-46 (2023).

H.J.Zeng, Research on Automatic Parking Control System

Based on Multi-Information Fusion, MA.Eng thesis,

Xi'An University of science and technology, 2020.

Y.F.Guan, Development of lane keeping and obstacle

avoidance system based on multi-sensor information

fusion ,MA.Eng thesis, Heilongjiang University, 2021.

J.H.Zhang et al., Journal of Zhejiang

University(Engineering Science), 57(11): 2170-

8(2023).

ICDSE 2024 - International Conference on Data Science and Engineering

200