AOI BASED NEUROFUZZY SYSTEM
TO EVALUATE SOLDER JOINT QUALITY
G. Acciani, G. Brunetti, G. Fornarelli, A. Giaquinto and D. Maiullari
Dipartimento di Elettrotecnica ed Elettronica, Politecnico di Bari, Via Orabona no. 4, 70125 Bari, Italy
Keywords: PCB, SMT, Neurofuzzy System, Solder Printing, Soldering Assessment, Quality Index.
Abstract: Surface Mount Technology is extensively used in the production of Printed Circuit Boards due to the high
level of density in the electronic device integration. In such production process several defects could occur
on the final electronic components, compromising their correct working. In this paper a neurofuzzy solution
to process information deriving from an automatic optical system is proposed. The designed solution
provides a Quality Index of a solder joint, by reproducing the modus operandi of an expert and making it
automatic. Moreover, the considered solution presents some attractive advantages: a complex acquisition
system is not needed, reducing the equipment costs and shifting the assessment of a solder joint on the fuzzy
parts. Finally, the typical low computational costs of the fuzzy systems could satisfy urgent time constrains
in the in-line detection of some industrial productive processes.
1 INTRODUCTION
Nowadays, Surface Mount Technology (SMT) is
extensively used in Printed Circuit Boards (PCBs)
electronic production, guaranteeing compact
devices, miniature and high density. Nevertheless, in
this kind of production different defects can occur.
In particular, defects involving solder joints play a
critical role, in fact when solder paste is deposited
and printed on a board automatically, parameter
variations of the printing process may produce faulty
solder joints that could lead to failures of the final
components (Krippner, 2004). In order to detect the
presence of such defects the traditional In-Circuit
and Functional Board Test could be unreliable
because these might not work correctly if solder
joints have faulty connections (Manjeshwar, 2006).
Therefore, a reliable inspection technique is required
to examine solder joints in a PCB assembly. For this
aim, several automated non-destructive inspection
techniques like Automated X-ray Inspection (AXI),
Automated Laser Inspection (ALI) and Automated
Optical Inspection (AOI) have been proposed
(Manjeshwar, 2006; Teramoto, 2007; Wu, 2001;
Zhang, 2006; Hsu-Nan, 2006).
AXI techniques seem to be the only way to
examine solder joints of multilayer or single-layer
double-sided PCBs. Nevertheless the X-ray-based
techniques are not suitable for in-line inspection
because of their low throughput and the complicated
algorithms used for image interpretation
(Manjeshwar, 2006; Teramoto, 2007). On the
contrary, systems based on ALI methods can be used
for 3D reconstruction or vibration analysis (Wu,
2001; Zhang, 2006). Solutions performing the first
target showed effective, but slow measurement
speed and the need of high-precision motion control
are required. If used for the second aim, ALI-based
systems achieve a high inspection speed, but it is
rather complex to provide a precise alignment
between the board and the excitation laser. AOI
approaches can perform the inspection of the solder
joint quality by making use of a 2D image and are
often used to carry out a 3D reconstruction (Hsu-
Nan, 2006).
The great part of these techniques showed
effective, but rather complex acquisition systems or
high computational time are often required. For this
reason, they are not suitable for in-line inspection. In
order to overcome these drawbacks, AOI solutions
are attractive when the evaluation of a product is
based on the analysis of 2D images. For this aim,
recently, in the field of classification several kinds of
inspection algorithms have been developed by using
fuzzy/rule-based expert systems (Ko, 2000) and
neural approaches (Jagannathan, 1997; Acciani,
2006), transferring the inspection burden to the
phase of image processing. In fact, AOI systems
50
Acciani G., Brunetti G., Fornarelli G., Giaquinto A. and Maiullari D. (2009).
AOI BASED NEUROFUZZY SYSTEM TO EVALUATE SOLDER JOINT QUALITY.
In Proceedings of the 11th International Conference on Enterprise Information Systems - Artificial Intelligence and Decision Support Systems, pages
51-54
DOI: 10.5220/0001859300510054
Copyright
c
SciTePress
base their classification on visual criteria like a
human operator does. Fuzzy systems represent a
good tool for reproducing human criteria, offering
the advantage of quantifying experts’ assessment,
whereas neural networks can generalize such
assessment by automating operator’s classification
capabilities.
On the basis of these considerations, in this paper
a neurofuzzy system for solder joint quality
evaluation is proposed. The target of the present
work is to obtain a Quality Index (QI) of a solder
joint in a SMT assembled PCB, starting from the
knowledge of a human assessment. The proposed
system is composed by three modules connected in
series: a feature extraction, a supervised neural
network-based module and a fuzzy one. The
proposed solution offers some interesting
advantages: a complex acquisition system is not
needed, implying reduced equipment costs;
moreover, a low computational time is guaranteed
by the fuzzy module.
2 PROPOSED ARCHITECTURE
The proposed architecture is constituted by three
fundamental modules connected in series: a Feature
Extraction Block, a Neural Module and a Fuzzy one.
The input of the first block is the 256 grey-level
image I of a solder joint, extracted from the image of
a PCB acquired by a camera. This extraction is
conducted by using the procedure described in
(Acciani, 2006). The aim of the Feature Extraction
Block is to evaluate a vector of 18 features, called
geometric-wavelet features (GW) and reported in
(Acciani, 2006), from the image I of a single pin.
Such vector, named GW, revealed efficient in the
classification of solder joints. It can be considered as
composed by two sub-vectors, codifying the
geometric and the wavelet features, respectively.
The former type provides global characteristics of
the image under test, the latter takes into account the
level of similarity among the image under test and a
reference one.
The Feature Extraction Block feeds the
subsequent Neural one, constituted by a Multilayer
Perceptron (MLP) Neural Network, whose outputs
are given by an expert, who is required to provide
his evaluation of how much a solder joint belongs to
each of the following five classes: "Poor",
"Acceptable Poor", "Good", "Acceptable Excessive"
and "Excessive". The defined classes are considered
as fuzzy sets, therefore the expert supplies the degree
of membership of every pin in each set, allowing to
construct the target set of the Neural Block.
Let v =[ v
P
, v
AP
, v
G
, v
AE
, v
E
]
T
[0,1]
5×1
be the
membership vector related to each pin, where v
P
,
v
AP
, v
G
, v
AE
and v
E
represent the degree of
membership of the solder joint in the class "Poor",
"Acceptable Poor", "Good", "Acceptable Excessive"
and "Excessive", respectively. Then, the human
operator identifies the class which the solder joint
belongs to and assigns a degree of membership
d[0,1] to the corresponding element of the vector
v. Subsequently, the value (1-d)/2 is assigned to the
vector components corresponding to the classes
which are contiguous to the classified one. If the
expert classifies the soldering as belonging to "Poor"
(respectively "Excessive"), then the value 1-d is
assigned to v
AP
(resp. v
AE
). The remaining elements
of the vector are set to 0. Let v
i
, i = 1, 2, …, n, be the
vectors computed for each image of a database of n
solder joints, then the matrix V
= [v
1
, v
2
, … , v
n
]
[0; 1]
5×n
can be defined. Let F
18×n
be the matrix
whose columns are the GW vectors, then the
matrices F and V provide proper sets for training the
MLP network. In this way the designed block is able
to reproduce and automate human experts’
assessment on which the quality evaluation of
soldering is based.
The generic output vector v constitutes the input
for the subsequent Fuzzy Block, whose target
consists of supplying an index to express the overall
quality of a soldering. In detail, the universe of
discourse is divided into the five sets Poor,
Acceptable Poor, Good, Acceptable Excessive and
Excessive, whose activation values are given by the
components of vector v. The generic output is
constituted by the vector y = [y
1
y
2
y
3
y
4
], whose
domain is characterized by four output fuzzy
subsets, defined as Few, To Increase, To Lower,
Too Much, respectively. These sets codify all the
possible situations including cases corresponding to
good quality. Such fuzzy sets are characterized by
four singleton membership functions, centred on the
real values
0123
aaaa
<
<< in the range [-1,1], as
shown in Figure 1.
Values |a
1
| |a
2
| are chosen, because the case of
“acceptable poor” soldering paste is considered less
desirable than the case of the “acceptable excessive”
one by the experts. The following linguistic rules
map the fuzzy input sets into the output ones:
Rule 1) IF Soldering is Poor THEN (y is Few)
Rule 2) IF Soldering is Acceptable Poor THEN
(y is Few) and (y is To Increase)
AOI BASED NEUROFUZZY SYSTEM TO EVALUATE SOLDER JOINT QUALITY
51
Rule 3) IF Soldering is Good THEN (y is To
Increase) and (y is To Lower)
Rule 4) IF Soldering is Acceptable Excessive
THEN (y is To Lower) and (y
is Too Much)
Rule 5) IF Soldering is Excessive THEN (y
is
Too Much).
Subsequently, the elements of the vector y are
defuzzified to compute the Quality Index (QI) using
the well known method of the “centre of gravity“.
The resulting value QI [-1, 1] describes the quality
of the solder joint under test.
Figure 1: Membership function of output fuzzy sets.
3 EXPERIMENTAL RESULTS
The performances of the proposed neurofuzzy
architecture have been investigated by means of a
database formed by 480 images. In Figure 2(a) the
image of a complete PCB is reported. Figure 2(b)-(f)
shows images of pins that an expert classified as
belonging to the five defined classes.
Figure 2: (a) PCB Sample (b)-(f) Solder joints belonging
to the defined classes.
The input vectors GW for the MLP network are
computed from the images of the database by the
Feature Extraction Block. In order to synthesize the
architectural parameters of the Neural Module, the
Mean Absolute Error (MAE) index is defined as:
2
=1
1
MAE = -
n
ii
i
yt
n
being y
i
and t
i
the output and the target of the
network under test, respectively, and n the number
of the data samples. The performances of the neural
network have been evaluated by minimizing the
mean value of the MAE index computed over r = 20
trainings with different initial weights. The designed
network has input, hidden and output layers formed
by 18, 12 and 5 neurons, whose activation functions
are logarithmic-sigmoid. The network has been
trained by the Levenberg-Marquardt algorithm,
minimizing the Mean Square Error. In order to show
the capabilities of the neural network in reproducing
the assessment of a human expert, in Figure 3 the
values of the target versus the output of the network
are reported. It is worth to observe that the more
points consolidate on the bisector line the more the
network is able to reproduce the assessment of the
expert.
Figure 3: Values of the MLP target vs MLP output.
Subsequently, the output of the network is
processed by the Fuzzy Block. In Figure 4 the values
of QI, computed for the samples in the considered
database and sorted by the membership to each
defined class, have been reported.
Figure 4: Values of QI computed for the considered
samples.
To evaluate the reliability of the outputs
provided by the whole neurofuzzy architecture, the
values of QI have been partitioned into five hard sets
by the four thresholds th
1
, th
2
, th
3
, and th
4
which are
computed as:
ICEIS 2009 - International Conference on Enterprise Information Systems
52
()
PAP
1mm
QI +QI / 2th =
()
AP G
2mm
QI +QI / 2th =
()
GAE
3mm
QI +QI / 2th =
()
AE E
4mm
QI +QI / 2th =
where
PAPGAE E
mm mm m
QI , QI , QI , QI and QI are the mean
QI value of the elements belonging to the class
Poor, Acceptable Poor, Good, Acceptable
Excessive
and Excessive, respectively. Such
partition of the data can be carried out by using these
thresholds as follows: (a) if QI<th
1
, then soldering is
Poor; (b) if th
1
<QI<th
2
, then soldering is
Acceptable Poor; (c) if th
2
<QI<th
3
, then soldering
is
Good; (d) if th
3
<QI<th
4
, then soldering is
Acceptable Excessive; (e) if QI>th
4
, then soldering
is
Excessive. The obtained partition is compared to
experts' one. In particular, a pin is considered as
correctly classified by the system if it belongs to the
same set when considering both the former and the
latter partition. On the contrary, if this condition is
not satisfied, then a misclassification takes place, as
it is shown by the diamond marks in Figure 4. The
performances of the architecture are measured by
defining the Recognition Rate index as:
C
TOT
N
RR = ×100
N
being N
C
and N
TOT
the number of correctly
classified cases and the number of the considered
ones, respectively. Values of RR equal to 96.87%
and 95.83% concerning training and testing data
have been obtained. The results can be considered
encouraging, in fact the obtained values show that
the designed neurofuzzy system yields a
classification similar to that given by the experts,
providing a refined evaluation of the solder joints.
4 CONCLUSIONS
In this paper a neurofuzzy architecture for
computing a Quality Index of a solder joint in a
SMT assembled PCB has been proposed. The
system offers some interesting advantages. In
particular, the suggested solution does not need a
complex illumination and positioning system,
implying that the equipment costs could be reduced
and the assessment of a solder joint could be shifted
on the fuzzy evaluation phase. Moreover, the typical
low computational costs of the fuzzy systems could
satisfy urgent time constrains in the in-line detection
of some industrial productive processes. The
proposed architecture provides a refined evaluation
of the solder joints, automating the human expert
classification.
Basing on the obtained results, it can be argued
that the correct working of the proposed system is
due to its capability to reproduce human experts’
modus operandi properly. Therefore, future
developments will be aimed at identifying the
characteristics, that a human operator evaluates in
order to express the assessments of solder joints. As
a consequence, the focus of future works will be
constituted by the identification of the features
which contain sufficient and useful information to
perform a correct diagnosis.
REFERENCES
Krippner, P., and Beer, D., 2004. “AOI Testing Positions
in Comparision”, Circuit Assembly, Apr. 2004, pp. 26-
32.
Manjeshwar, P., Craik, J., Phadnis, S., and Srihari, K.,
2006. ‘Effectiveness Study of an Automated 3D
Laminography X-Ray Inspection System in a High
Volume–Low-Mix SMT line’, The Int. J. of Adv.
Manuf. Technol., vol. 30 (11-12), pp. 1191 – 1201.
Teramoto, A., Murakoshi, T., Tsuzaka, M., and Fujita, H.,
2007. “Automated Solder Inspection Technique for
BGA-Mounted Substrates by Means of Oblique
Computed Tomography”, IEEE Trans. on Electronics
Packaging Manufacturing, vol. 30 (4) , pp. 285 – 292.
Wu, Y. P., Tu, P. L., and Chan, Y. C., 2001. “The effect of
solder paste volume and reflow ambient atmosphere
on reliability of CBGA assemblies,” J. Electron.
Packag., vol. 123 (3), pp. 284–289.
Zhang, L., Ume, I.C., Gamalski, J., and Galuschki K. P.,
2006. “Detection of Flip Chip Solder Joint Cracks
Using Correlation Coefficient and Auto-Comparison
Analyses of Laser Ultrasound Signals”, IEEE Trans.
on Comp. and Pack. Technol., vol. 29 (1), pp 13-19.
Hsu-Nan, Y., Du-Ming, T., and Jun-Yi Y., 2006. “Full-
Field 3-D Measurement of Solder Pastes Using LCD-
Based Phase Shifting Techniques”, IEEE Trans. on
Electron. Packag. Manuf., vol. 29 (1), pp. 50 – 57.
Ko, K.W., and Cho H.S., 2000. “Solder joint inspection
using a neural network and fuzzy rule-based
classification method”, IEEE Trans. on Electron.
Packag. Manuf., vol. 23 (2), pp. 93–103.
Jagannathan, S., 1997. “Automatic Inspection of Wave
Soldered Joints Using Neural Networks”, J. of Manuf.
Syst., vol. 16 (6), pp. 389-398.
Acciani, G., Brunetti, G., and Fornarelli G., 2006. “A
Multiple Neural Network System to Classify Solder
Joints of Integrated Circuits”, Int. J. of Computational
Intelligence Research., vol. 2 (4), pp. 337-348.
AOI BASED NEUROFUZZY SYSTEM TO EVALUATE SOLDER JOINT QUALITY
53