AOI BASED NEUROFUZZY SYSTEM

TO EVALUATE SOLDER JOINT QUALITY

G. Acciani, G. Brunetti, G. Fornarelli, A. Giaquinto and D. Maiullari

Dipartimento di Elettrotecnica ed Elettronica, Politecnico di Bari, Via Orabona no. 4, 70125 Bari, Italy

Keywords: PCB, SMT, Neurofuzzy System, Solder Printing, Soldering Assessment, Quality Index.

Abstract: Surface Mount Technology is extensively used in the production of Printed Circuit Boards due to the high

level of density in the electronic device integration. In such production process several defects could occur

on the final electronic components, compromising their correct working. In this paper a neurofuzzy solution

to process information deriving from an automatic optical system is proposed. The designed solution

provides a Quality Index of a solder joint, by reproducing the modus operandi of an expert and making it

automatic. Moreover, the considered solution presents some attractive advantages: a complex acquisition

system is not needed, reducing the equipment costs and shifting the assessment of a solder joint on the fuzzy

parts. Finally, the typical low computational costs of the fuzzy systems could satisfy urgent time constrains

in the in-line detection of some industrial productive processes.

1 INTRODUCTION

Nowadays, Surface Mount Technology (SMT) is

extensively used in Printed Circuit Boards (PCBs)

electronic production, guaranteeing compact

devices, miniature and high density. Nevertheless, in

this kind of production different defects can occur.

In particular, defects involving solder joints play a

critical role, in fact when solder paste is deposited

and printed on a board automatically, parameter

variations of the printing process may produce faulty

solder joints that could lead to failures of the final

components (Krippner, 2004). In order to detect the

presence of such defects the traditional In-Circuit

and Functional Board Test could be unreliable

because these might not work correctly if solder

joints have faulty connections (Manjeshwar, 2006).

Therefore, a reliable inspection technique is required

to examine solder joints in a PCB assembly. For this

aim, several automated non-destructive inspection

techniques like Automated X-ray Inspection (AXI),

Automated Laser Inspection (ALI) and Automated

Optical Inspection (AOI) have been proposed

(Manjeshwar, 2006; Teramoto, 2007; Wu, 2001;

Zhang, 2006; Hsu-Nan, 2006).

AXI techniques seem to be the only way to

examine solder joints of multilayer or single-layer

double-sided PCBs. Nevertheless the X-ray-based

techniques are not suitable for in-line inspection

because of their low throughput and the complicated

algorithms used for image interpretation

(Manjeshwar, 2006; Teramoto, 2007). On the

contrary, systems based on ALI methods can be used

for 3D reconstruction or vibration analysis (Wu,

2001; Zhang, 2006). Solutions performing the first

target showed effective, but slow measurement

speed and the need of high-precision motion control

are required. If used for the second aim, ALI-based

systems achieve a high inspection speed, but it is

rather complex to provide a precise alignment

between the board and the excitation laser. AOI

approaches can perform the inspection of the solder

joint quality by making use of a 2D image and are

often used to carry out a 3D reconstruction (Hsu-

Nan, 2006).

The great part of these techniques showed

effective, but rather complex acquisition systems or

high computational time are often required. For this

reason, they are not suitable for in-line inspection. In

order to overcome these drawbacks, AOI solutions

are attractive when the evaluation of a product is

based on the analysis of 2D images. For this aim,

recently, in the field of classification several kinds of

inspection algorithms have been developed by using

fuzzy/rule-based expert systems (Ko, 2000) and

neural approaches (Jagannathan, 1997; Acciani,

2006), transferring the inspection burden to the

phase of image processing. In fact, AOI systems

Acciani G., Brunetti G., Fornarelli G., Giaquinto A. and Maiullari D. (2009).

AOI BASED NEUROFUZZY SYSTEM TO EVALUATE SOLDER JOINT QUALITY.

In Proceedings of the 11th International Conference on Enterprise Information Systems - Artiﬁcial Intelligence and Decision Support Systems, pages

51-54

DOI: 10.5220/0001859300510054

 SciTePress

base their classification on visual criteria like a

human operator does. Fuzzy systems represent a

good tool for reproducing human criteria, offering

the advantage of quantifying experts’ assessment,

whereas neural networks can generalize such

assessment by automating operator’s classification

capabilities.

On the basis of these considerations, in this paper

a neurofuzzy system for solder joint quality

evaluation is proposed. The target of the present

work is to obtain a Quality Index (QI) of a solder

joint in a SMT assembled PCB, starting from the

knowledge of a human assessment. The proposed

system is composed by three modules connected in

series: a feature extraction, a supervised neural

network-based module and a fuzzy one. The

proposed solution offers some interesting

advantages: a complex acquisition system is not

needed, implying reduced equipment costs;

moreover, a low computational time is guaranteed

by the fuzzy module.

2 PROPOSED ARCHITECTURE

The proposed architecture is constituted by three

fundamental modules connected in series: a Feature

Extraction Block, a Neural Module and a Fuzzy one.

The input of the first block is the 256 grey-level

image I of a solder joint, extracted from the image of

a PCB acquired by a camera. This extraction is

conducted by using the procedure described in

(Acciani, 2006). The aim of the Feature Extraction

Block is to evaluate a vector of 18 features, called

geometric-wavelet features (GW) and reported in

(Acciani, 2006), from the image I of a single pin.

Such vector, named GW, revealed efficient in the

classification of solder joints. It can be considered as

composed by two sub-vectors, codifying the

geometric and the wavelet features, respectively.

The former type provides global characteristics of

the image under test, the latter takes into account the

level of similarity among the image under test and a

reference one.

The Feature Extraction Block feeds the

subsequent Neural one, constituted by a Multilayer

Perceptron (MLP) Neural Network, whose outputs

are given by an expert, who is required to provide

his evaluation of how much a solder joint belongs to

each of the following five classes: "Poor",

"Acceptable Poor", "Good", "Acceptable Excessive"

and "Excessive". The defined classes are considered

as fuzzy sets, therefore the expert supplies the degree

of membership of every pin in each set, allowing to

construct the target set of the Neural Block.

Let v =[ v

, v

]

∈ [0,1]

5×1

be the

membership vector related to each pin, where v

, v

and v

represent the degree of

membership of the solder joint in the class "Poor",

"Acceptable Poor", "Good", "Acceptable Excessive"

and "Excessive", respectively. Then, the human

operator identifies the class which the solder joint

belongs to and assigns a degree of membership

d∈[0,1] to the corresponding element of the vector

v. Subsequently, the value (1-d)/2 is assigned to the

vector components corresponding to the classes

which are contiguous to the classified one. If the

expert classifies the soldering as belonging to "Poor"

(respectively "Excessive"), then the value 1-d is

assigned to v

(resp. v

). The remaining elements

of the vector are set to 0. Let v

, i = 1, 2, …, n, be the

vectors computed for each image of a database of n

solder joints, then the matrix V

= [v

, v

, … , v

]∈

[0; 1]

5×n

can be defined. Let F∈ ℜ

18×n

be the matrix

whose columns are the GW vectors, then the

matrices F and V provide proper sets for training the

MLP network. In this way the designed block is able

to reproduce and automate human experts’

assessment on which the quality evaluation of

soldering is based.

The generic output vector v constitutes the input

for the subsequent Fuzzy Block, whose target

consists of supplying an index to express the overall

quality of a soldering. In detail, the universe of

discourse is divided into the five sets Poor,

Acceptable Poor, Good, Acceptable Excessive and

Excessive, whose activation values are given by the

components of vector v. The generic output is

constituted by the vector y = [y

], whose

domain is characterized by four output fuzzy

subsets, defined as Few, To Increase, To Lower,

Too Much, respectively. These sets codify all the

possible situations including cases corresponding to

good quality. Such fuzzy sets are characterized by

four singleton membership functions, centred on the

real values

0123

aaaa

<< in the range [-1,1], as

shown in Figure 1.

Values |a

| ≥ |a

| are chosen, because the case of

“acceptable poor” soldering paste is considered less

desirable than the case of the “acceptable excessive”

one by the experts. The following linguistic rules

map the fuzzy input sets into the output ones:

Rule 1) IF Soldering is Poor THEN (y is Few)

Rule 2) IF Soldering is Acceptable Poor THEN

(y is Few) and (y is To Increase)

AOI BASED NEUROFUZZY SYSTEM TO EVALUATE SOLDER JOINT QUALITY

Rule 3) IF Soldering is Good THEN (y is To

Increase) and (y is To Lower)

Rule 4) IF Soldering is Acceptable Excessive

THEN (y is To Lower) and (y

is Too Much)

Rule 5) IF Soldering is Excessive THEN (y

Too Much).

Subsequently, the elements of the vector y are

defuzzified to compute the Quality Index (QI) using

the well known method of the “centre of gravity“.

The resulting value QI ∈ [-1, 1] describes the quality

of the solder joint under test.

Figure 1: Membership function of output fuzzy sets.

3 EXPERIMENTAL RESULTS

The performances of the proposed neurofuzzy

architecture have been investigated by means of a

database formed by 480 images. In Figure 2(a) the

image of a complete PCB is reported. Figure 2(b)-(f)

shows images of pins that an expert classified as

belonging to the five defined classes.

Figure 2: (a) PCB Sample (b)-(f) Solder joints belonging

to the defined classes.

The input vectors GW for the MLP network are

computed from the images of the database by the

Feature Extraction Block. In order to synthesize the

architectural parameters of the Neural Module, the

Mean Absolute Error (MAE) index is defined as:

MAE = -

∑

being y

and t

the output and the target of the

network under test, respectively, and n the number

of the data samples. The performances of the neural

network have been evaluated by minimizing the

mean value of the MAE index computed over r = 20

trainings with different initial weights. The designed

network has input, hidden and output layers formed

by 18, 12 and 5 neurons, whose activation functions

are logarithmic-sigmoid. The network has been

trained by the Levenberg-Marquardt algorithm,

minimizing the Mean Square Error. In order to show

the capabilities of the neural network in reproducing

the assessment of a human expert, in Figure 3 the

values of the target versus the output of the network

are reported. It is worth to observe that the more

points consolidate on the bisector line the more the

network is able to reproduce the assessment of the

expert.

Figure 3: Values of the MLP target vs MLP output.

Subsequently, the output of the network is

processed by the Fuzzy Block. In Figure 4 the values

of QI, computed for the samples in the considered

database and sorted by the membership to each

defined class, have been reported.

Figure 4: Values of QI computed for the considered

samples.

To evaluate the reliability of the outputs

provided by the whole neurofuzzy architecture, the

values of QI have been partitioned into five hard sets

by the four thresholds th

, th

, and th

which are

computed as:

ICEIS 2009 - International Conference on Enterprise Information Systems

()

PAP

1mm

QI +QI / 2th =

()

AP G

2mm

QI +QI / 2th =

()

GAE

3mm

QI +QI / 2th =

()

AE E

4mm

QI +QI / 2th =

where

PAPGAE E

mm mm m

QI , QI , QI , QI and QI are the mean

QI value of the elements belonging to the class

Poor, Acceptable Poor, Good, Acceptable

Excessive

and Excessive, respectively. Such

partition of the data can be carried out by using these

thresholds as follows: (a) if QI<th

, then soldering is

Poor; (b) if th

<QI<th

, then soldering is

Acceptable Poor; (c) if th

<QI<th

, then soldering

Good; (d) if th

<QI<th

, then soldering is

Acceptable Excessive; (e) if QI>th

, then soldering

Excessive. The obtained partition is compared to

experts' one. In particular, a pin is considered as

correctly classified by the system if it belongs to the

same set when considering both the former and the

latter partition. On the contrary, if this condition is

not satisfied, then a misclassification takes place, as

it is shown by the diamond marks in Figure 4. The

performances of the architecture are measured by

defining the Recognition Rate index as:

TOT

RR = ×100

being N

and N

TOT

the number of correctly

classified cases and the number of the considered

ones, respectively. Values of RR equal to 96.87%

and 95.83% concerning training and testing data

have been obtained. The results can be considered

encouraging, in fact the obtained values show that

the designed neurofuzzy system yields a

classification similar to that given by the experts,

providing a refined evaluation of the solder joints.

4 CONCLUSIONS

In this paper a neurofuzzy architecture for

computing a Quality Index of a solder joint in a

SMT assembled PCB has been proposed. The

system offers some interesting advantages. In

particular, the suggested solution does not need a

complex illumination and positioning system,

implying that the equipment costs could be reduced

and the assessment of a solder joint could be shifted

on the fuzzy evaluation phase. Moreover, the typical

low computational costs of the fuzzy systems could

satisfy urgent time constrains in the in-line detection

of some industrial productive processes. The

proposed architecture provides a refined evaluation

of the solder joints, automating the human expert

classification.

Basing on the obtained results, it can be argued

that the correct working of the proposed system is

due to its capability to reproduce human experts’

modus operandi properly. Therefore, future

developments will be aimed at identifying the

characteristics, that a human operator evaluates in

order to express the assessments of solder joints. As

a consequence, the focus of future works will be

constituted by the identification of the features

which contain sufficient and useful information to

perform a correct diagnosis.

REFERENCES

Krippner, P., and Beer, D., 2004. “AOI Testing Positions

in Comparision”, Circuit Assembly, Apr. 2004, pp. 26-

32.

Manjeshwar, P., Craik, J., Phadnis, S., and Srihari, K.,

2006. ‘Effectiveness Study of an Automated 3D

Laminography X-Ray Inspection System in a High

Volume–Low-Mix SMT line’, The Int. J. of Adv.

Manuf. Technol., vol. 30 (11-12), pp. 1191 – 1201.

Teramoto, A., Murakoshi, T., Tsuzaka, M., and Fujita, H.,

2007. “Automated Solder Inspection Technique for

BGA-Mounted Substrates by Means of Oblique

Computed Tomography”, IEEE Trans. on Electronics

Packaging Manufacturing, vol. 30 (4) , pp. 285 – 292.

Wu, Y. P., Tu, P. L., and Chan, Y. C., 2001. “The effect of

solder paste volume and reflow ambient atmosphere

on reliability of CBGA assemblies,” J. Electron.

Packag., vol. 123 (3), pp. 284–289.

Zhang, L., Ume, I.C., Gamalski, J., and Galuschki K. P.,

2006. “Detection of Flip Chip Solder Joint Cracks

Using Correlation Coefficient and Auto-Comparison

Analyses of Laser Ultrasound Signals”, IEEE Trans.

on Comp. and Pack. Technol., vol. 29 (1), pp 13-19.

Hsu-Nan, Y., Du-Ming, T., and Jun-Yi Y., 2006. “Full-

Field 3-D Measurement of Solder Pastes Using LCD-

Based Phase Shifting Techniques”, IEEE Trans. on

Electron. Packag. Manuf., vol. 29 (1), pp. 50 – 57.

Ko, K.W., and Cho H.S., 2000. “Solder joint inspection

using a neural network and fuzzy rule-based

classification method”, IEEE Trans. on Electron.

Packag. Manuf., vol. 23 (2), pp. 93–103.

Jagannathan, S., 1997. “Automatic Inspection of Wave

Soldered Joints Using Neural Networks”, J. of Manuf.

Syst., vol. 16 (6), pp. 389-398.

Acciani, G., Brunetti, G., and Fornarelli G., 2006. “A

Multiple Neural Network System to Classify Solder

Joints of Integrated Circuits”, Int. J. of Computational

Intelligence Research., vol. 2 (4), pp. 337-348.

AOI BASED NEUROFUZZY SYSTEM TO EVALUATE SOLDER JOINT QUALITY