R. J. Kuo
Department of Industrial Management, National Taiwan University of Science and Technology
No. 43, Section 4, Keelung Road, Taipei, Taiwan
M. J. Wang, T. W. Huang
Department of Industrial Engineering and Management, National Taipei University of Technology
No. 1, Section 3, Chung-Hsiao East Road, Taipei, Taiwan
Tung-Lai Hu
Department of Business Management, National Taipei University of Technology
No. 1, Section 3, Chung-Hsiao East Road, Taipei, Taiwan
Keywords: Clustering analysis, SMT production system, ART2 neural network, Particle swarm optimization algorithm,
Abstract: Surface mount technology (SMT) production system set up is quite time consuming for industrial personal
computers (PC) because of high level of customization. Therefore, this study intends to propose a novel
two-stage clustering algorithm for grouping the orders together before scheduling in order to reduce the
SMT setup time. The first stage first uses the adaptive resonance theory 2 (ART2) neural network for
finding the number of clusters and then feed the results to the second stage, which uses particle swarm K-
means optimization (PSKO) algorithm. An internationally well-known industrial PC manufacturer provided
the related evaluation information. The results show that the proposed clustering method outperforms other
three clustering algorithms. Through order clustering, scheduling products belonging to the same cluster
together can reduce the production time and the machine idle time.
Unlike regular personal computer (PC)
manufacturing, high mix low volume, or high
customization, is one of the characteristics of the
industrial PC industry. It is also the biggest
challenge facing the industrial PC industry. In this
industry, printed circuit board (PCB) assembly is a
fundamental manufacturing process, in which
surface mount technology (SMT) plays a very
important role. By applying an SMT production
system, not only can there be more components on
the limited space of a PCB, but also the production
efficiency and product stability can be enhanced.
However, the high mix low volume production style
is still a difficulty for the SMT production system. In
this production system, production line-change is a
very serious bottleneck since before line-change,
production labor must prepare for the materials and
also bind these materials, which is very time
consuming. If production labor can not complete
these tasks before the next order starts, it would both
cause idle time for these expensive machines and
decrease the production capability utilization.
Currently, this problem is frequently encountered in
the SMT production system, and the only way it can
be handled is to prepare for more materials and
increase the binding labor. Thus, how to reduce the
setup time for SMT line-change operation is a very
important issue.
Kuo R., Wang M., Huang T. and Hu T. (2009).
In Proceedings of the 11th International Conference on Enterprise Information Systems - Artificial Intelligence and Decision Support Systems, pages
DOI: 10.5220/0001860300550060
Therefore, this study applies a clustering
technique to cluster bills of materials (BOMs) for all
orders or products. After clustering, production
engineers can to schedule similar orders together for
production. This can simplify the material-
preparation time in order to reach reducing SMT
line-change time. For the clustering technique, this
study proposes a novel two-stage clusering
algorithm. The first stage employs an adaptive
resonance theory 2 (ART2) neural network to find
the number of clusters and feed it to the second
stage. The second stage applies particle swarm K-
means optimization (PSKO) algorithm to reach the
final solution.
In the model evaluation, data provided by
Advantech Company which is an internationally
well-known industrial PC manufacturer, shows that
the proposed two-stage clustering algorithm is better
than three other algorithms in accuracy. This can
dramatically reduce the SMP production system
setup time and increase the utilization of production
This section briefly presents the general literature
survey for the clustering analysis and particle swarm
optimization algorithm.
2.1 Clustering Analysis
There have been many algorithms being applied in
clustering analysis. With artificial intelligence (AI)
and soft computing, many clustering algorithms
based on these techniques and theories have been
proposed. Thus, the following discussions will focus
on AI related clustering algorithms (Xu and
Wunsch, 2005).
An artificial neural network (ANN) is a system
that has been derived through models from
neurophysiology. ANN-based clustering has been
dominated by self-organizing feature maps (SOM)
and adaptive resonance theory (ART) (Kohonen,
1990, Carpenter and Grossberg, 1987, and Carpenter
and Grossberg, 1987).
Fuzzy clustering is different from hard clustering
since its restriction is relaxed and the object can
belong to all of the clusters with a certain degree of
membership. The methods of fuzzy clustering
include fuzzy C-means (Hoppner et al., 1999) and
fuzzy c-shells (Bezdek and Hathaway, 1992).
Clustering can also be regarded as a category of
optimization problems that use evolutionary
algorithms, like genetic algorithms (GA) or ant
colony optimization algorithms. But the main
drawback of these clustering algorithms is the
process of parameter selection. Many techniques
supporting this method such as genetic K-means
algorithm (Krishna and Murty, 1999), Tabu search
clustering (Al-Sultan, 1995), simulated annealing
clustering (Brown and Huntley, 1992 and Smyth,
1998) and ant colony clustering algorithm (Kuo et
al., 2005b).
Due to limitations of some clustering algorithms,
which need to know the number of clusters before
implementing clustering, a two-stage framework is
proposed here to overcome this problem. The basic
idea of the two-stage clustering algorithm is to first
find the number of clusters using an automatically
clustering algorithm, such as an ART2 neural
network, and then feed this number to the other
clustering algorithm to find the final solution.
Kuo et al. (2002) proposed a two-stage method
which integrates both the SOM and K-means, with
results indicating that the proposed method is much
better than using only SOM or K-means. Then, Kuo
and his colleagues modified the GKA by Krishna
and Murty (1999) and used SOM’s solution as the
initial solution for the modified GKA (Kuo et al.,
2004). The results showed that this method was
better than the previously published method, SOM +
K-means. Kuo and his colleagues also presented a
method integrating of ART2 and a genetic algorithm
(Kuo et al., 2005a), and one that used a different
coding method (Kuo et al., 2006).
2.2 Particle Swarm Optimization
The PSO algorithm shares many similarities with
evolutionary computation techniques such as
Genetic Algorithms (GA). The system is initialized
with a population of random solutions and searches
for optima by updating generations. However, unlike
GA, the PSO algorithm has no evolutionary
operators, such as crossover and mutation. In the
PSO algorithm, the potential solutions, called
particles, move through the problem space by
following the current optimum particles. The main
developments in the PSO algorithm can be generally
divided into three phases as follows.
Xiao et al. (2003) proposed a hybrid clustering
approach based on an SOM neural network. The
proposed algorithm uses a PSO algorithm to evolve
the weight for the SOM neural network. The weights
are trained by the SOM neural network in the first
stage, and in the second stage they are optimized by
the PSO algorithm. The experimental results show
that the hybrid method tries to tune the original
SOM such that it can achieve a better tradeoff
between the average quantization error and the
topographic error.
Merwe and Engelbrecht (2003) proposed two
new approaches using a PSO algorithm to cluster
data. The first method, called the PSO clustering
algorithm, shows how the PSO algorithm can be
used to find the centroids of a user-specified number
of clusters. The second method, called the Hybrid
PSO algorithm, first uses K-means clustering to seed
the initial swarm and then uses PSO algorithm to
refine the clusters formed by K-means. These new
PSO algorithms were evaluated on six data sets, and
compared with the performance of K-means
clustering. Results show that both PSO clustering
techniques have much potential. Chen and Ye also
proposed a clustering analysis algorithm based on
the PSO algorithm (Chen and Ye, 2004). But the
concept of that algorithm is similar to others in the
literature (Merwe and Engelbrecht, 2003), and its
main difference is only in the fitness function.
Finally, the experimental results obtained using four
artificial data sets show the algorithm proposed here
has better performance than K-means and fuzzy C-
This section presents the proposed order clustering
system which includes data collection and
transformation, principle component analysis, and
clustering analysis. The following subsections
briefly discuss these three components.
3.1 Data Collection
and Transformation
In order to classify the product group for the up
coming orders belonging to it, all the previous orders
must first be collected and make clustering analysis
made for them. Basically, for most companies, these
data can be retrieved from the enterprise resources
planning system. Then, the bill of materials (BOM)
for each order is used as the feature for each order,
or product.
3.2 Principle Component Analysis
Due to the vast number of materials used for each
order, it is very time consuming to use these data for
clustering analysis, and it is more feasible to
condense these data in advance. A statistical method,
principle component analysis, is employed for this
purpose so that each order will have a limited
number of features. This can result in shorter
computational time and without influencing the
computational results.
3.3 Clustering Analysis
After implementing principle component analysis,
the data obtained is applied for clustering analysis
using a two-stage clustering with ART2 neural
network and a proposed particle swarm K-means
optimization algorithm as well.
3.3.1 ART2 Neural Network
ART2 architectures are designed for processing
analog as well as binary input patterns. The ART2
neural network consists of F1 and F2 layers. There
are seven nodes in the F1 layer (W, X, U, V, P, Q).
The input signal is processed by the F1 layer and
then is passed from the bottom to top value (b
). The
result of the bottom-to-top value is an input signal of
F2 layer. The nodes of F2 layer compete with each
other to produce a winning unit and the winning unit
returns the signal to the F1 layer. The match value is
then calculated with top to bottom value (t
) in the
F1 layer and compared with the vigilance value. If
the match value is greater than the vigilance value,
then the weight of b
and t
is updated. Otherwise,
the reset signal is sent to the F2 layer and the
winning unit is inhibited. After inhibition, the other
winning unit will be found in the F2 layer. If all of
the F2 layer nodes are inhibited, the F2 layer will
produce a new node and generate the initial
corresponding weights to the new node. The detailed
learning procedure can be found in [Grossbert
3.3.2 PSKO Algorithm
Particle swarm optimization, like GA, is a
population-based stochastic search process. The
algorithm maintains a population of particles, where
each particle represents a potential solution to an
optimization problem. By referring to Krishna and
Murtys concept (1999) of integrating GA with K-
means, this article proposes to integrate K-means
with PSO algorithm (Merwe and Engelbrecht,
2003). It is called the particle swarm K-means
optimization (PSKO) algorithm and the
computational procedures are as follows.
Step 1: Set up parameters including population
size (number of particles), maximum
ICEIS 2009 - International Conference on Enterprise Information Systems
velocity, V
, inertia weight, W, and two
learning factors, c
and c
Step 2: Initialize each particle randomly with
initial position X
and velocity V
. In
PSO clustering, we need in advance to
know the number of clusters, k. In this
study ART2 neural network will provide
this information to the PSKO algorithm.
Since each particle is a vector containing
k cluster centroids, each clusters
position, X
, can be represented as:
1 ikijiid
where z
denotes the jth cluster’s
centroid for the ith particle.
Step 3: Calculate fitness value for each particle.
where x denotes the data vector, n
denotes the number of data for jth cluster
of the ith particle, and
the Euclidean distance of data vector to
all cluster centroids.
Step 4: Update the local best, P
, and global best,
Step 5: According to the best position of P
, update the velocity and position for
each particle using Equations (3) and (4).
It should be noted that V can not be
larger than V
2211 idgdidid
XPrandcXPrandcVWV ++=
VXX +=
where c
and c
are learning factors,
respectively, while rand
and rand
denote random numbers between (0, 1).
Step 6: Calculate the Euclidean distance of every
piece of data x to all cluster centorids for
each particle.
Step 7: For each particle, assign each piece of
data, x, to the cluster with the closest
cluster centroid.
Step 8: Recalculate the cluster centroid vectors for
every particle, using
Step 9: Stop if the specified number of iterations
is satisfied; otherwise, go back to Step 3.
This section will apply the proposed clustering
method for order clustering in order to reduce the
SMT production system setup time. Advantech
Company provided the related data for assessment.
The setup of production line change for different
orders has become a critical bottleneck in SMT
production system because that implementing
material-preparation and material-binding tasks by
hand for production line change is very time
consuming. If this can not be ready before next order
starts production, it will cause machines idle time
and result in low production capability utilization.
Therefore, we apply a two-stage clustering method
(Kuo et al. 2002) to group similar products or orders
together and find the same materials (shared
materials) that will be used for all the products in the
same group. This enables the production engineers
to put the same needed materials in the same
material positions for SMT retrieval equipment, so
production engineers can arrange for similar
products to be produced together. Thus, the material
preparation operation can be simplified in order to
reduce the SMT setup time.
4.1 Data Collection
A material report was provided by Advantech
Company. Using ACCESS SQL query, there is a
total of 2826 different products, or orders, and a total
of 2517 materials used. In order to cluster these
products, it is necessary to filter the noise of product
and material data and then transform them into a
2826×2517 two-dimension matrix.
4.2 Principle Component Analysis
That there are 2517 features for product feature
matrix and it is very time consuming for clustering
analysis if there are many features. In order to
reduce the computational time and also maintain the
solution quality, this study uses Matlab 6.5.1 to
make the principle component analysis for the
product feature matrix in order to extract the
dimensions for the principle component factors. The
selection of factor is according to the Eigne value
which should be larger than 1. After analysis, there
are a total of 200 principle component factors. The
cumulative explanation variance is 81.98%.
4.3 Clustering Analysis
After reducing the original feature matrix with size
of 2826×2517 to 2826×200 through principle
component analysis, it is necessary to know the
number of clusters in advance. Based on our
previous research (Kuo et al., 2005), this study
applies a two-stage clustering method, ART2+PSKO
which is a kind of method for processing gray value
in the first stage, ART2 automatically finds the
number of clusters, while the second stage uses the
PSKO algorithm to find the final solution.
4.3.1 Determination of Cluster Number
The ART2 algorithm was coded by using Matlab
6.5.1. Since in ART2 algorithm the number of
clusters is determined by the vigilance value, once it
is well determined ART2 can automatically cluster
the data. Thereafter, the second stage employs K-
means, PSO clustering, hybrid PSO, and PSKO
algorithms to find the final solutions. Since the
vigilance value will dramatically affect the
clustering outcome, this study uses Wilk’s Lambda
value as the indicator for determining the number of
clusters. Wilk’s Lambda is frequently applied by
MANOVA as the indicator for determining the
number of clusters. If Wilk’s Lambda value
suddenly increases in two different numbers of
clusters, then the number of clusters before variance
can be treated as the best number of clusters.
Theoretically, Wilk’s Lambda value is defined as:
Lambda '
sWilk =
where SS
and SS
are the within-cluster and
total variances, respectively.
Table 1 lists the corresponding Wilk’s Lambda
values for different vigilance values. The result
shows that 35 is the best number of clusters.
Table 1: Wilk’s Lambda value of ART2.
Lambda value
0.9372 43 0.67823
0.9347 41 0.685
0.932 39 0.69358
0.9295 37 0.69955
0.972 35 0.70688
0.926 33 0.72989
4.3.2 Comparison of Different Clustering
Based on the ART2 algorithm’s result, four different
clustering algorithms are used to further find the
final solution for comparison. Table 2 depicts the
Sum of Euclidean Distances (SEDs) of four
algorithms, which indicates that the ART2+PSKO
algorithm has the smallest SED value, 83373.859.
Table 2: SED values for each clustering algorithm.
SED 90276 94407 90153 83373
4.4 Shared Materials
According to the clustering results of ART2+PSKO,
the products are grouped into thirty five clusters.
Totally, there are eleven clusters whose numbers of
shared materials are over one hundred.
4.5 Performance Evaluation
This study simulates the production line-change
efficiency through on-field collection of the
production plan. We first use the regular scheduling
method to schedule the jobs for two days. Then, this
study further groups similar products belonging to
similar clusters to be scheduled together for
production in order to reduce the SMT line-change
time. According to these simulation results, we
found that using the proposed ART2 PSKO
clustering algorithm really can efficiently reduce
material binding time by 10.9% due to considering
shared materials. Also, total production time
decreased 7.3%, and the most dramatic improvement
was the 89.3% reduction of machine idle time.
This study has demonstrated that the use of a
clustering technique can reduce both production
time and machine idle time since similar products or
orders are scheduled for production together. In
addition, this study also proposed a novel clustering
algorithm that integrates both the K-means
algorithm and the PSO algorithm for order
clustering. Integration of the PSO clustering
ICEIS 2009 - International Conference on Enterprise Information Systems
algorithm and the K-means algorithm gives the
particles both global and local search capabilities.
The results show that its performance is better than
those of other three clustering algorithms for
Advantech Company’s order clustering problem.
K. Al-Sultan, “A Tabu search approach to the clustering
problem,” Pattern Recognition, Vol. 28, No. 9, 1995,
pp. 1443–1451.
J. Bezdek and R. Hathaway, “Numerical convergence and
interpretation of the fuzzy c-shells clustering
algorithms,” IEEE Transaction on Neural Networks,
Vol. 3, No. 5, 1992, pp. 787–793.
D. Brown and C. Huntley, “A practical application of
simulated annealing to clustering,” Pattern
Recognition, vol. 25, no. 4, 1992, pp. 401–412.
G. Carpenter and S. Grossberg, “A massively parallel
architecture for a self-organizing neural pattern
recognition machine,” Computer Vision Graphic and
Image Process, Vol. 37, 1987, pp. 54–115.
G. Carpenter and S. Grossberg, “ART2:Self-organization
of stable category recognition codes for analog input
patterns,” Applied Optics, Vol.26, No.23, 1987,
C. Y. Chen and F. Ye, “Particle swarm optimization
algorithm and its application to clustering analysis,
2004 IEEE International Conference on Networking,
2004, pp. 789 – 794.
M. Clerc, “The swarm and the queen: towards a
deterministic and adaptive particle swarm
optimization,” Evolutionary Computation, 1999.
R. Eberhart and J. Kennedy, “A new optimizer using
particle swarm theory,” Proceedings of the Sixth
International Symposium on Micro Machine and
Human Science, 1995, pp.39-43.
S. Grossberg, “Adaptive pattern recognition and universal
encoding II: Feedback, expectation, olfaction, and
illusions,” Biological Cybernetics, Vol. 23, 1976,
F. Höppner, F. Klawonn and R. Kruse, Fuzzy Cluster
Analysis: Methods for Classification, Data Analysis,
and Image Recognition. New York: Wiley, 1999.
L. Kaufman and P. Rousseeuw, Finding Groups in Data:
An Introduction to Cluster Analysis: Wiley, 1990.
T. Kohonen, “The self-organizing map,” Proc. IEEE, Vol.
78, No. 9, 1990, pp.1464–1480.
K. Krishna and M. N. Murty, “Genetic K-means
algorithm,” IEEE Transactions on Systems, Man and
Cybernetics, Vol. 29, No. 3, 1999, pp. 433–439.
R. J. Kuo, L.M. Ho and C. M. Hu, “Integration of Self-
Organizing Feature Map and K-Means Algorithm for
Market Segmentation,” International Journal of
Computers and Operations Research, 29, 2002, pp.
R. J. Kuo, Chang, K., and Chien S.Y., “Integration of self-
organizing feature maps and genetic algorithm based
clustering method for market segmentation,” Journal
of Organizational Computing and Electronic
Commerce, Vol. 14, No. 1, pp.43-60, 2004.
R. J. Kuo, J. L. Liao and C. Tu, "Integration of ART2
neural network and genetic K-means algorithm for
analyzing Web browsing paths in electronic
commerce," Decision Support Systems Vol. 40, Issue:
2, 2005a, pp. 355-374.
R. J. Kuo, Wang, H. S., Hu, Tung-Lai, and Chou, S. H.,
“Application of Ant K-means on Clustering Analysis
in Data Mining,“ International Journal of Computers
and Mathematics with Applications,50, pp.1709-1724,
November-December, 2005b.
R. J. Kuo, An, Y.L., Wang, H.S., and Chung, W.J.,
“Integration of Self-Organizing Feature Maps Neural
Network and Genetic K-means Algorithm for Market
Segmentation," International Journal of Expert
Systems with Applications, 30(2), pp.313-324,
February, 2006.
D. W. Van der Merwe and A. P. Engelbrecht, “Data
clustering using particle swarm optimization,” The
2003 Congress on Evolutionary Computation, 2003,
pp. 215 - 220.
Y. Shi and R. Eberhart, “A modified particle swarm
optimizer,” Proceedings of the IEEE International
Conference on Evolutionary Computation, 1998a, pp.
Y. Shi and R. Eberhart, “Parameter selection in particle
swarm optimization,” In Evolutionary Programming
VI/: Proc. EP98, New York: Springer-Verlag, 1998b.
P. Smyth, “Model selection for probabilistic clustering
using cross validated likelihood,” Statistics and
Computer, Vol. 10, 1998, pp. 63–72.
X. Xiao, E. R. Dow, R. Eberhart, Z. B. Miled and R. J.
Oppelt, “Gene clustering using self-organizing maps
and particle swarm optimization,” Proceedings of the
International Parallel and Distributed Processing
Symposium, 2003, pp.22-28.
R. Xu and D. Wunsch, “Survey of clustering algorithms,”
IEEE Transactions on Neural Networks, Vol.16, Issue
3, 2005, pp. 645 – 678.