Performance and Scalability of Particle Swarms with Dynamic and

Partially Connected Grid Topologies

Carlos M. Fernandes, Agostinho C. Rosa

Laseeb: Evolutionary Systems and Biomedical Engineering, Technical University of Lisbon, Lisbon, Portugal

Juan L. J. Laredo

Faculty of Sciences, Technology and Communications, University of Luxembourg, Walferdange, Luxemburgo

Carlos Cotta

Departamento de Lenguages y Ciencias de la Computación, University of Malaga, Malaga, Spain

J. J. Merelo

Departamento de Arquitectura y Tecnología de Computadores, University of Granada, Granada, Spain

Keywords: Particle Swarm Optimization, Population Structure.

Abstract: This paper investigates the performance and the scalability of dynamic and partially connected 2-

dimensional topologies for Particle Swarms, using von Neumann and Moore neighborhoods. The particles

are positioned on 2-dimensional grids of nodes, where they move randomly. The von Neumann or Moore

neighborhood is used to decide which particles influence each individual. Structures with growing size are

tested on a classical benchmark and compared to the lbest, gbest and the standard von Neumann and Moore

configurations. The results show that the partially connected grids with von Neumann neighborhood

structure perform more consistently than the other strategies, while the Moore partially connected structure

performs similarly to the standard Moore configuration. Furthermore, the proposed structure scales similarly

or better than the standard configuration when the problem size grows.

1 INTRODUCTION

The Particle Swarm Optimization (PSO) algorithm

(Kennedy and Eberhart, 1995) is a population-based

metaheuristics that was inspired by the social

behavior of bird flocks and fish schools. Since its

inception, PSO has been applied with success to a

number of problems and motivated several lines of

research that investigate its main working

mechanisms. One of these research lines deals with

the population topology, which is the structure that

defines the connections between the particles and the

flow of information through the population.

Therefore, the chosen structure deeply affects the

convergence skills of the algorithm.

In PSO, the particles are interconnected so that

they acquire information on the regions explored by

other particles. In fact, it has been claimed that the

uniqueness of the algorithm lies in the dynamic

interactions of the particles (Kennedy and Mendes,

2002). These networks of individuals may be of any

possible structure, from sparse to dense (or even

fully connected) graphs, with different degrees of

connectivity and clustering in between. The most

commonly used PSO population structures are the

lbest (which connects the individuals to a local

neighborhood) and the gbest (in which each particle

is connected to every other individual). These

topologies are well-studied and the major

conclusions are that gbest is fast but is frequently

trapped in local optima, while lbest is slower but

converges more often to the neighborhood of the

global optima. Since the first experiments on these

47

Fernandes C., C. Rosa A., L. J. Laredo J., Cotta C. and J. Merelo J..

Performance and Scalability of Particle Swarms with Dynamic and Partially Connected Grid Topologies.

DOI: 10.5220/0004558600470055

In Proceedings of the 5th International Joint Conference on Computational Intelligence (ECTA-2013), pages 47-55

ISBN: 978-989-8565-77-8

Copyright

c

2013 SCITEPRESS (Science and Technology Publications, Lda.)

topologies, researchers have tried to design

structures that hold both lbest and gbest qualities.

Some studies also try to understand what makes a

good structure. In (Kennedy and Mendes, 2002), for

instance, Kennedy and Mendes investigate several

types of topologies and recommend the use of a

lattice with von Neumann neighborhood (which

results in a connectivity degree between that of lbest

and gbest).

This paper extends the concept of von Neumann

configuration and investigates the behavior of a

partially connected topology with von Neumann

neighborhood, where not all the neighbors’ cells of a

given one are occupied. A similar structure with

Moore neighborhood is also tested and compared to

the standard Moore configuration. The particles are

distributed on a grid of nodes. The size of the grid is

set so that the number of nodes is larger than the

number of particles. The particles are placed

randomly on the grid and a simple set of rules guides

their movements through the nodes during the run.

The population structure is defined by the von

Neumann or Moore neighborhood between the

nodes, which means that the degree of connectivity

of each particle varies between 1 and 5 during the

run, for the von Neumann version, and between 1

and 9, for the Moore. Preliminary tests are

conducted with local neighborhood random

structures, that is, the particles move randomly

through the grid, choosing between free adjacent

nodes.

The structures are tested on a classical

benchmark test set and compared to the lbest, gbest

and standard von Neumann and Moore

configurations. The results show that the partially

connected von Neumann structure with random

movement is able to improve the standard

configuration. Furthermore, the proposed structure

performs more consistently than the other

topologies. It is believed that these results, together

with the simplicity of the approach and its potential

as a basis for more complex movement rules (based

on fitness or Euclidean distance between the

particles, for instance) validate this study.

The present work is organized as follows. The

next section briefly describes the PSO and its

topologies, while giving a general overview on

previous studies of population structures for PSO.

Section 3 describes the random partially connected

structures used in this investigation. Section 4

describes the experiments and discuses the results.

Finally, Section 5 concludes the paper and outlines

future lines of research.

2 PARTICLE SWARMS AND

POPULATION STRUCTURE

PSO is a population-based algorithm in which a

group of solutions travels through the search space

according to a set of rules that favor their movement

towards optimal regions of the space. The algorithm

is described by a simple set of equations that define

the velocity and position of each particle. The

position vector of the i-th particle is given by

,

,

,

,…

,

), where is the dimension of

the search space. The velocity is given by

,

,

,

,…

,

). The particles are evaluated with a

fitness function

in each time step and then

their positions and velocities are updated by:

,

,

1

,

,

1

,

,

1

(1)

,

,

1

,

(2)

were

is the best solution found so far by particle

and

is the best solution found so far by the

neighborhood. Parameters

and

are random

numbers uniformly distributed in the range 0,1] and

and

are acceleration coefficients that tune the

relative influence of each term of the formula. The

first term, influenced by the particle’s best solution

found so far, is known as the cognitive part, since it

relies on the particle’s own experience. The last term

is the social part, since it describes the influence of

the community in the velocity of the particle.

In order to prevent particles from stepping out of

the limits of the search space, the positions

,

of

the particles are limited by constants that, in general,

correspond to the domain of the problem:

,

∊

,

. Velocity may also be limited

within a range in order to prevent the explosion of

the velocity vector:

,

∊

,

.

Usually, .

Although the classical PSO may be very efficient

on numerical optimization, it requires a proper

balance between local and global search, as it often

gets trapped in local optima. In order achieve a

better balancing mechanism, Shi and Eberhart

(1998) added the inertia weight, that allows a fine-

tuning of the local and global search abilities of the

algorithm. The modified velocity equation is:

,

.

,

1

,

,

1

,

,

1

(3)

By adjusting (usually within the range [0, 1.0])

together with the constants

and

, it is possible to

IJCCI2013-InternationalJointConferenceonComputationalIntelligence

48

balance exploration and exploitation abilities of the

PSO.

The neighborhood of the particle (which defines

in each time-step the value of

) is a key factor in

the performance of PSO. Most of the PSOs use one

of two simple sociometric principles for defining the

neighborhood network. One connects all the

members of the swarm to one another, and it is

called gbest, were g stands for global. The degree of

connectivity of gbest is , where n is the

number of particles. The other typical configuration,

called lbest (where l stands for local), creates a

neighborhood that comprises the particle itself and

its nearest neighbors. The most common lbest

topology is the ring structure, in which the particles

are arranged in a ring structure (resulting in a degree

of connectivity 3, including the particle).

As stated above, the topology of the population

affects the performance of the PSO and one must

chose the configuration according to the target-

problem. Furthermore, each topology has its own

typical behavior and its choice may also depend on

the objectives or tolerance of the optimization

process. Since all the particles are connected to

every other and information spreads easily through

the network, the gbest topology is known to

converge fast but unreliably (it often converges to

local optima). The lbest converges slower than the

gbest structure because information spreads slower

through the network. However, and for the same

reason, it is also less prone to converge prematurely

to local optima.

In summary, the choice of the structure affects

the performance and in-between the ring structure

with 3 and the gbest with there are

several types of structure, each one with its

advantages on a certain type of scenarios.

Sometimes it is not possible to choose the best

configuration: the structure of the problem may be

unknown, or the time requirements do not permit

preliminary tests. Therefore, the research community

has dedicated substantial efforts on studying the

properties of PSO’s population structures.

In 2002, Kennedy and Mendes (Kennedy and

Mendes, 2002) published an exhaustive study on

population structures for PSO. They tested several

types of structures, including the lbest, gbest and

Von Neumann configuration. They also tested

populations arranged in graphs that were randomly

generated and optimized to meet some criteria. They

concluded that when the configurations were ranked

by the performance at 1000 iterations the structures

with k = 5 perform better, but when ranked

according to the number of iterations needed to meet

the criteria, configurations with higher degree of

connectivity perform better. These results are

consistent with the premise that low connectivity

favors robustness, while higher connectivity favors

convergence speed (at the expense of reliability).

Amongst the large set of graphs tested in (Kennedy

and Mendes, 2002), the Von Neumann configuration

performed more consistently, and in the conclusions

the authors recommend its use.

In Parsopoulos and Vrahatis proposed a unified

PSO (UPSO) which combines both the gbest and

lbest configurations. Equation 1 is modified in order

to include a term with

and a term with

. A

parameter balances the weight of each term. The

authors argue that the proposed scheme exploits the

good properties of gbest and lbest. The same

algorithm was later applied to dynamic optimization

problems (Parsopoulos and Vrahatis, 2005).

Peram et al., (2003) proposed the fitness–

distance-ratio-based PSO (FDR-PSO). The

algorithm defines the “neighborhood” of a particle

as its closest particles in the population (measured

in Euclidean distance). A selective scheme is also

included: the particle selects near particles that have

also visited a position of higher fitness. The

algorithm is compared to a standard PSO and the

authors claim that FDR-PSO performs better on

several test functions. However, the FDR-PSO is

compared only to a gbest configuration, which is

known to converge frequently to local optima in the

majority of the functions of the test set.

More recently, a comprehensive-learning PSO

(CLPSO) (Liang et al., 2006) was proposed. Its

learning strategy abandons the global best

information and introduces a complex and dynamic

scheme that uses all other particles’ past best

information. CLPSO can significantly improve the

performance of the original PSO on multimodal

problems.

More complex strategies deal with the population

in a centralized manner. For instance, in (Hseig et

al., 2009), the PSO varies the size of the swarm

during the run, while running a solution-sharing

scheme that, like in (Liang et al., 2006), uses the

past best information from every particle.

This work uses a 2-dimensional framework to

force a dynamic behavior in the population structure

and variability in the connectivity degree. The main

objective is to search for a good compromise

between high and low connectivity schemes, using

dynamic connections and local interactions provided

by the supporting framework. Since the Von

Neumann configuration was recommended in

(Kennedy and Mendes, 2002), we use it as a base-

PerformanceandScalabilityofParticleSwarmswithDynamicandPartiallyConnectedGridTopologies

49

structure, but we also test a Moore-based structure.

3 PARTIALLY CONNECTED

STRUCTURES

This paper proposes a framework for partially

connected 2-dimensional PSO population structures.

In the beginning of the run, the particles are

randomly distributed on a 2-dimensional toroidal

grid of nodes with size , where is

the swarm size. In each time-step, each particle

moves randomly to an adjacent free node. The

candidate nodes are defined by the Moore

neighborhood. If a particle is surrounded by other

particles (i.e., all the nodes in the particle’s Moore

neighborhood are occupied by other particles), it

remains in the same site until a node in the

neighborhood is freed.

The configuration of the swarm on the grid in

each time-step defines the

positions in Equation

1. If the best position found so far by any individual

in the von Neumann (or Moore) neighborhood of the

particle is better than the current

, then the new

is set to that position.

The particles are supplied with a kind of

memory: while a new

is not transmitted to the

particle by one of its current neighbors, the particle

continues to update its velocity and position with the

previous

, which may correspond to a particle that

is no longer in its neighborhood. On the other hand,

the particle is no longer connected to the particle that

transmitted the

value, and if that particle visits a

better position, it will not be transmitted to the

individual.

With the 2-dimensional framework, the

connectivity is limited by the neighborhood. Please

note that the most commonly used population

topologies may be configured by this model: lbest is

configured by a one-dimensional lattice with size

1, with ; the standard von Neumann and

Moore configurations are described by a grid with

size and von Neumann or Moore

neighboord with Manhattan distance 1; finally,

a gbest configuration may modeled by setting

with Moore neighborhood with range

/21.

This paper studies the performance of structures

with growing size. The particles are allowed to move

within a Moore neighborhood with range 1. The

interaction is defined by the von Neumann

neighborhood with Manhattan distance 1. The

dynamic particle swarm on partially connected grid

is summarized in Table 1.

Table 1: PSO on a dynamic and partially connected grid.

PSO on a partially connected random structure

1. For each particle 1→:

1.1. Initialize particle

1.2. Evaluate particle’s position

:

1.3. Set

2. Set grid size:

3. Place the particles randomly on the grid

4. For each particle 1→

4.1. If the fitness of the best position found so far

b

y any of the

p

articles in the Von Neumann or Moore neighborhood o

f

p

article is better than

, then

4.2. Choose randomly a free node in the Moore neighborhood an

d

move the particle to that node.

5. For each particle

5.1. Update velocity and position using equations 2 and 3.

5.2. Evaluate particle’s position

:

5.2. If

, then

5. If stop criterion not met, go to 4

4 EXPERIMENTS AND RESULTS

This section describes the experiments and

comparisons between the different population

structures. The connectivity degree of the proposed

dynamic and partially connected topology is given,

as well as a simple scalability test that aims at

investigating the performance of the partially

connected von Neumann topology with growing

problem size.

4.1 Performance Analysis: Von

Neumann Neighborhood

For testing the various topologies, an experimental

setup was constructed with five benchmark

unimodal and multimodal functions that are

commonly used for investigating the performance of

PSO (see (Kennedy and Mendes, 2002);

(Parsopoulos and Vrahatis, 2004) and (Trelea,

2003), for instance). The functions are described in

Table 2. The optimum (minimum) of all functions is

located in the origin with fitness 0. The dimension

of the search space is set to 30 (except

Schaffer, with 2 dimensions).The population size

is set to 40. The acceleration coefficients were set to

1.494 and the inertia weight is 0.729, as in Trelea

(2003) is defined as usual by the domain’s

upper limit and . A total of 50 runs

for each experiment are conducted. Asymmetrical

initialization was used (the initialization range for

each function is given in Table 2).

Two sets of experiments were conducted. In the

IJCCI2013-InternationalJointConferenceonComputationalIntelligence

50

first set, the algorithms were run for a limited

amount of iterations (3000 for

and

, 10000 for

,

and

) and the fitness of the best solution

found was averaged over the 50 runs. In the second

set of experiments the algorithms were all run for

20000 iterations or until reaching a stop criterion.

The criteria were taken from (Kennedy and Mendes,

2002) and are given in Table 2. The number of

iterations required to meet the criterion was recorded

and averaged over the 50 runs. A success measure

was defined as the number of runs in which an

algorithm attains the fitness value established as the

stop criterion. These experiments are similar to those

described by Kennedy and Mendes (2002).

Table 2: Benchmarks for the experiments. Dynamic range,

initialization range and stop criteria.

function

mathematical

representation

Rangeof

search/

Rangeof

initialization

stop

Sphere

f

1

100,100

(50,100

0.01

Rosenbrock

f

2

100

1

100,100

15,30

100

Rastrigin

f

3

10cos

2

10

10,10

2.56,5.12

100

Griewank

f

4

1

1

4000

cos

√

600,600

300,600

0.05

Schaffer

f5

0.5

sin

0.5

1.00.001

100,100

15,30

0.00001

PSOs with lbest, gbest and Von Neumann

configurations were tested on the five benchmark

problems. Then, partially connected structures with

size 77, 88,99 and1010 were also

tested. The experiments return three independent

performance metrics: best fitness, iterations to a

solution, and success rate. It is difficult to compare

all the versions of the algorithms in all the functions

considering the complete set of metrics. Success rate

and iterations to a solution, for instance, are

particular difficult to compare, because an algorithm

may be very fast in meeting the criteria, while

meeting it in a few number of runs. Therefore, we

start by comparing each configuration in each

function.

Table 3 and Table 4 compare the von Neumann

standard configuration with partially connected von

Neumann structures. Table 3gives the averaged best

fitness found by the swarms. Table 4 gives, for each

algorithm and each function, the averaged number of

iterations required to meet the criterion, and the

number of runs in which the criterion was met.

An inspection of the tables shows that some

partially connected Neumann structures are able to

improve the von Neumann configuration in the

majority of the problems.

Table 3: Von Neumann topologies. Best fitness values

averaged over 50 runs.

f

1

f

2

f

3

f

4

f

5

VN

1.05e‐35 1.31e+01 6.99e+01 6.25e‐03 1.94e‐04

±1.06e‐35 ±2.16e+01 ±1.83e+01 ±8.23e‐03 ±1.37e‐03

VN

(7×7)

2.69e‐39 1.00e+01 7.19e+01 7.73e‐03 9.72e‐04

±6.81e‐39 ±1.14e+01 ±1.59e+01 ±8.57e‐03 ±2.94e‐03

VN

(8×8)

9.37e‐38 1.41e+01 6.87e+01 7.14e‐03 1.94e‐04

±2.29e‐37 ±2.52e+01 ±1.93e+01 ±1.00e‐02 ±1.37e‐03

VN

(9×9)

9.13e‐37 9.72e+00 6.89e+01 7.68e‐03 1.94e‐04

±2.10e‐36 ±1.88e+01 ±1.71e+01 ±9.56e‐03 ±1.37e‐03

VN

(

10×10

)

7.66e‐36 1.12e+01 6.66e+01 6.40e‐03 1.94e‐04

±2.10e‐36 ±2.16e+01 ±1.94e+01 ±7.69e‐03 ±1.37e‐03

Table 4: Von Neumann topologies. Iterations to a solution

averaged over 50 runs and number of successful runs.

f

1

f

2

f

3

f

4

f

5

VN

489.86 1443.24 748.98 458.36 454.56

±18.55

(50)

±1547.11

(50)

±86.20

(49)

±29.10

(50)

±659.27

(50)

VN

(7×7)

444.50 1432.20 267.00 408.80 309.42

±23.19

(50)

±1845.74

(50)

±78.12

(47)

±25.45

(50)

±425.56

(45)

VN

(8×8)

458.16 2135.12 278.39 421.24 299.92

±19.44

(50)

±2417.81

(50)

±87.21

(46)

±26.88

(50)

±461.57

(49)

VN

(9×9)

474.96 1589.56 314.43 450.56 264.80

±22.60

(50)

±2137.00

(50)

±81.37

(49)

±54.45

(50)

±395.90

(49)

VN

(10×10)

492.32 2416.00 320.63 452.60 206.94

±23.47

(50)

±2069.21

(50)

±69.97

(48)

±24.96

(50)

±196.17

(49)

The structure with size 99, for instance,

improves the standard configuration fitness in

functions

,

,

. In

the standard structure is

better, while in

the result is the same. As for the

PerformanceandScalabilityofParticleSwarmswithDynamicandPartiallyConnectedGridTopologies

51

average iterations to a solution, the 9×9 structure is

faster than the standard von Neumann configuration

in every function except

.

The 99 grid has 81 nodes, which is

approximately twice the number of particles in the

swarm. This ratio gave good results throughout the

test set. The ratio can also be adjusted for optimal

performance. However, in order to avoid introducing

extra parameters that require tuning, it is better to

analyze the results and establish a consistent size

that performs well throughout a wide range of

scenarios. For the moment, and according to the

results attained in the five-function benchmark, we

suggest a 1:2 ratio between the size of the swarm

and the size of the grid.

Non-parametric Mann–Whitney U statistical

tests (with 0.05 level of significance) comparing the

fitness values attained by each configuration in each

function return the following results: the 99

structure is significantly better than the standard

configuration on

; in the remaining functions the

two topologies are statistically equivalent.

Applying the Mann–Whitney U tests to the

iterations metrics, the conclusions are that the 99

structure is statistically better on

,

,

and

.

The algorithms are statistically equivalent in

.

Therefore, the partially connected structure

significantly improves the performance of the

standard von Neumann configuration in every

function except

(in which the algorithms were

found to be statistically equivalent in both fitness

and convergence speed).

Table 5 and Table 6compare the 99partially

connected von Neumann structures with the lbest

and gbest strategies. The proposed structure is able

to improve lbest fitness values in

,

,

and

; in

and

the differences are statistically significant.

The differences in

are also significant but in this

case lbest is better. As for the average iterations for a

solution, the partially structured Von Neumann

structure improves lbest in every function, with

statistical differences between the results.

Table 5: lbest, gbest and 99 partially connected von

Neumann topology. Best fitness values averaged over 50

runs.

f

1

f

2

f

3

f

4

f

5

lbest

2.61e‐25 1.40e+01 1.07e+02 4.93e‐04 3.89e‐04

4.33e‐25 3.53e+01 2.23e+01 1.99e‐03 1.92e‐03

gbest

4.00e+03 4.91e+00 1.05e+02 5.42e+01 2.33e‐03

6.06e+03 1.26e+01 2.89e+01 6.82e+01 4.19e‐03

VN

(9×9)

9.13e‐37 9.72e+00 6.89e+01 7.68e‐03 1.94e‐04

±2.10e‐36 ±1.88e+01 ±1.71e+01 ±9.56e‐03 ±1.37e‐03

Table 6: lbest, gbest and 99 partially connected Von

Neumann topology Iterations to a solution averaged over

50 runs and number of successful runs.

f

1

f

2

f

3

f

4

f

5

lbest

662.30 1800.69 2014.77 618.22 708.08

±21.81

(50)

±1650.07

(49)

±2331.92

(22)

±31.87

(50)

±849.52

(50)

gbest

489.86 891.42 211.13 315.08 395.05

±18.55

(50)

±1066.82

(50)

±77.46

(23)

±56.67

(24)

±795.04

(40)

VN

(9×9)

474.96 1589.56 314.43 450.56 264.80

±22.60

(50)

±2137.00

(50)

±81.37

(49)

±54.45

(50)

±395.90

(49)

Table 7: Iterations to a solution averaged over 50 runs and

number of successful runs.

f

1

f

2

f

3

f

4

f

5

lbest

6.50e+02 1.80e+03 2.01e+03 5.94e+02 3.87e+02

gbest

3.53e+02 8.05e+02 2.02e+02 3.15e+02 3.95e+02

VN

4.79e+02 1.32e+03 2.78e+02 4.36e+02 2.40e+02

VN (9×9)

4.63e+02 1.40e+03 2.51e+02 4.20e+02 1.51e+02

The differences between the best fitness values

attained by gbest and 99 structure are statistically

different for every function. von Neumann 99 is

better in

,

,

and

, while gbest is better in

.

Comparing the proposed structure with gbest is not

trivial because gbest fails very often in meeting the

stop criteria. It is faster in three functions (

,

,

)

but in

and

the topology fails to meet the criteria

in more 50% of the runs. Therefore, we may

conclude that von Neumann 99 performs more

consistently than gbest throughout the test set.

In the above reported statistical tests on the

averaged iterations to a solution, when a

configuration meets the criterion on less runs that

the other configuration, the best results are

selected and compared, where is the number of

runs in which the least successful configuration (of

two) met the criterion. When considering the results

of the four configuration in each function, and select

only the best iterations results, where is the

number of runs in which the least successful

configuration of all four met the criterion, different

iteration to solution values are obtained, which are

given Table 7. Under these criteria, the 9

9partially connected Von Neumann structure still

performs better than lbest and Von Neumann in the

majority of the scenarios. The gbest is the fastest

configuration in four functions but its fitness values

and success rates, as already stated, are very poor

when compared to the other algorithms.

The boxplot in Fig. 1 summarizes the results

IJCCI2013-InternationalJointConferenceonComputationalIntelligence

52

of the algorithms according to the success metrics.

The gbest configuration is clearly the worst

algorithm in the test set under this criterion. The

standard Von Neumann configuration is the most

consistent (in the total 250 runs, it only failed in one

run), but the 99 Von Neumann attains similar

results: in 250 runs it only failed twice.

Figure 1: Rank by success rates. Von Neuammn random

partially connected structure (vNR), Von Neumann (vN),

lbest (lB) and gbest (gB).

A general evaluation of the four topologies

according to fitness, speed and success results in the

following ranking: 99 von Neumann (1.7),

standard von Neumann (2.1), lbest (3.0) and gbest

(3.2). The proposed structure ranks first. Figure 2

shows the boxplot of the ranking.

As demonstrated above, the proposed partially

connected structures are able to improve the

standard configuration and the classical lbest and

gbest topologies. The question that arises now is

what makes these random structures better. The

differences to the standard configuration are the

candidates for explaining the differences: different

average connectivity, dynamic connectivity and

neighborhood, and memory (please remember that a

particle retains a

, even if the informant is no

longer in the neighborhood, until a better pg is

transmitted by a neighbor).

Figure 2: Rank by overall performance.

Some tests with non-memory versions of the

dynamic structures showed that the memory version

performs generally better. However, non-memory

structures do not necessarily perform worst and this

strategy may be useful under higher connectivity

partially connected structures (with Moore

neighborhood, for instance). This study is beyond

the scope of this paper and the main conclusion at

this moment is that the memory scheme is beneficial

for the proposed Von Neumann structure.

Table 8: Moore topologies. Best fitness values averaged

over 50 runs.

f

1

f

2

f

3

f

4

f

5

Moore

2.04e‐41 1.23e+01 6.78e+01 9.98e‐03 1.94e‐04

±2.78e‐41 ±2.28e+01 ±1.54e+01 ±1.42e‐02 ±1.37e‐03

Moore

(7×7)

1.80e‐42 9.51e00 7.61e+01 9.15e‐03 1.55e‐03

±6.81e‐39 ±1.94e+00 ±2.28e+01 ±1.31e‐02 ±3.60e‐03

Moore

(8×8)

7.04e‐41 6.02e00 7.62e+01 1.20e‐02 2.77e‐04

±1.46e‐40 ±1.88e+01 ±2.23e+01 ±1.50e‐02 ±2.66e‐03

Moore

(9×9)

9.08e‐40 1.02e+01 6.86e+01 9.54e‐03 1.17e‐03

±1.26e‐39 ±1.94e+01 ±1.98e+01 ±1.42e‐02 ±3.19e‐03

Moore

(10×10)

9.78e‐39 1.12e+01 6.87e+01 6.61e‐03 5.83e‐04

±1.76e‐38 ±2.16e+01 ±1.79e+01 ±1.03e‐02 ±1.37e‐03

Table 9: Von Neumann topologies. Iterations to a solution

averaged over 50 runs and number of successful runs.

f

1

f

2

f

3

f

4

f

5

Moore

419.58 1092.70 338.78 395.88 295.96

±17.56

(50)

±1209.86

(50)

±427.94

(49)

±28.19

(49)

±263.47

(50)

Moore

(7×7)

410.10 1605.58 250.71 385.45 521.52

±22.29

(50)

±1955.77

(50)

±78.12

(42)

±29.22

(49)

±703.56

(45)

Moore

(8×8)

427.98 1745.70 320.63 396.35 427.15

±17.55

(50)

±1805.28

(50)

±69.971

(45)

±26.88

(49)

±1026.02

(49)

Moore

(9×9)

440.26 2199.04 658.71 405.02 345.38

±21.13

(50)

±2233.83

(50)

±1270.512

(47)

±27.35

(47)

±939.29

(49)

Moore

(10×10)

452.08 1485.10 512.38 418.46 794.18

±17.51

(50)

±1720.67

(50)

±194.09

(47)

±24.96

(50)

±2259.27

(49)

4.2 Moore Neighborhood

Table 8 and Table 9 compare the Moore standard

configuration with partially connected Moore

structures. Table 8 gives the averaged best fitness

found by the swarms, while Table 9 gives, for each

algorithm and each function, the averaged number of

PerformanceandScalabilityofParticleSwarmswithDynamicandPartiallyConnectedGridTopologies

53

iterations required to meet the criterion, and the

number of runs in which the criterion was met.

The Moore dynamic structure with size 7×7, for

instance, is clearly better than the standard

configuration in functions f

1,

f

2

and f

3

, while being

outperformed in function f

5

. However, the structure

with size 9×9 does not improve significantly the

performance in any function, while being

outperformed in f

1

and f

5.

It seems that a sparse

connectivity degrades the performance of the Moore

structure, especially in the convergence speed of the

algorithm.

Table 10: 15. Best fitness values averaged over 50

runs.

f

1

f

2

f

3

f

4

VN

8.06e‐22 6.57e00 1.26e+01 1.46e‐02

±1.09e‐21 ±2.28e+01 ±5.72e00 ±1.70e+02

VN

(9×9)

1.68e‐22 1.73e00 1.50e+01 3.44e‐02

±2.03e‐22 ±3.29e00 ±6.58e00 ±2.73e‐02

Table 11: 15. Iterations to a solution averaged over

50 runs and number of successful runs.

f

1

f

2

f

3

f

4

VN

236.62 491.82 52.84 267.74

±11.12

(50)

±863.31

(49)

±12.41

(50)

±78.55

(46)

VN

(9×9)

233.38 351.82 55.52 297.56

±9.62

(50)

±370.88

(50)

±15.60

(50)

±92.07

(39)

4.3 Scalability

A simple scalability test of the von Neumann

structures was conducted by setting the

dimensionality of the functions

,

,

and

to

15and 60. Like in Section 4.1, two sets of

experiments were conducted. In the first set, the

algorithms were run for a limited amount of

iterations: with 15, 1000 iterations for

and

6000 for

,

and

; with 60, 3000 iterations

for

and 20000 for

,

and

. In the second set

of experiments the algorithms were all run for

20000 iterations or until reaching a stop criterion.

The criteria are as in Section 4.1, except with the

60-dimensional

function, for which the criteria

was set to 300 (because none of the algorithms

could meet the criterion set for the 30 version).

The number of iterations required to meet the

criterion was recorded and averaged over the 50

runs. A success measure was defined as the number

of runs in which an algorithm attains the fitness

value established as the stop criterion.

Results comparing the standard von Neumann

configuration and the 99 partially connected

configuration are in Tables 10-13. With

, the

standard and the partially connected configurations

are statistically equivalent for both 15 and

60. With

, the 99 topology is significantly

better than the standard von Neumann configuration

when 15 and 60. With

, the two

configuration are equivalent for 15 and the

99 topology is significantly better for 60.

Finally, with

, the partially connected 99

topology is worse when 15, but it is statistically

equivalent to the standard topology when 60.

Non-parametric Mann–Whitney U statistical

tests (with 0.05 level of significance) were used. A

version of the algorithm was considered statistically

better if at least one of the measures (average best

solution and average number of iterations to a

solution) was found to be statistically better, while

the other is at least equivalent.

Table 12: 60. Best fitness averaged over 50 runs.

f

1

f

2

f

3

f

4

VN

4.49e‐15 4.67e+01 2.79e+02 4.96e‐03

±3.68e‐15 ±5.51e+01 ±4.91e+01 ±1.04e‐02

VN

(9×9)

5.28e‐15 2.25e+01 2.50e+02 5.55e‐03

±8.54e‐15 ±3.50e+01 ±4.44e+01 ±1.26e‐02

Table 13: 60. Iterations to a solution averaged over

50 runs and number of successful runs.

f

1

f

2

f

3

f

4

VN

1054.24 6047.84 785.52 936.49

±29.11

(50)

±4693.06

(44)

±738.50

(31)

±42.52

(49)

VN

(9×9)

1042.76 6045.90 524.41 933.90

±52.70

(50)

±4264.85

(49)

±143.45

(44)

±56.99

(49)

These results, together with those discussed in

Section 4.1, show that the proposed partially

connected topology scales similarly to the standard

von Neumann topology on

,

and better on

and

. Please note that the 99 topology was used

here, i.e., no tuning of the grid size was done for

optimizing the performance. This particular

configuration is not only consistent throughout the

proposed test set, but also robust to the problem size.

IJCCI2013-InternationalJointConferenceonComputationalIntelligence

54

5 CONCLUSIONS

This paper describes a study on the effects of

alternative population structures on the behavior of

the Particle Swarm Optimization (PSO). Dynamic

and partially connected structures were tested by

placing the particles on a grid of nodes larger than

the swarm size. The particles move randomly on the

grid and the network of information is defined in

each iteration by the particle’s position in the grid

and by its neighborhood.

Von Neumann Structures with growing size were

tested on a classical test set and compared to

standard topologies. The results demonstrate that the

proposed structure performs consistently throughout

the test set, improving the performance of other

topologies in the majority of the scenarios and under

different performance evaluation criteria. The

structure is robust to the ratio between the grid size

and the swarm size and a fixed size with ratio 1:2

performs well on every function. A scalability test

was conducted by varying the dimensionality of four

functions in the test set. The proposed topology

scales similarly to the standard von Neumann

topology in two functions, and better in the two

other functions.

In the future, the test set will include more

functions. Non-random strategies for the movement

based on the fitness and the Euclidean distance

between the particles will also be considered.

ACKNOWLEDGEMENTS

The first author wishes to thank FCT, Ministério da

Ciência e Tecnologia, his Research Fellowship

SFRH/BPD/66876/2009). This work was supported

by FCT PROJECT [PEst-OE/EEI/LA0009/2011],

Spanish Ministry of Science and Innovation project

TIN2011-28627-C04-02, Andalusian Regional

Government P08-TIC-03903 and CEI-BioTIC UGR

project CEI2013-P-14.

REFERENCES

Hseigh, S.-T., Sun, T.-Y, Liu, C.-C., Tsai, S.-J. 2009.

Efficient Population Utilization Strategy for Particle

Swarm Optimizers. IEEE Transactions on Systems,

Man and Cybernetics—part B, 39(2), 444-456.

Kennedy, J., Eberhart, R. 1995. Particle Swarm

Optimization. In Proceedings of IEEE International

Conference on Neural Networks, Vol.4, 1942–1948.

Kennedy, J., Mendes, R., 2002. Population structure and

particle swarm performance. In Proceedings of the

IEEE World Congress on Evolutionary Computation,

1671–1676.

Liang, J. J., Qin, A. K., Suganthan, P. N., Baskar, S.,

2006. Comprehensive learning particle swarm

optimizer for global optimization of multimodal

functions. IEEE Trans. Evolutionary Computation,

10(3), 281–296.

Parsopoulos, K. E., Vrahatis, M. N., 2004. UPSO: A

Unified Particle Swarm Optimization Scheme, Lecture

Series on Computer and Computational Sciences, Vol.

1, Proceedings of the International Conference of

"Computational Methods in Sciences and

Engineering" (ICCMSE 2004), 868-87

Parsopoulos, K. E., Vrahatis, M. N., 2005. Unified Particle

Swarm Optimization in Dynamic Environments.

Lecture Notes in Computer Science (LNCS), Vol.

3449, Springer, 590-599.

T. Peram, K. Veeramachaneni, C. K. Mohan, Fitness-

distance-ratio based particle swarm optimization. In

Proc. Swarm Intell. Symp., 2003, pp. 174–181.

Shi, Y. Eberhart, R. C. 1998. A Modified Particle Swarm

Optimizer. In Proceedings of IEEE 1998 International

Conference on Evolutionary Computation, IEEE

Press, 69–73.

Trelea, I. C. 2003. The Particle Swarm Optimization

Algorithm: Convergence Analysis and Parameter

Selection. Information Processing Letters, 85, 317-

325.

PerformanceandScalabilityofParticleSwarmswithDynamicandPartiallyConnectedGridTopologies

55