Enhanced Differential Grouping for Large Scale Optimization
Mohamed A. Meselhi, Ruhul A. Sarker, Daryl L. Essam and Saber M. Elsayed
School of Engineering and Information Technology, University of New South Wales at Canberra, Canberra, 2600, Australia
Keywords:
Evolutionary Algorithms, Cooperative Coevolution, Problem Decomposition, Large Scale Global Optimiza-
tion.
Abstract:
The curse of dimensionality is considered a main impediment in improving the optimization of large scale
problems. An intuitive method to enhance the scalability of evolutionary algorithms is cooperative co-
evolution. This method can be used to solve high dimensionality problems through a divide-and-conquer
strategy. Nevertheless, its performance deteriorates if there is any interaction between subproblems. Thus, a
method that tries to group interdependent variables in the same group is demanded. In addition, the compu-
tational cost of current decomposition methods is relatively expensive. In this paper, we propose an enhanced
differential grouping (EDG) method, that can efficiently uncover separable and nonseparable variables in the
first stage. Then, nonseparable variables are furthermore examined to detect their direct and indirect inter-
dependencies, and all interdependent variables are grouped in the same subproblem. The efficiency of the
EDG method was evaluated using large scale global optimization benchmark functions with up to 1000 vari-
ables. The numerical experimental results indicate that the EDG method efficiently decomposes benchmark
functions with fewer fitness evaluations, in comparison with state-of-the-art methods. Moreover, EDG was
integrated with cooperative co-evolution, which shows the efficiency of this method over other decomposition
methods.
1 INTRODUCTION
In real-world problems (such as computational chem-
istry, design problems, operations research and bio-
logical applications), the growing number of decision
variables is considered the main reason for the in-
creased complexity of solving large scale global op-
timization (LSGO) problems. Firstly, the computa-
tional cost of solving these problems using traditional
evolutionary algorithms (EAs) is often excessively
expensive (Dong et al., 2013). In addition, the per-
formance of these problems deteriorates because of
expansion in the search space, which increases expo-
nentially with increases in problem dimension. Fur-
thermore, the complexity of such problems usually
leads to a local optimum (Bhattacharya et al., 2016).
There has been great interest in handling LSGO,
thus several approaches have been employed in or-
der to solve problems with a large number of decision
variables, including but not limited to cooperative co-
evolution (CC) (Potter and De Jong, 1994), memetic
algorithms (Molina et al., 2011), initialization (Kaz-
imipour et al., 2014; Segredo et al., 2018) and par-
allelization(Meselhi et al., 2017). The decomposition
approach is considered as the first attempt for address-
ing the curse of dimensionality, which is based on a
divide-and-conquer strategy.
The classic CC approach decomposes the LSGO
problems into smaller dimension subproblems, and
EAs are used to optimize each subproblem cooper-
atively. This approach has been used successfully to
solve different LSGO problems, including combina-
torial (Mei et al., 2014), continuous (Omidvar et al.,
2014), constrained (Sayed et al., 2015) and multi-
objective (Cao et al., 2017). However, the major
drawback in the CC approach is that its performance
potentially decreases when solving nonseparable op-
timization problems, due to interdependencies among
their subproblems (Salomon, 1996). In nonseparable
problems, it was shown that the changing of one sub-
component will lead to a deformation in other inter-
dependent subcomponents’ fitness landscapes (Kauff-
man and Johnsen, 1991). Thus, decomposition tech-
niques that are capable of identifying interacting vari-
ables, and so group them into independent subprob-
lems, are desired to improve the performance of the
CC approach with large scale optimization problems.
Basically, decomposition methods that have been
proposed to use variable grouping for LSGO prob-
lems, can be classified into two general approaches,
Meselhi, M., Sarker, R., Essam, D. and Elsayed, S.
Enhanced Differential Grouping for Large Scale Optimization.
DOI: 10.5220/0006938902170224
In Proceedings of the 10th International Joint Conference on Computational Intelligence (IJCCI 2018), pages 217-224
ISBN: 978-989-758-327-8
Copyright © 2018 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
217

i.e. static and dynamic grouping methods. In the
static grouping method, an n dimension LSGO prob-
lem is decomposed into s subproblems of size k in the
beginning of CC, and the arrangement of variables in
each subproblem remains the same throughout the op-
timization stage. This method has shown its efficiency
only on small dimension problems (i.e. up to 100).
In the second method, dynamic grouping, each
subproblem structure is changed dynamically over the
optimization process. Decomposition approaches in
this method fall into two main categorizes, namely,
fixed and automatic decomposition methods. The
fixed decomposition method has a fixed subproblem
size, that needs to be specified manually; for instance,
random grouping (RG) (Yang et al., 2007) and delta
grouping [Omidvar et al., 2010b]. The main draw-
backs to these approaches is that the values of s and
k do not often align with the structure of variables in-
teractions. However, in the automatic decomposition
method, the number of subproblems and their sizes
is determined automatically, depending on the inter-
action between an objective function’s variables; for
example, Variable Interactive Learning (VIL) (Chen
et al., 2010), correlation based Adaptive Variable Par-
titioning (AVP) (Ray and Yao, 2009) and differen-
tial grouping (DG) (Omidvar et al., 2014). However,
the computational cost of these approaches in the de-
composition stage is relatively expensive. In this pa-
per, we propose an enhanced differential grouping
(EDG) method. EDG begins by isolating separable
variables from nonseparable variables. This separa-
tion saves a significant amount of computational cost
that would otherwise be wasted in detecting interde-
pendencies in separable variables. Then, nonsepara-
ble variables are further examined to detect any in-
terdependencies among them. The EDG method is
evaluated using the CEC’2010 benchmark problems
(Tang et al., 2009), and the experimental results show
100% grouping accuracy in a smaller number of fit-
ness evaluations, in comparison with state-of-the-art
decomposition methods.
The rest of this paper is organized as follows. Sec-
tion 2 presents the theoretical basis and introduces re-
lated work. The proposed algorithm is described in
Section 3. Section 4 presents and discusses the exper-
imental studies. Section 5 concludes the paper.
2 LITERATURE REVIEW
In this section, the theoretical bases of separability
and variable interaction are presented. This is fol-
lowed by an overview of various decomposition meth-
ods that have been proposed for CC in the literature.
2.1 Separability and Variable
Interaction
Separability indicates the degree of interaction be-
tween decision variables. A partially separable func-
tion is defined as follows: given f (X) , where X =
x
1
,...,x
n
. If there exists m subproblems (k
1
,...,k
m
) so
that:
f (x) =
m
∑
i=1
f
i
(x
k
i
); 2 ≤ m ≤ n, (1)
then f (x) is a partially separable function.
If m = n (i.e., k
1
,...,k
m
are one-dimensional sub-
problems), then f (x) is a fully separable function.
The interaction between two decision variables
can be either direct or indirect. Consider the follow-
ing example:
f (X ) = x
1
· x
3
+ x
2
· x
3
(2)
both x
1
↔ x
3
and x
2
↔ x
3
have direct interaction with
each other, denoted by ↔. However, there is no di-
rect interaction between x
1
and x
2
; instead both of
them interact indirectly through x
3
. Definitions of di-
rect and indirect interactions are illustrated as follows
(Sun et al., 2015). Let f (x) be an additively separable
function , and X
∗
is a candidate solution in the deci-
sion space. If
f
0
x
i
x
j
(X
∗
) 6= 0, (3)
then the decision variable x
i
and x
j
have a direct in-
teraction between them. If
f
0
x
i
x
j
(X
∗
) = 0, (4)
and there is an indirect link between x
i
and x
j
through
other decision variables in the decision space, then
the decision variables x
i
and x
j
have an indirect in-
teraction between them. Based on the definition of
interaction in Equation (3), and if there is no indirect
interaction with other variables, then x
i
and x
j
are in-
dependent.
2.2 Cooperative Co-evolution (CC)
Cooperative Co-evolution (CC) is the first attempt
to solve LSGO problems using a divide-and-conquer
strategy. It decomposes a high-dimensional prob-
lem into several smaller dimension subproblems. The
typical CC framework has three stages: decomposi-
tion, optimization and cooperation. In the decompo-
sition stage, the high-dimension problem is decom-
posed into several smaller dimension subproblems.
For example, the initial CC methods are the one-
dimension based- (Potter and De Jong, 1994) and
splitting-in-half strategies (Potter and Jong, 2000). In
the former method, an n dimension problem is divided
IJCCI 2018 - 10th International Joint Conference on Computational Intelligence
218

into n one-dimensional subproblems, while the latter
method divides an n dimension problem into two
n
2
subproblems.
In the optimization stage, each subproblem is op-
timized independently by a traditional optimization
algorithm for a certain number of generations. In
the classical CC framework (Potter and De Jong,
1994), subproblems evenly share the available com-
putational resources in a round-robin strategy. It
has been shown recently that contribution-based CC
(Yang et al., 2017) can allocate the available com-
putational resources effectively, by increasing the as-
signed resources to the subproblems with a higher
contribution to fitness improvement. Finally, the co-
operative stage exchanges information among all sub-
problems, and the final solution is constructed by
merging solutions from all subproblems.
2.3 Classification of Decomposition
Methods
2.3.1 Static Grouping
Static grouping decomposes LSGO problems in the
beginning of CC. The CC framework was inte-
grated with Fast Evolutionary Programming (FEP)
to solve LSGO problems with 100-1000 variables
called FEPCC (Liu et al., 2001). The obtained results
showed the inability of a traditional CC framework to
deal with nonseparable functions.
Van den Bergh and Engelbrecht (Van den Bergh
and Engelbrecht, 2004) incorporated Particle Swarm
Optimization to a CC framework called CPSO. An
n dimension problem is decomposed into k subprob-
lems of size s. However, CPSO were tested on prob-
lems with up to 30 dimensions. Shi et al. (Shi et al.,
2005) presented cooperative co-evolutionary differen-
tial evolution (CCDE), which partitioned the search
space into two equally-sized subproblems. Thus, this
decomposition method does not maintain its perfor-
mance when dimensionality increases.
The Cooperative Micro-Differential Evolution
(COMDE) (Parsopoulos, 2009) algorithm divides an
LSGO problem into a set of subproblems of size 5,
which are easier to evolve. Nevertheless, increasing
the number of subproblems leads to expensive com-
putational cost. The interdependencies among the
subproblems are not considered in the static group-
ing methods, which often affect optimization perfor-
mance. It is clear that the static grouping method is in-
effective on partially separable or nonseparable large
scale problems.
2.3.2 Dynamic Grouping
In the dynamic grouping method, instead of using
static grouping, the arrangement of variables is dy-
namically changed to deal with the variable interac-
tions. There are two categories based on the charac-
teristics of subproblems, namely a) fixed and b) auto-
matic decomposition methods.
(a) Fixed decomposition methods
Yang et al. (Yang et al., 2007) proposed the
random grouping decomposition strategy. In each
generation, the arrangement of variables is randomly
grouped into k s-dimensional subproblems. Although
this method achieved good performance on a set of
LSGO problems with dimension of 500 and 1000
(Yang et al., 2008a), it has been shown that the proba-
bility of grouping more than four interacting variables
in one subproblem reaches approximately 0 (Omidvar
et al., 2010).
Ray and Yao (Ray and Yao, 2009) developed a
CC Algorithm using AVP, called CCEA-AVP, that de-
tects/measures variables interaction using the Pear-
son correlation coefficient. All the decision variables
are optimized for 5 generations, and then the cor-
relation coefficients that depend on the top 50% of
the current population are calculated. According to
the obtained correlation coefficients, the variables are
grouped into two subproblems, with correlation co-
efficients greater than a threshold value in one sub-
problem and all the rest in another subproblem. This
method outperforms the standard CC method (Pot-
ter and De Jong, 1994) on solving nonseprable prob-
lems. However, it does not detect nonlinear interac-
tions among variables (Sayed et al., 2012).
A different method, called delta grouping [Omid-
var et al., 2010b], sorts the decision variables based on
their absolute magnitude of change across the popu-
lation between two consecutive generations. Despite
the superiority of this method over the random group-
ing method on solving CEC’2010 benchmark prob-
lems (Tang et al., 2009), it has low performance on
solving problems with more than one group of non-
seprable variables.
Elsayed et al. (2012) presented a Dependency
Identification (DI) technique, which divides a LSGO
problem into m smaller subproblems of V dependent
variables. The goal of this method is to find the
best arrangement of variables that minimizes the dif-
ference between the evaluation of the complete so-
lution vector F(x) and the sum of each subproblem
∑
m
k=1
f
k
(x
v
), v = [1, V ]. Then, subproblems are op-
timized using a memetic algorithm. DI outperformed
random grouping on 8 out of 12 problems. However, a
random arrangement of variables is generated at each
Enhanced Differential Grouping for Large Scale Optimization
219

iteration; the information from the current arrange-
ment is ignored.
A major drawback of these techniques is that a
predetermined number of subproblems needs to be
specified. However, the appropriate size of each
subproblem depends on each problem’s characteris-
tics. For separable variable problems, small sized
sub-problems are easier to optimise, while for non-
separable problems, a large size subproblem is use-
ful by increasing the probability of the existence of
dependent variables in the same subproblem. There-
fore, a decomposition algorithm that is automatically
able to identify the best number of subproblems and
their sizes, based on a problem’s characteristics, is re-
quired.
(b) Automatic decomposition methods
To the best of our knowledge, Cooperative
Co-evolution with Variable Interaction Learning
(CCVIL) (Chen et al., 2010) is the first attempt that
aims at constructing subproblems based on a prob-
lem’ characteristics. CCVIL assumes that all decision
variables are independent; thus, it starts by decompos-
ing a problem of size N into N one-dimensional sub-
problems. Then it discovers pairwise interaction be-
tween decision variables by using non-monotonicity
detection (Munetomo and Goldberg, 1999) . It then
merges them into common groups if they affect each
other. The CCVIL method is more efficient than fixed
grouping methods in identifying variable interactions.
However, this method creates N subproblems and so
is computationally expensive. In addition, it uses up
to 60% of available fitness evaluations for identifying
variable interactions.
Omidvar et al. (Omidvar et al., 2014) presented
an automated decomposition algorithm, called Dif-
ferential Grouping (DG), which is able to recognize
variable interactions by monitoring the effect of per-
turbing decision variables on fitness value. The inter-
action between the first decision variable and all other
decision variables is checked in a pair-wise fashion. If
any interaction is identified, the algorithm locates the
interacting variables in the same subproblem. Other-
wise, the first variable is considered a separable vari-
able. This process is repeated until there are no any
decision variables left. DG has achieved superior per-
formance over CCVIL on the CEC’2010 benchmark
problems (Tang et al., 2009). However, it has been
shown that DG is not able to correctly detect overlap-
ping functions. In addition to this, it is sensitive to a
threshold parameter which needs to be predetermined
by the user. Moreover, the computational cost for
fully separable n-dimensional functions is relatively
high O(n
2
).
Sun et al. (Sun et al., 2015) proposed an improved
version of DG, extended differential grouping (XDG),
that can address the problem of detecting overlapping
functions. After detecting the direct interacting deci-
sion variables with the same DG method, any indirect
interactions are determined between overlapped sub-
problems. Finally, both direct and indirect interacting
decision variables are grouped into the same subprob-
lems. Despite its improved performance of identify-
ing interactions on CEC’2010 benchmark functions,
it is also usually computationally expensive (O(n
2
)).
3 ENHANCED DIFFERENTIAL
GROUPING
In this section, the proposed decomposition method
is described in detail. The proposed algorithm starts
by separating separable variables from nonseparable
variables. Then, in the second stage, directly inter-
acting decision variables are identified. Finally, in-
direct interaction between decision variables is de-
tected. Thus, the proposed algorithm has three main
stages.
Based on the theoretical definition of variable in-
teraction in Equations (3) and (4), the following pro-
cedures can be used to identify interaction between
any two groups of decision variables (Sun et al.,
2017):
1. Calculate ∆
1
= f (x
∗
+ l
1
u
1
+ l
2
u
2
) − f (x
∗
+ l
2
u
2
).
2. Calculate ∆
2
= f (x
∗
+ l
1
u
1
) − f (x
∗
).
3. An interaction is detected if the difference be-
tween ∆
1
and ∆
2
is greater than a threshold ε.
where u
1
and u
2
are two unit vectors, l
1
and l
2
are two
real numbers > 0, and x
∗
is a candidate solution in the
search space.
In the first stage, the algorithm begins by detecting
both separable and nonseparable variables. Now if
separable variables are found, and excluded from the
next detection stages, a number of fitness evaluations
can be saved. So to identify variable’s separability,
the interaction between each decision variable and all
the remaining variables is examined. The EDG begins
by identifying the separability of the first decision
variable x
1
. All decision variables are set to a lower
bound, denoted by X
l,l
(line 1 of DeltaDi f f erence
function), and then x
1
will be perturbed to the upper
bound to form X
u,l
(line 2). The difference in the fit-
ness values at X
l,l
and X
u,l
will be calculated, denoted
by ∆
1
(line 3). Then all the variables in X
l,l
and X
u,l
will be perturbed to the middle of the decision space,
except x
1
, to form X
l,m
and X
u,m
respectively, and the
difference in the fitness values between them is then
IJCCI 2018 - 10th International Joint Conference on Computational Intelligence
220

Algorithm 1: EDG.
Input: f , ub,lb,ε
Output: sep,nonsep,nonsep
Groups,FEs
1: sep = nonsep = [ ];
2: FEs = 0;
3: dim = [1, 2, 3, ..., 1000];
4: for k = 1 : length(dim) do
5: X
1
= k;
6: X
2
= dim − {k};
7: di f f = DeltaDi f f erence(X
1
,X
2
);
8: if di f f ≤ ε then
9: sep = sep ∪ X
1
;
10: else
11: nonsep = nonsep ∪ X
1
;
12: end if
13: end for
14: nonsep
Groups = {};
15: while nonsep is not empty do
16: X
1
= nonsep(1);
17: for j = 2 : length(nonsep) do
18: X
2
= nonsep( j);
19: di f f = DeltaDi f f erence(X
1
,X
2
);
20: if di f f > ε then
21: X
1
= X
1
∪ X
2
;
22: end if
23: end for
24: nonsep = nonsep − X
1
;
25: X
1
= IndirectInteraction(X
1
,nonsep);
26: nonsep = nonsep − X
1
;
27: nonsep Groups = {nonsep Groups,X
1
};
28: end while
calculated, and is denoted by ∆
2
(lines 4-6). The dif-
ference of the delta values is calculated, denoted by
di f f (line 7). If this difference is less than the thresh-
old ε, then x
1
is classified as a separable variable and
is placed in the separable group, and the algorithm
in this stage moves to the following decision variable
x
2
. Otherwise, x
1
is considered to be a nonsepara-
ble variable (lines 4-13 of Algorithm 1). This pro-
cess will be repeated until the separability of all the
decision variables are identified. For fully separable
functions, the interaction detection between decision
variables is stopped at the end of this stage, whereas
in the case of partial or fully nonseparable functions,
further interdependency identification is required in
the next stages.
In the second stage, all the variables that inter-
act directly, will be detected and grouped in common
subproblems. A pairwise interaction of each nonsep-
arable decision variable, with all other nonseparable
decision variables, is examined using the same tech-
nique as in the traditional DG method; however, in
this stage we are concerned only with nonseparable
Function: DeltaDifference(X
1
,X
2
).
1: X
l,l
= lb;
2: X
u,l
= ub(X
1
);
3: ∆
1
= f (X
l,l
) − f (X
u,l
) ;
4: X
l,m
= (lb(X
2
) + ub(X
2
))/2;
5: X
u,m
= (lb(X
2
) + ub(X
2
))/2;
6: ∆
2
= f (X
l,m
) − f (X
u,m
) ;
7: di f f =| ∆
1
− ∆
2
|;
Function: IndirectInteraction(X
1
,X
2
).
1: while DeltaDi f f erence(X
1
,nonsep) > ε do
2: if length (nonsep) == 1 then
3: X
1
= X
1
∪ nonsep;
4: else
5: divide nonsep into two equally sized groups
nonsep
1
and nonsep
2
;
6: X
1
a
= IndirectInteraction(X
1
,nonsep
1
);
7: X
1
b
= IndirectInteraction(X
1
,nonsep
2
);
8: X
1
= X
1
a
∪ X
1
b
;
9: end if
10: end while
decision variables, which were detected in the first
stage. The algorithm checks the direct interaction be-
tween any two decision variables x
i
and x
j
by measur-
ing the difference between ∆
1
and ∆
2
(line 19). If the
difference between ∆
1
and ∆
2
is greater than ε, then x
j
interacts directly with x
i
, and x
j
will be located in the
same interdependent subproblem with x
i
, denoted as
X
1
. This process will be continued until all variables
that interact directly with the decision variable x
i
are
detected, and the subproblem is shaped. Then, this
subproblem will be excluded from the nonseparable
group nonsep, as shown in lines 16-24.
In the third stage, as far as indirect interaction
variables are concerned, the grouped directly inter-
acted variables in X
1
are further recursively examined
together with all other remaining nonseparable deci-
sion variables, to find any indirect interaction among
them. As shown in IndirectInteraction function, if
any interaction appears, all the nonseparable decision
variables will be divided into two groups with the
same size, and then the interaction between X
1
and
each group will be identified separately. This process
is executed repeatedly until all the indirect decision
variables that interact with x
i
are detected and merged
in nonsep Groups with x
i
.
Both stages 2 and 3 are repeated for all the remaining
decision variables until all subproblems are formed,
and finally the EDG returns all the independent sub-
problems.
Enhanced Differential Grouping for Large Scale Optimization
221

Table 1: Decomposition results on CEC’2010 benchmark functions.
Func
EDG RDG XDG DG
FEs accuracy FEs accuracy FEs accuracy FEs accuracy
f
1
3.01E+03 100% 3.01E+03 100% 1.00E+06 100% 1.00E+06 100%
f
2
3.01E+03 100% 3.01E+03 100% 1.00E+06 100% 1.00E+06 100%
f
3
5.00E+03 100% 6.00E+03 100% 1.00E+06 100% 1.00E+06 100%
f
4
3.10E+03 100% 4.21E+03 100% 8.05E+04 - 1.45E+04 100%
f
5
3.10E+03 100% 4.15E+03 100% 9.98E+05 100% 9.05E+05 100%
f
6
3.10E+03 100% 5.03E+04 100% 9.98E+05 100% 9.06E+05 100%
f
7
3.10E+03 100% 4.23E+03 100% 9.98E+05 100% 6.77E+04 68%
f
8
3.84E+03 100% 5.61E+03 100% 1.21E+05 - 2.32E+04 100%
f
9
8.48E+03 100% 1.40E+04 100% 9.77E+05 100% 2.70E+05 100%
f
10
8.48E+03 100% 1.40E+04 100% 9.77E+05 100% 2.72E+05 100%
f
11
1.50E+04 100% 1.37E+04 100% 9.78E+05 100% 2.70E+05 99.8%
f
12
8.48E+03 100% 1.43E+04 100% 9.77E+05 100% 2.71E+05 100%
f
13
2.60E+04 100% 2.92E+04 100% 1.00E+06 100% 5.03E+04 31.8%
f
14
2.40E+04 100% 2.06E+04 100% 9.53E+05 100% 2.10E+04 100%
f
15
2.40E+04 100% 2.05E+04 100% 9.53E+05 100% 2.10E+04 100%
f
16
2.40E+04 100% 2.09E+04 100% 9.56E+05 100% 2.11E+04 99.6%
f
17
2.40E+04 100% 2.08E+04 100% 9.53E+05 100% 2.10E+04 100%
f
18
6.48E+04 100% 4.99E+04 100% 9.99E+05 100% 3.96E+04 23%
f
19
5.00E+03 100% 6.00E+03 100% 3.99E+03 100% 2.00E+03 100%
f
20
9.99E+03 100% 5.09E+04 100% 1.00E+06 100% 1.55E+05 28.7%
4 EXPERIMENTS AND RESULTS
To evaluate the performance of the proposed EDG al-
gorithm, the CEC’2010 benchmark problems (Tang
et al., 2009) on large-scale global optimization were
used. The CEC’2010 benchmark functions includes
20 functions which are grouped into five categories as
follows:
1. Fully separable functions ( f
1
- f
3
)
2. Single-group m-nonseparable functions ( f
4
- f
8
)
3.
D
2m
-group m-nonseparable functions ( f
9
- f
13
)
4.
D
m
-group m-nonseparable functions( f
14
- f
18
)
5. Fully non-separable functions ( f
19
- f
20
)
where D is the problem’s dimension and m is the num-
ber of variables in each nonseparable subproblem. In
this paper, D and m are set to 1000 and 50, respec-
tively for all benchmark functions.
In the grouping stage, rather than a fixed threshold
ε value that does not suit all test functions (Omidvar
et al., 2014), an adaptive ε is adopted. The best ε
value for each specific function is determined using
the magnitude of the objective values in the decision
space (Mei et al., 2016):
ε = α · min{| f (x
1
) |,..., | f (x
K
) |},
where α is the control coefficient (set to 10
−10
),
and k random solutions x
1
,...,x
K
in the decision space
(Mei et al., 2016). While for the optimization stage,
a variant of Differential Evolution – SaNSDE (Yang
et al., 2008b) is used to optimize each subproblem co-
operatively. The experimental results are based on 25
independent runs, where the population size is 50, and
the maximum number of fitness evaluations, divided
between grouping and optimizing stages, is 3 × 10
6
.
4.1 Decomposition Results
The decomposition results of EDG, RDG (Sun et al.,
2017), XDG (Sun et al., 2015) and DG (Omidvar
et al., 2014) are presented in Table 1, which shows
that the EDG and RDG methods achieve 100% group-
ing accuracy on all the 20 benchmark functions, as
they use the same estimation of the threshold ε. In
contrast, XDG and DG correctly decompose 18 and
12 benchmark functions, respectively. DG has poor
decomposition accuracy on indirect interaction func-
tions.
Table 1 also illustrates the number of fitness eval-
uations used by each method. For the first three func-
tions that are fully separable functions ( f
1
- f
3
), the
number of fitness evaluations used by XDG and DG
are 1.00E+06. While both EDG and RDG identify the
separability of all decision variables on f
1
and f
2
, us-
ing 3.01E+03 FEs. EDG uses 5.00E+03 FEs to iden-
tify f
3
, in comparison to 6.00E+03 FEs for RDG.
For partially separable functions with one nonsep-
arable group of 50 variables ( f
4
- f
8
), EDG uses the
smallest number of fitness evaluations to correctly
identify the 50 nonseparable variables and 950 sep-
arable variables.
Category 2 and 3 contain one and 10 nonseparable
group, respectively, each with 50 variables ( f
4
- f
13
).
EDG again uses the smallest number of fitness eval-
uations to correctly identify all the 50 nonseparable
IJCCI 2018 - 10th International Joint Conference on Computational Intelligence
222

variables groups and separable variables group. This
holds for all test functions, except for f
11
, where EDG
uses slightly more fitness evaluation than RDG.
For category 4 (partially separable functions that
contain 20 nonseparable groups), EDG uses more fit-
ness evaluations than RDG. This is because of the fit-
ness evaluations used in detecting the separable part
of the decision variables, that does not exist in this
category. In category 5, the DG and EDG methods
use the smallest number of fitness evaluations on f
19
and f
20
, respectively.
As the aforementioned results show, the EDG
method correctly identifies all decision variables on
all benchmark functions, while using the smallest
number of fitness evaluations on 13 out of 20 bench-
mark problems.
4.2 Optimization Results
This section shows the performance of EDG, DI, DG
and D (delta grouping) methods, when integrated in
the DECC cooperative co-evolutionary framework.
Table 2 reports the experimental results of the com-
pared decomposition algorithms for 25 independent
runs on the CEC’2010 benchmark problems. The
EDG and DG methods group all separable variables
in one subproblem. Thus, on fully separable functions
( f
1
- f
3
), the D method outperforms both the EDG and
DG methods (where subproblem size = 1000).
The EDG method achieved the best mean results
on 11 out of 20 optimization functions, namely f
4
- f
8
,
f
9
, f
10
, f
14
, f
15
, f
17
, f
18
.
This demonstrates that good decomposition can
effectively enhance the performance of the optimiza-
tion stage in the CC framework. On f
19
and f
20
, de-
spite the correct decomposition, EDG has worse opti-
mization performance than DI. This is because of its
grouping of all 1000 nonseparable decision variables
into one interdependent subproblem.
5 CONCLUSIONS
This paper introduced an enhanced differential group-
ing (EDG) for LSGO problems. In the proposed
method, separable and nonseparable variables can be
classified in the first stage. Thus, a significant number
of fitness evaluations which would be used to iden-
tify interdependency among decision variables can be
saved. Then, direct and indirect interdependencies of
nonseparable variables are detected. Results from nu-
merical experiments indicate that EDG can achieve
100% grouping accuracy on all benchmark functions
with fewer fitness evaluations. EDG was also embed-
Table 2: Optimization results on CEC’2010 benchmark
functions.
Func Stats EDG DI DG D
f
1
Mean 2.64E+05 8.28E-06 1.12E+04 4.07E-24
Std 3.19E+05 3.24E-05 3.37E+04 1.75E-23
f
2
Mean 4.14E+03 5.44E+02 4.42E+03 2.82E+02
Std 4.65E+02 1.16E+02 1.59E+02 2.40E+01
f
3
Mean 1.10E+01 6.33E+00 1.67E+01 1.52E-13
Std 6.45E-01 9.38E-01 3.05E-01 8.48E-15
f
4
Mean 3.10E+10 2.83E+12 4.63E+12 4.12E+12
Std 1.42E+10 1.01E+12 1.35E+12 1.46E+12
f
5
Mean 7.12E+07 2.44E+08 1.98E+08 2.48E+08
Std 1.47E+07 3.20E+07 4.58E+07 4.79E+07
f
6
Mean 1.60E+01 2.21E+06 1.62E+01 5.34E+07
Std 7.46E+03 2.88E+05 2.82E-01 8.79E+07
f
7
Mean 1.31E+04 7.27E+07 1.63E+04 6.89E+07
Std 7.46E+03 3.62E+08 8.93E+03 4.96E+07
f
8
Mean 3.27E+05 2.15E+07 2.51E+07 1.09E+08
Std 1.10E+06 2.76E+07 2.54E+07 4.87E+07
f
9
Mean 2.34E+07 8.19E+07 5.60E+07 6.13E+07
Std 8.28E+06 8.59E+06 6.59E+06 6.28E+06
f
10
Mean 3.01E+03 9.96E+03 5.22E+03 1.29E+04
Std 2.22E+02 2.54E+03 1.28E+02 2.27E+02
f
11
Mean 2.58E+01 9.13E+01 9.94E+00 1.55E-13
Std 2.64E+00 1.75E+01 9.57E-01 8.19E-15
f
12
Mean 1.89E+04 2.95E+04 2.83E+03 4.30E+06
Std 9.14E+03 8.59E+06 9.92E+02 1.79E+05
f
13
Mean 1.11E+04 2.48E+03 5.35E+06 1.19E+03
Std 4.36E+03 2.54E+03 4.89E+06 5.02E+02
f
14
Mean 2.15E+07 2.50E+08 3.43E+08 1.93E+08
Std 2.28E+06 1.75E+01 2.23E+07 1.06E+07
f
15
Mean 2.94E+03 1.01E+04 5.84E+03 1.60E+04
Std 2.67E+02 4.44E+03 8.93E+01 4.24E+02
f
16
Mean 1.91E+01 2.47E+02 7.32E-13 1.70E+01
Std 2.80E+00 2.47E+01 4.62E-14 8.48E+01
f
17
Mean 7.50E+00 1.12E+05 3.99E+04 7.48E+06
Std 1.40E+00 2.46E+04 1.80E+03 4.03E+05
f
18
Mean 1.20E+03 7.30E+03 1.44E+10 3.32E+03
Std 1.44E+02 2.93E+03 2.50E+09 7.09E+02
f
19
Mean 9.24E+05 6.16E+05 1.72E+06 2.32E+07
Std 8.95E+04 5.17E+04 6.83E+06 5.56E+06
f
20
Mean 8.48E+07 2.92E+03 6.69E+10 1.18E+03
Std 2.49E+08 1.83E+02 9.24E+09 8.25E+01
ded in a DECC framework, where it achieved better
performance than state-of-the-art algorithms.
REFERENCES
Bhattacharya, M., Islam, R., and Abawajy, J. (2016). Evolu-
tionary optimization: a big data perspective. Journal
of Network and Computer Applications, 59:416–426.
Cao, B., Zhao, J., Lv, Z., and Liu, X. (2017). A dis-
tributed parallel cooperative coevolutionary multiob-
jective evolutionary algorithm for large-scale opti-
mization. IEEE Transactions on Industrial Informat-
ics, 13(4):2030–2038.
Chen, W., Weise, T., Yang, Z., and Tang, K. (2010). Large-
scale global optimization using cooperative coevolu-
tion with variable interaction learning. In Interna-
tional Conference on Parallel Problem Solving from
Nature, pages 300–309. Springer.
Dong, W., Chen, T., Tino, P., and Yao, X. (2013). Scaling
up estimation of distribution algorithms for continu-
Enhanced Differential Grouping for Large Scale Optimization
223

ous optimization. IEEE Transactions on Evolutionary
Computation, 17(6):797–822.
Kauffman, S. A. and Johnsen, S. (1991). Coevolution to
the edge of chaos: coupled fitness landscapes, poised
states, and coevolutionary avalanches. Journal of the-
oretical biology, 149(4):467–505.
Kazimipour, B., Li, X., and Qin, A. K. (2014). A review
of population initialization techniques for evolution-
ary algorithms. In Evolutionary Computation (CEC),
2014 IEEE Congress on, pages 2585–2592. IEEE.
Liu, Y., Yao, X., Zhao, Q., and Higuchi, T. (2001). Scal-
ing up fast evolutionary programming with coopera-
tive coevolution. In Evolutionary Computation, 2001.
Proceedings of the 2001 Congress on, volume 2,
pages 1101–1108. Ieee.
Mei, Y., Li, X., and Yao, X. (2014). Cooperative coevolu-
tion with route distance grouping for large-scale ca-
pacitated arc routing problems. IEEE Transactions on
Evolutionary Computation, 18(3):435–449.
Mei, Y., Omidvar, M. N., Li, X., and Yao, X. (2016).
A competitive divide-and-conquer algorithm for un-
constrained large-scale black-box optimization. ACM
Transactions on Mathematical Software (TOMS),
42(2):13.
Meselhi, M. A., Elsayed, S. M., Essam, D. L., and Sarker,
R. A. (2017). Fast differential evolution for big opti-
mization. In Software, Knowledge, Information Man-
agement and Applications (SKIMA), 2017 11th Inter-
national Conference on, pages 1–6. IEEE.
Molina, D., Lozano, M., S
´
anchez, A. M., and Herrera, F.
(2011). Memetic algorithms based on local search
chains for large scale continuous optimisation prob-
lems: Ma-ssw-chains. Soft Computing, 15(11):2201–
2220.
Munetomo, M. and Goldberg, D. E. (1999). Linkage iden-
tification by non-monotonicity detection for overlap-
ping functions. Evolutionary computation, 7(4):377–
398.
Omidvar, M. N., Li, X., Mei, Y., and Yao, X. (2014). Co-
operative co-evolution with differential grouping for
large scale optimization. IEEE Transactions on evo-
lutionary computation, 18(3):378–393.
Omidvar, M. N., Li, X., Yang, Z., and Yao, X. (2010).
Cooperative co-evolution for large scale optimization
through more frequent random grouping. In Evolu-
tionary Computation (CEC), 2010 IEEE Congress on,
pages 1–8. IEEE.
Parsopoulos, K. E. (2009). Cooperative micro-differential
evolution for high-dimensional problems. In Proceed-
ings of the 11th Annual conference on Genetic and
evolutionary computation, pages 531–538. ACM.
Potter, M. A. and De Jong, K. A. (1994). A cooperative
coevolutionary approach to function optimization. In
International Conference on Parallel Problem Solving
from Nature, pages 249–257. Springer.
Potter, M. A. and Jong, K. A. D. (2000). Cooperative coevo-
lution: An architecture for evolving coadapted sub-
components. Evolutionary computation, 8(1):1–29.
Ray, T. and Yao, X. (2009). A cooperative coevolution-
ary algorithm with correlation based adaptive vari-
able partitioning. In Evolutionary Computation, 2009.
CEC’09. IEEE Congress on, pages 983–989. IEEE.
Salomon, R. (1996). Re-evaluating genetic algorithm per-
formance under coordinate rotation of benchmark
functions. a survey of some theoretical and practical
aspects of genetic algorithms. BioSystems, 39(3):263–
278.
Sayed, E., Essam, D., and Sarker, R. (2012). Using hybrid
dependency identification with a memetic algorithm
for large scale optimization problems. In Asia-Pacific
Conference on Simulated Evolution and Learning,
pages 168–177. Springer.
Sayed, E., Essam, D., Sarker, R., and Elsayed, S.
(2015). Decomposition-based evolutionary algorithm
for large scale constrained problems. Information Sci-
ences, 316:457–486.
Segredo, E., Paechter, B., Segura, C., and Gonz
´
alez-Vila,
C. I. (2018). On the comparison of initialisation strate-
gies in differential evolution for large scale optimisa-
tion. Optimization Letters, 12(1):221–234.
Shi, Y.-j., Teng, H.-f., and Li, Z.-q. (2005). Cooperative
co-evolutionary differential evolution for function op-
timization. In International Conference on Natural
Computation, pages 1080–1088. Springer.
Sun, Y., Kirley, M., and Halgamuge, S. K. (2015). Extended
differential grouping for large scale global optimiza-
tion with direct and indirect variable interactions. In
Proceedings of the 2015 Annual Conference on Ge-
netic and Evolutionary Computation, pages 313–320.
ACM.
Sun, Y., Kirley, M., and Halgamuge, S. K. (2017). A re-
cursive decomposition method for large scale continu-
ous optimization. IEEE Transactions on Evolutionary
Computation.
Tang, K., Li, X., Suganthan, P., Yang, Z., and Weise, T.
(2009). Benchmark functions for the cec 2010 spe-
cial session and competition on large-scale global op-
timization: Nature inspired computation and applica-
tions laboratory, university of science and technology
of china. Applicat. Lab., Univ. Sci. Technol. China,
Hefei, China, Tech. Rep.
Van den Bergh, F. and Engelbrecht, A. P. (2004). A cooper-
ative approach to particle swarm optimization. IEEE
transactions on evolutionary computation, 8(3):225–
239.
Yang, M., Omidvar, M. N., Li, C., Li, X., Cai, Z., Kaz-
imipour, B., and Yao, X. (2017). Efficient resource
allocation in cooperative co-evolution for large-scale
global optimization. IEEE Transactions on Evolution-
ary Computation, 21(4):493–505.
Yang, Z., Tang, K., and Yao, X. (2007). Differential evolu-
tion for high-dimensional function optimization. In
Evolutionary Computation, 2007. CEC 2007. IEEE
Congress on, pages 3523–3530. IEEE.
Yang, Z., Tang, K., and Yao, X. (2008a). Large scale evo-
lutionary optimization using cooperative coevolution.
Information Sciences, 178(15):2985–2999.
Yang, Z., Tang, K., and Yao, X. (2008b). Self-
adaptive differential evolution with neighborhood
search. In Evolutionary Computation, 2008. CEC
2008.(IEEE World Congress on Computational Intel-
ligence). IEEE Congress on, pages 1110–1116. IEEE.
IJCCI 2018 - 10th International Joint Conference on Computational Intelligence
224