Parameter Sensitivity Patterns in the Plant Propagation Algorithm

Marleen de Jonge

and Daan van den Berg

Informatics Institute, University of Amsterdam, The Netherlands

Keywords:

Plant Propagation Algorithm, Evolutionary Algorithms, Parameterization, Parameter Sensitivity, Parameter

Robustness, Offspring Number, Population Size, Metaheuristics, Function Optimization.

Abstract:

The parameter sensitivity of the plant propagation algorithm’s performance, deﬁned as the maximum impact

on attained objective values, is studied on function optimization. The number of offspring and population size

are varied across a large range of values and tested on multidimensional benchmark test functions. As the

dimensionality of a function increases, the parametric sensitivity shows one of three distinct different patterns:

sublinear increase, superlinear increase or an up-down phase transition. We conjecture that the difference in

algorithmic behaviour might be due to the intrinsic mathematical properties of the functions themselves.

1 INTRODUCTION

Hard optimization problems come in a broad bou-

quet of both theoretical and real-world applica-

tions. Continuous problems such as molecular dock-

ing, scientiﬁc imaging techniques and process ﬂow-

sheet optimization in chemical plants (Meier et al.,

2010)(Chambolle and Pock, 2016)(Salhi and Fraga,

2011) are countered by discrete exemplars such

as vehicle routing, protein design and computa-

tional creativity (Kara et al., 2007)(Allouche et al.,

2014)(Paauw and Van den Berg, 2019). To date,

techniques to obtain good solutions for such prob-

lems can roughly be divided into two categories: ex-

act methods and heuristic methods(Puchinger and

Raidl, 2005)

. Exact methods exhaustively search

the complete or partially pruned state space, thereby

guaranteeing to ﬁnd an optimal solution, but run-

times tend to increase exponentially (or worse)

with problem instance size, prohibiting practical de-

ployment on any real-world problem. By sacri-

ﬁcing the guarantee of optimality, heuristic meth-

ods produce solutions which are good (enough),

and reachable within reasonable time budgets

(Puchinger and Raidl, 2005). For this reason, the de-

velopment (meta)heuristic algorithms has ﬂourished

as an approach for a multitude of optimization tasks.

https://orcid.org/0000-0003-4911-2647

https://orcid.org/0000-0001-5060-3342

For some spun out examples, see (Sleegers et al., 2020)

and (van den Berg et al., 2016).

All is not rosegardens however. As the ﬁeld has

witnessed a blossoming proliferation of new methods,

the often creative metaphorical references (to nature)

obfuscate real progress in the ﬁeld and prevent trans-

parency, replicability and scientiﬁc rigor (S

orensen,

2015). Furthermore, the community has been com-

plaining for ages that the wildgrowth of new algo-

rithms is seldomly parameterically tested, and the per-

formance across different problems is poorly under-

stood (S

orensen, 2015)(Eiben and Smit, 2011). As

new research on heuristic methods seems primarily

focused on achieving slight improvements on a sin-

gle problem instance, in-depth understanding of al-

gorithmic behaviour and performance dependencies

obtain much less attention. Kenneth S

orensen has in

fact devoted a whole paper to it, rightfully criticizing

this development as it “threatens to come at the ex-

pense of scientiﬁc rigor in the ﬁeld of metaheuristics”

orensen, 2015). The authors of this study generally

agree to these viewpoints.

Extensive understanding of parameter settings and

their contribution to algorithmic performance can be

of great importance, both in terms of its best found ob-

jective value (or ﬁtness) and in terms of its robustness

(Smit and Eiben, 2009). Unfortunately, the vast num-

ber of possible parameter conﬁgurations, the com-

plexity of their interaction and the diversity of mea-

suring methods make the process of ﬁnding optimal

parameter settings often harder than disentangling a

wall of brambles.

One relatively successful algorithm lacking a pa-

rameterization study is the plant propagation algo-

de Jonge, M. and van den Berg, D.

Parameter Sensitivity Patterns in the Plant Propagation Algorithm.

DOI: 10.5220/0010134300920099

In Proceedings of the 12th International Joint Conference on Computational Intelligence (IJCCI 2020), pages 92-99

ISBN: 978-989-758-475-6

Figure 1: Nine multidimensional benchmark test functions used in this study. All are continuous and dimensionally scalable,

but vary in properties such as modality, range, domain, and elementary mathematical constituents. Shown here are the 2D

versions.

rithm (PPA), introduced by Abdellah Salhi and Eric

S. Fraga (Salhi and Fraga, 2011). Even though it

is metaphorically linked to nature, its paradigm is

rooted in simple operations, and its parametric canopy

is quite modest. It has been demonstrated to work

well on a variety of continuous benchmark test func-

tions as well as on some discrete problems such as the

Traveling Salesman Problem, University Timetabling

and even explicitly tracking down NP-complete prob-

lem instances (Sulaiman et al., 2014)(Salhi and Fraga,

2011) (Selamo

glu and Salhi, 2016)(Vrielink and

van den Berg, 2019) (Geleijn et al., 2019)(Sleegers

and van den Berg, 2020). Additionally, the avail-

ability of open source python repositories lower the

threshold for implementation, experimentation, test-

ing, and replication(De Jonge, 2020).

In most experimentation on PPA, its parame-

ters are arbitrarily set, and little is known about

their individual or interdependent impact on perfor-

mance. Following a preliminary investigation on two-

dimensional functions(de Jonge and van den Berg,

2020), we will address multidimensional functions,

which behave substantially different. We investigate

the interplay of two interdependent parameters of the

algorithm, the population size (popSize) and the max-

imum number of offspring per individual (n

max

). We

explore 400 parameter settings of PPA on nine care-

fully chosen n-dimensional benchmark test functions.

Adopting a practical approach advocated by Selmar

Smit and Gusz Eiben, we choose parameter values

that optimize the algorithm’s performance while si-

multaneously mapping how its performance depends

on its parameter settings, referring to the latter as ‘ro-

bustness’ or ‘sensitivity’. The setup will yield some

puzzling results, and raise questions about the inter-

play between the algorithm and the problem instance

it is trying to solve

. Luckily, we also get some

straight up answers.

The rest of the paper is organized as follows: the

following section will provide an explanation on PPA,

whereas Section 3 will introduce the benchmark test

An elaborate discussion on exactly this topic took place

in the GECCO-workshop “Good Benchmarking Practices

for Evolutionary Computation”, which took place (online)

at GECCO2020, and also supplied a public paper (Bartz-

Beielstein et al., 2020).

Parameter Sensitivity Patterns in the Plant Propagation Algorithm

functions used in this study. Thereafter, Section 4

offers a detailed description of the experiment after

which the results and conclusions will be presented

in Section 5. We ﬁnalize the paper by discussing the

choices we made along the way, and the shortcomings

our approach might have in Section 6.

2 PLANT PROPAGATION

ALGORITHM

In its seminal form, PPA is an implementation on con-

tinuous optimization problems (Algorithm 1) (Salhi

and Fraga, 2011), and the enveloping paper also in-

cluded some runs on a suite of benchmark test func-

tions, all of which have been included in this study

(Figure 1, Table 1). The only difference is that for

this investigation, all global minima are normalized

to zero by applying the necessary ‘vertical’ shifts.

Algorithm 1: PPA pseudo code.

Require: objective f(x), x ∈ R

1: Generate population P = {p

, p

, ..., p

popSize

}

2: g ←1

3: for g ←1 to g

max

4: compute N

= f (p

), ∀p

∈ P

5: sort P in descending order of N

6: create new population φ

7: for each p

∈ P do {take best popSize only}

8: r

← set of offspring where both the size

of the set and the mutability for each new individ-

ual is functionally related to the ﬁtness N

9: φ ← φ ∪ r

merge population with off-

spring; death occurs by selection of popSize best

individuals

10: end forP ←φ new population

11: end for

12: return P, population of solutions

In this population based algorithm, an individual

is a vector of D points, in which D is the dimension-

ality of the benchmark function at hand, and its ob-

jective value is calculated from the dimensionalized

benchmark function on the vector. After randomly

initializing popSize individuals, calculating objective

values and normalizing them to [0, 1] (Eq. 1), the

hyperbolic tangent function maps the normalized ﬁt-

ness N

nonlinearly to (0,1) (Eq. 2). This asymptotic

function serves to prevent offspring to be generated at

the exact same location as the parent, and “provide[s]

a means of emphasising further better solutions over

those which are not as good” (Salhi and Fraga, 2011).

Then, each generation, individuals within the popula-

tion with higher mapped ﬁtness N

will generate more

offspring, whereas individuals with the lower N

will

generate fewer offspring (Eq. 3). But whereas the

number of offspring n

is proportional to N

, the muta-

bility m

of an individual’s offspring is inversely pro-

portional to the ﬁtness (Eq. 4).

z(x

) =

f (x

max

) − f (x

)

f (x

max

) − f (x

min

)

(1)

(tanh(4z(x

) −2) + 1) (2)

= [n

max

∗N

∗r] (3)

= 2(1 −N

)(r −0.5) (4)

In these equations, r is random number ∈[0, 1), drawn

anew every time it is used, and m

is applied to every

dimension of the benchmark function at hand. The

newly created offspring is then added tot the popula-

tion, which is sorted to ﬁtness, and the popSize best

individuals are selected for the next generation. Note

that thereby, PPA always retains its best individual,

and adopts an elitist approach – its best ﬁtness cannot

decrease during a run.

3 BENCHMARK FUNCTIONS

Benchmarking is a relatively standardized way of

measuring performance and behaviour of algorithms

(Jamil and Yang, 2013). Literature on test functions

however, is quite diverse and sometimes poorly de-

ﬁned, causing multiple function deﬁnitions and spec-

iﬁcations to circulate. For this reason, we explicit

list the used formulae (Table 1) and supply pub-

licly accessible source code (De Jonge, 2020). The

suite was compiled by Wouter Vrielink and forms

a union of functions earlier used on the ﬁreworks

algorithm or the plant propagation algorithm (Tan

and Zhu, 2010)(Salhi and Fraga, 2011)(Vrielink and

van den Berg, 2019). We use the identical multi-

dimensional test functions from Vrielink’s suite to

enable direct comparison between all forementioned

studies. All code is written in Python version 3.6.3

and can be found in a publicly accessible online

repository(De Jonge, 2020).

ECTA 2020 - 12th International Conference on Evolutionary Computation Theory and Applications

Table 1: The entire test suite of multidimensional benchmark functions used in this study. Test have been performed on six

different dimensionalities with 400 parameter settings on each dimensionality.

Name Function Bounds

Tablet

f (x) = 10000x

∑

i=2

x ∈[−100, 100]

Ellipse

f (x) =

∑

i=1

10000

(

i−1

D−1

)

x ∈[−100, 100]

Cigar

f (x) = x

∑

i=2

10000x

x ∈[−100, 100]

Sphere

f (x) =

∑

i=1

x ∈[−100, 100]

Rosenbrock

f (x) =

∑

D−1

i=1

(100(x

i+1

−x

)

+ (x

−1)

) x ∈[−5, 10]

Griewank

f (x) = 1 +

∑

i=1

4000

−

∏

i=1

cos

√

x ∈[−600, 600]

Schwefel

f (x) = 418.9829D −

∑

i=1

·sin(|

√

|)) x ∈[−500, 500]

Rastrigin

f (x) = 10D +

∑

i=1

−10cos(2πx

)) x ∈[−5.12,5.12]

Ackley

f (x) = −20 ·exp(−0.2 ·

∑

i=1

) −exp(

∑

i=1

cos(2πx

)) + 20 + e x ∈[−100, 100]

4 EXPERIMENT

The experimental setup is as massive as it is

straightforward; we perform optimization runs with

PPA on nine different n-dimensional benchmark

functions (Table 1) in six dimensionalities D ∈

{2, 5, 10, 20, 50, 100} resulting in 54 heatmaps (see

Figure 2). Every heatmap is comprised of 400 cells

for 400 different parameter settings 1 ≤ popSize ≤40,

1 ≤ n

max

≤ 10, and every cell contains ten PPA-runs

of 10,000 function evaluations, adding up to a grand

total of roughly 2.16 billion function evaluations for

the whole experiment. Due to parallelization efforts

however, the runtime for the experiment could be lim-

ited to approximately 34 hours on 10 nodes with 12 to

48 cores each, SURFSara’s LISA cluster computer at

Amsterdam Science Park.

After completing ten runs for one parameter set-

ting, the median was taken to a heatmap’s cell (e.g.

Rosenbrock’s six heatmaps in Figure 2). Generally

speaking, but surely in Rosenbrocks case, the average

error increases with the dimensionality of the bench-

mark function, resulting in ever darker heatmaps. But

this is to be expected; for all benchmark test functions

used in this study, the dimensionality is reﬂected in

the sum over its dimensions (

∑

i=1

), whose terms often

cannot be negative within the domain. Therefore, the

randomly chosen initial values are much higher on the

50-dimensional function than on the same function in

10 dimensions.

So how to account for this dimensional upscal-

ing when quantifying the parametric inﬂuence on the

attained objective value? Considering these obser-

vations, the degree to which PPA is susceptible to

changes in parameter values can be seen as the max-

imal difference its various parameterizations might

have within the 10,000 evaluation range of this exper-

iment. We therefore choose to deﬁne the parameter

sensitivity as the interval between the best and worst

heatmap cell relative to the range in the attained ob-

jective value:

Max( f

popSize,n

max

(x)) −Min( f

popSize,n

max

(x))

benchmark

(5)

Finally, for µ

benchmark

, we take the mean of 10,000

randomly sampled points (see Figure 3). This value is

alone sufﬁcient for these functions, as the global min-

imum of any of these in any dimensionality is zero.

5 RESULTS & CONCLUSION

From a simple and straightforward experimental setup

come some surprising results. For all functions, per-

formance of the plant propagation algorithm deterio-

rated with dimensionality, comparable to earlier re-

sults (Vrielink and van den Berg, 2019). The ex-

ception is the Ackley function, on which the found

minimum is near 20 for all D > 2, but this too is

known from literature (although the dimensionality

Parameter Sensitivity Patterns in the Plant Propagation Algorithm

Figure 2: Performance of the plant propagation algorithm with 400 different parameter settings on the Rosenbrock function.

Every cell contains the median value of ten runs of 10,000 evaluations. As the dimensionality increases, so does the mean

error (µ = 0.0005 to µ = 112011) and the variance thereof (σ = 0.0002 to σ = 21744). The absolute difference between the

best and worst parameter settings increases faster than its mean error, signifying an increase of parameter sensitivity in D.

bound might be slightly more accurate in (Vrielink

and van den Berg, 2019)). The parameter settings, in

all cases except the last three, did not make a differ-

ence of more than a magnitude.

A second, more interesting pattern, can be ob-

served by assessing the absolute parameter sensitivi-

ties in respect to the dimensionalities. For two bench-

mark test functions, the Tablet and the Ellipse, the

parameter sensitivity apparently shows a sublinear

increase with the dimensionality, which could pos-

sibly be characterized by a polynomial of O(n

) with

0 < c < 1. For four others, the Cigar, Sphere, Rosen-

brock and Griewank functions, the parameter sensi-

tivity displays a superlinear increase in dimension-

ality, which might be best characterized by a polyno-

mial of O(n

) with c > 1. In the three last functions,

Schwefel, Rastrigin and Ackley, the parameter sen-

sitivity shows something that looks like a up-down

phase transition. A dramatic initial increase in sen-

sitivity reaches an early summit somewhere between

5 and 10 dimensions, after which a rapidly collapse

follows, in one case even to near zero.

But a third, perhaps even more intriguing pat-

tern, can be discerned from ordering the nine sensi-

tivity patterns (Figure 4). With some imagination, one

can see the dimensional parameter sensitivity patterns

undergoing a structural phase transition themselves,

from sublinear, to superlinear, to up-down. When

looking closely, the up-down phase transition pattern

might already be materializing in the Griewank func-

tion, perhaps even in the Rosenbrock. Going back to

Table 1, one could easily wonder whether it is a coin-

cidence that the ‘simple quadratics’ only show sub-

linear and superlinear sensitivity patterns (the table

is sorted in the same order as Figure 4). The sec-

ond category, ’quadratic plus trigonometric’ shows

only superlinear or up-down phase transition patterns.

The third category, ’trigonometric plus exponential’

only has an up-down phase transition pattern, and it

is hard not to conjecture that these functional deﬁni-

tions themselves shelter the structural properties that

determine the parameter sensitivities of PPA, and per-

haps other (population based) heuristic algorithms.

An inquiry into what property constitutes which pat-

tern could be extremely valuable. And computation-

ally expensive, not to mention.

ECTA 2020 - 12th International Conference on Evolutionary Computation Theory and Applications

Figure 3: Parameter sensitivity is deﬁned as the difference between the absolute best parameterization and absolute worst

parameterization relative to the average initial condition, which, for most benchmark functions, increases with the dimen-

sionality of the function. PPA’s senstivity is 0.16 on the Schwefel 10D function, but 0.12 on the Schwefel 50D function, a

counterintuitive result which can be seen in more detail in Figure 4.

Figure 4: Parameter sensitivity patterns for PPA on benchmark test functions comes in three types: sublinear increase, su-

perlinear increase and up-down phase transition. A pattern could be to some degree related to a function’s mathematical

properties, such as being quadratic-only, quadratic-trigonometric, or exponential.

Parameter Sensitivity Patterns in the Plant Propagation Algorithm

6 DISCUSSION

From a simple and straightforward experimental

came some not-so-very-simple-and-straightforward

answers. Along the way, we made some choices that

are well open for discussion. First of all, we stud-

ied just two interdependent PPA-parameters and there

is absolutely no reason to presume that these results

are stable under replacement of the tanh-function, dif-

ferent mutability rates, or a different population se-

lection procedure. Second, the choice of benchmark

test functions could matter. Does a ‘combination of

Schwefel and Rastrigin’ show an intermittent stabil-

ity pattern? Third, the measure of sensitivity could be

different, based on standard deviations, or based on

the number of evaluations required to reach a certain

target value v, as suggested in (Eiben and Smit, 2011).

It could also be seen as a measure of uncertainty rel-

ative to a run’s performance, which is in turn relative

to the budget of function evaluations (set to 10,000

in this study). Fourth, considering the difference in

sensitivity patterns for continuous benchmark func-

tions, bifurcated study into more real-life examples

such as the optimization of chemical plant parame-

terizations might provide useful insights. First, the

parameter sensitivity could be assessed, similar to the

study presented here. Second, an algebraic compari-

son between it parameterization-ﬁtness projection and

the known benchmark functions could be made to fur-

ther our knowledge of the relation between parameter

sensitivity and the properties of the continuity it is try-

ing to minimize. Last but not least, the best-to-worst

order of individual heatmap cells could be studied; is

there a pattern to be found, even if just modest? All in

all, many possible roads lead into the future, and we

should make efforts to progress pedestrianally

ACKNOWLEDGEMENTS

Reitze Jansen (UvA), meticulous and accurate as ever,

was kind enough to pull some mistakes from this pa-

per. Thanks Reitze.

REFERENCES

Allouche, D., Andr

e, I., Barbe, S., Davies, J., de Givry, S.,

Katsirelos, G., O’Sullivan, B., Prestwich, S., Schiex,

Abdellian saying from ‘The Book of Strawberries’;

“Pedestrianally” (adj., adv.): calmly, considerately, unor-

ganizedly, chaotically, improvisingly. Wisely taking things

one step at the time; seeing where the road leads as events

unfold.

T., and Traor

e, S. (2014). Computational protein de-

sign as an optimization problem. Artiﬁcial Intelli-

gence, 212:59–79.

Bartz-Beielstein, T., Doerr, C., Bossek, J., Chandrasekaran,

S., Eftimov, T., Fischbach, A., Kerschke, P., Lopez-

Ibanez, M., Malan, K. M., Moore, J. H., Naujoks, B.,

Orzechowski, P., Volz, V., Wagner, M., and Weise, T.

(2020). Benchmarking in optimization: Best practice

and open issues.

Chambolle, A. and Pock, T. (2016). An introduction to

continuous optimization for imaging. Acta Numerica,

25:161–319.

De Jonge, M. (2020). Source code of algorithm used in

the experiment. https://github.com/marleentheyoung/

BScKI Thesis PPA.

de Jonge, M. and van den Berg, D. (2020). Plant propa-

gation parameterization: Offspring & population size.

Evo* 2020, page 19.

Eiben, A. E. and Smit, S. K. (2011). Parameter tuning for

conﬁguring and analyzing evolutionary algorithms.

Swarm and Evolutionary Computation, 1(1):19–31.

Geleijn, R., van der Meer, M., van der Post, Q., and van den

Berg, D. (2019). The plant propagation algorithm on

timetables: First results. EVO* 2019, page 2.

Jamil, M. and Yang, X.-S. (2013). A literature survey

of benchmark functions for global optimization prob-

lems. arXiv preprint arXiv:1308.4008.

Kara, I., Kara, B. Y., and Yetis, M. K. (2007). Energy mini-

mizing vehicle routing problem. In International Con-

ference on Combinatorial Optimization and Applica-

tions, pages 62–71. Springer.

Meier, R., Pippel, M., Brandt, F., Sippl, W., and Baldauf, C.

(2010). Paradocks: a framework for molecular dock-

ing with population-based metaheuristics. Journal of

chemical information and modeling, 50(5):879–889.

Paauw, M. and Van den Berg, D. (2019). Paintings,

polygons and plant propagation. In International

Conference on Computational Intelligence in Music,

Sound, Art and Design (Part of EvoStar), pages 84–

97. Springer.

Puchinger, J. and Raidl, G. R. (2005). Combining meta-

heuristics and exact algorithms in combinatorial opti-

mization: A survey and classiﬁcation. In International

work-conference on the interplay between natural and

artiﬁcial computation, pages 41–53. Springer.

Salhi, A. and Fraga, E. S. (2011). Nature-inspired optimi-

sation approaches and the new plant propagation algo-

rithm.

Selamo

glu, B.

I. and Salhi, A. (2016). The plant propaga-

tion algorithm for discrete optimisation: The case of

the travelling salesman problem. In Nature-inspired

computation in engineering, pages 43–61. Springer.

Sleegers, J. and van den Berg, D. (2020). Plant propagation

& hard hamiltonian graphs. Evo* 2020, page 10.

Smit, S. K. and Eiben, A. E. (2009). Comparing parameter

tuning methods for evolutionary algorithms. In 2009

IEEE congress on evolutionary computation, pages

399–406. IEEE.

ECTA 2020 - 12th International Conference on Evolutionary Computation Theory and Applications

orensen, K. (2015). Metaheuristics—the metaphor ex-

posed. International Transactions in Operational Re-

search, 22(1):3–18.

Sulaiman, M., Salhi, A., Selamoglu, B. I., and Kirikchi,

O. B. (2014). A plant propagation algorithm for con-

strained engineering optimisation problems. Mathe-

matical problems in engineering, 2014.

Tan, Y. and Zhu, Y. (2010). Fireworks algorithm for opti-

mization. In International conference in swarm intel-

ligence, pages 355–364. Springer.

van den Berg, D., Braam, F., Moes, M., Suilen, E., and Bhu-

lai, S. (2016). Almost squares in almost squares: Solv-

ing the ﬁnal instance. DATA ANALYTICS 2016, page

81.

Vrielink, W. and van den Berg, D. (2019). Fireworks algo-

rithm versus plant propagation algorithm.

Parameter Sensitivity Patterns in the Plant Propagation Algorithm