Authors:
Anikó Csébfalvi
and
György Csébfalvi
Affiliation:
University of Pécs, Hungary
Keyword(s):
Heuristic Algorithm, Population-based Metaheuristic Algorithm, Statistical Test, Nonparametric Test, Statistical Comparison, Sampling Theorem, Sample Design and Analysis Kolmogorov-Smirnov Test.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Computational Intelligence
;
Evolutionary Computing
;
Hybrid Systems
;
Soft Computing
Abstract:
17 years ago, Hooker (1995) presented a pioneering work with the following title: "Testing Heuristics: We Have It All Wrong". If we ask the question now: "Do we have it all wrong?" the answer will be undoubtedly yes. The problem of the fair comparison remained essentially the same in the heuristic community. When we use stochastic methods in the optimization (namely heuristics or metaheuristics with several tunable parameters and starting seeds) then the usual presentation practice: "one problem - one result" is extremely far from the fair comparison. From statistical point of view, the minimal requirement is a so-called "small-sample" which is a set of results generated by independent runs and an appropriate "small-sample-test" according to the theory of the experimental design and evaluation and the protocol used for example, in the drug development processes. The viability and efficiency of the proposed statistically correct "bias-free" nonparametric methodology is demonstrated us
ing a well-known nonlinear structural optimization example on the set of state-of-the-art heuristics. In the motivating example we used the presented solutions as a small-sample generated by a "hyperheuristic" and we test its quality against ANGEL, where the "supernatural" hybrid metaheuristic ANGEL combines ant colony optimization (AN), genetic algorithm (GE) and a gradient-based local search (L) strategy. ANGEL is an "essence of the different but at the same time similar heuristic approaches". The extremely simple and practically tuning-free ANGEL presents a number of interesting aspects such as extremely good adaptability and the ability to cope with totally different large real applications from the highly nonlinear structural optimization to the long-term optimization of the geothermal energy utilization.
(More)