Studying and Tackling Noisy Fitness in Evolutionary Design of Game Characters

J. J. Merelo, Pedro A. Castillo, Antonio Mora, Antonio Fernández-Ares, Anna I. Esparcia-Alcázar, Carlos Cotta, Nuria Rico

2014

Abstract

In most computer games as in life, the outcome of a match is uncertain due to several reasons: the characters or assets appear in different initial positions or the response of the player, even if programmed, is not deterministic; different matches will yield different scores. That is a problem when optimizing a game-playing engine: its fitness will be noisy, and if we use an evolutionary algorithm it will have to deal with it. This is not straightforward since there is an inherent uncertainty in the true value of the fitness of an individual, or rather whether one chromosome is better than another, thus making it preferable for selection. Several methods based on implicit or explicit average or changes in the selection of individuals for the next generation have been proposed in the past, but they involve a substantial redesign of the algorithm and the software used to solve the problem. In this paper we propose new methods based on incremental computation (memory-based) or fitness average or, additionally, using statistical tests to impose a partial order on the population; this partial order is considered to assign a fitness value to every individual which can be used straightforwardly in any selection function. Tests using several hard combinatorial optimization problems show that, despite an increased computation time with respect to the other methods, both memory-based methods have a higher success rate than implicit averaging methods that do not use memory; however, there is not a clear advantage in success rate or algorithmic terms of one method over the other

References

  1. Aizawa, A. N. and Wah, B. W. (1994). Scheduling of genetic algorithms in a noisy environment. Evolutionary Computation, 2(2):97-122.
  2. Castillo, P. A., González, J., Merelo-Guervós, J.-J., Prieto, A., Rivas, V., and Romero, G. (1999). G-Prop-III: Global optimization of multilayer perceptrons using an evolutionary algorithm. In GECCO-99: Proceedings Of The Genetic And Evolutionary Computation Conference, page 942.
  3. Cauwet, M.-L., Liu, J., Teytaud, O., et al. (2014). Algorithm portfolios for noisy optimization: Compare solvers early. In Learning and Intelligent Optimization Conference.
  4. Costa, A., Vargas, P., and Tinós, R. (2013). Using explicit averaging fitness for studying the behaviour of rats in a maze. In Advances in Artificial Life, ECAL, volume 12, pages 940-946.
  5. Deb, K. and Goldberg, D. E. (1992). Analyzing deception in trap functions. In FOGA, volume 2, pages 98-108.
  6. García-Ortega, R. H., García-Sánchez, P., and Merelo, J. J. (2014). Emerging archetypes in massive artificial societies for literary purposes using genetic algorithms. ArXiv e-prints. Available at http://adsabs.harvard.edu/abs/2014arXiv1403.3084G.
  7. Goldberg, D. E., Deb, K., and Horn, J. (1992). Massive multimodality, deception, and genetic algorithms. In R. Männer and Manderick, B., editors, Parallel Problem Solving from Nature, 2, pages 37-48, Amsterdam. Elsevier Science Publishers, B. V.
  8. Jin, Y. and Branke, J. (2005). Evolutionary optimization in uncertain environments - a survey. IEEE Transactions on Evolutionary Computation, 9(3):303-317. cited By (since 1996)576.
  9. Jun-hua, L. and Ming, L. (2013). An analysis on convergence and convergence rate estimate of elitist genetic algorithms in noisy environments. Optik - International Journal for Light and Electron Optics, 124(24):6780 - 6785.
  10. Liu, J., Saint-Pierre, D. L., Teytaud, O., et al. (2014). A mathematically derived number of resamplings for noisy optimization. In Genetic and Evolutionary Computation Conference (GECCO 2014).
  11. Merelo-Guervós, J.-J., Prieto, A., and Morán, F. (2001). Optimization of classifiers using genetic algorithms, chapter 4, pages 91-108. MIT press. ISBN: 0262162016; draft available from http://geneura.ugr.es/pub/papers/g-lvq-book.ps.gz.
  12. Merelo-Guervs;, J.-J. (2014). Using a Wilcoxon-test based partial order for selection in evolutionary algorithms with noisy fitness. Technical report, GeNeura group, university of Granada. Available at http://dx.doi.org/10.6084/m9.figshare.974598.
  13. Merelo-Guervs, J.-J., Castillo, P.-A., and Alba, E. (2010). Algorithm::Evolutionary, a flexible Perl module for evolutionary computation. Soft Computing, 14(10):1091-1109. Accesible at http://sl.ugr.es/000K.
  14. Miller, B. L. and Goldberg, D. E. (1996). Genetic algorithms, selection schemes, and the varying effects of noise. Evolutionary Computation, 4(2):113-131.
  15. Mora, A. M., Fernández-Ares, A., Guervós, J. J. M., García-Sánchez, P., and Fernandes, C. M. (2012). Effect of noisy fitness in real-time strategy games player behaviour optimisation using evolutionary algorithms. J. Comput. Sci. Technol., 27(5):1007-1023.
  16. Mora, A. M., Montoya, R., Merelo, J. J., Snchez, P. G., Castillo, P. A., Laredo, J. L. J., Martnez, A. I., and Espacia, A. (2010). Evolving bots ai in unreal. In di Chio et al., C., editor, Applications of Evolutionary Computing, Part I, volume 6024 of Lecture Notes in Computer Science, pages 170-179, Istanbul, Turkey. Springer-Verlag.
  17. Qian, C., Yu, Y., and Zhou, Z.-H. (2013). Analyzing evolutionary optimization in noisy environments. CoRR, abs/1311.4987.
  18. Rada-Vilela, J., Johnston, M., and Zhang, M. (2014). Population statistics for particle swarm optimization: Resampling methods in noisy optimization problems. Swarm and Evolutionary Computation, 0(0):-. In press.
  19. Rudolph, G. (2001). A partial order approach to noisy fitness functions. In Proceedings of the IEEE Conference on Evolutionary Computation, ICEC, volume 1, pages 318-325.
  20. Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80-83.
Download


Paper Citation


in Harvard Style

Merelo J., Castillo P., Mora A., Fernández-Ares A., Esparcia-Alcázar A., Cotta C. and Rico N. (2014). Studying and Tackling Noisy Fitness in Evolutionary Design of Game Characters . In Proceedings of the International Conference on Evolutionary Computation Theory and Applications - Volume 1: ECTA, (IJCCI 2014) ISBN 978-989-758-052-9, pages 76-85. DOI: 10.5220/0005085700760085


in Bibtex Style

@conference{ecta14,
author={J. J. Merelo and Pedro A. Castillo and Antonio Mora and Antonio Fernández-Ares and Anna I. Esparcia-Alcázar and Carlos Cotta and Nuria Rico},
title={Studying and Tackling Noisy Fitness in Evolutionary Design of Game Characters},
booktitle={Proceedings of the International Conference on Evolutionary Computation Theory and Applications - Volume 1: ECTA, (IJCCI 2014)},
year={2014},
pages={76-85},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005085700760085},
isbn={978-989-758-052-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Evolutionary Computation Theory and Applications - Volume 1: ECTA, (IJCCI 2014)
TI - Studying and Tackling Noisy Fitness in Evolutionary Design of Game Characters
SN - 978-989-758-052-9
AU - Merelo J.
AU - Castillo P.
AU - Mora A.
AU - Fernández-Ares A.
AU - Esparcia-Alcázar A.
AU - Cotta C.
AU - Rico N.
PY - 2014
SP - 76
EP - 85
DO - 10.5220/0005085700760085