Particles Gradient: A New Approach to Perform MLP Neural Networks Training based on Particles Swarm Optimization

César Daltoé Berci, Celso Pascoli Bottura

2009

Abstract

The use of heuristic algorithms in neural networks training is not a new subject. Several works have already accomplished good results, however not competitive with procedural methods for problems where the gradient of the error is well defined. The present document proposes an alternative for neural networks training using PSO(Particle Swarm Optimization) to evolve the training process itself and not to evolve directly the network parameters as usually. This way we get quite superior results and obtain a method clearly faster than others known methods for training neural networks using heuristic algorithms.

References

  1. R. Battiti and F. Masulli. Bfgs optimization for faster and automated supervised learning. INNC 90 Paris, International Neural Network Conference,, pages 757-760, 1990.
  2. Csar Dalto Berci. Observadores Inteligentes de Estado: Propostas. Tese de Mestrado, LCSI/FEEC/UNICAMP, Campinas, Brasil, 2008.
  3. Csar Dalto Berci and Celso Pascoli Bottura. Modulao de velocidade em otimizao por enxame de partculas. Proceedings of 7th Brazilian Conference on Dynamics, Control and Applications. Presidente Prudente, Brasil, 7:241-248, 2008.
  4. Csar Dalto Berci and Celso Pascoli Bottura. Observador inteligente adaptativo neural no baseado em modelo para sistemas no lineares. Proceedings of 7th Brazilian Conference on Dynamics, Control and Applications. Presidente Prudente, Brasil, 7:209-215, 2008.
  5. Csar Dalto Berci and Celso Pascoli Bottura. Speed modulation: A new approach to improve pso performance. Accepted Article: IEEE Swarm Intelligence Symposium 2008, Sheraton Westport at St. Louis, Missouri, 2008.
  6. F. den van Bergh. An Analysis of Particle Swarm Optimization. PhD thesis, Departament of Computer Science, University of Pretoria, 2002.
  7. Jü rgen Branke. Evolutionary algorithms for neural network design and training. In 1st Nordic Workshop on Genetic Algorithms and its Applications, 1995. Vaasa, Finland, January 1995.
  8. D.J. Chalmers. The evolution of learning: An experiment in genetic connectionism. Proceedings of the 1990 Connectionist Summer School, pages 81-90, 1990.
  9. A. Fiszelew, P. Britos, A. Ochoa, H. Merlino, E. Fernndez, and R. Garca-Martnez. Finding optimal neural network architecture using genetic algorithms. Software & Knowledge Engineering Center. Buenos Aires Institute of Technology. Intelligent Systems Laboratory. School of Engineering. University of Buenos Aires., 2004.
  10. C. Fyfe. Artificial Neural Networks. Department of Computing and Information Systems, The University of Paisley, Edition 1.1, 1996.
  11. F. Heppner and U. Grenander. A Stochastic Nonlinear Model For Coordinate Bird Flocks. AAAS Publications, Washington, DC, 1990.
  12. J. Kennedy and R.C Eberhart. Particle swarm optimization. Proceedings of IEEE Int. Conf. on Neural Network, Piscataway, NJ:1942-1948, 1995.
  13. D.G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley, 2nd edition, 1984.
  14. M.F. Mller. Learning by conjugate gradients. The 6th International Meeting of Young Computer Scientists, 1990.
  15. M.F. Mller. A scaled conjugate gradient algorithm for fast supervised learning. Computer Science Department, University of Aarhus Denmark, 6:525-533, 1990.
  16. D. Montana and L. Davis. Training feedforward neural networks using genetic algorithms. Proceedings of the International Joint Conference on Artificial Intelligence, pages 762-767, 1989.
  17. C.W. Reynolds. Flocks, herds, and schools: A distributed behavior model. Computer Graphics, 21(4):25-34, 1987.
  18. D. E. Rumelhart, R. Durbin, R. Golden, and Y. Chauvin. Backpropagation: The basic theory. Lawrence Erlbaum Associates, Inc., 1995.
  19. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation, in: Parallel distributed processing: Exploration in the microstructure of cognition. Eds. D.E. Rumelhart, J.L. McClelland, MIT Press, Cambridge, MA., pages 318- 362, 1986.
  20. Udo Seiffert. Multiple layer perceptron training using genetic algorithms. ESANN'2001 proceedings - European Symposium on Artificial Neural Networks, pages 159-164, 2001.
  21. Zhi-Hua Zhou, Jian-Xin Wu, Yuan Jiang, and Shi-Fu Chen. Genetic algorithm based selective neural network ensemble. Proceedings of the 17th International Joint Conference on Artificial Intelligence., 2:797-802, 2001.
Download


Paper Citation


in Harvard Style

Berci C. and Bottura C. (2009). Particles Gradient: A New Approach to Perform MLP Neural Networks Training based on Particles Swarm Optimization . In Proceedings of the 5th International Workshop on Artificial Neural Networks and Intelligent Information Processing - Volume 1: Workshop ANNIIP, (ICINCO 2009) ISBN 978-989-674-002-3, pages 115-123. DOI: 10.5220/0002214501150123


in Bibtex Style

@conference{workshop anniip09,
author={César Daltoé Berci and Celso Pascoli Bottura},
title={Particles Gradient: A New Approach to Perform MLP Neural Networks Training based on Particles Swarm Optimization},
booktitle={Proceedings of the 5th International Workshop on Artificial Neural Networks and Intelligent Information Processing - Volume 1: Workshop ANNIIP, (ICINCO 2009)},
year={2009},
pages={115-123},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002214501150123},
isbn={978-989-674-002-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Workshop on Artificial Neural Networks and Intelligent Information Processing - Volume 1: Workshop ANNIIP, (ICINCO 2009)
TI - Particles Gradient: A New Approach to Perform MLP Neural Networks Training based on Particles Swarm Optimization
SN - 978-989-674-002-3
AU - Berci C.
AU - Bottura C.
PY - 2009
SP - 115
EP - 123
DO - 10.5220/0002214501150123