Evolving Classifier Ensembles using Dynamic Multi-objective Swarm Intelligence

Jean-François Connolly, Eric Granger, Robert Sabourin

2013

Abstract

Classification systems are often designed using a limited amount of data from complex and changing pattern recognition environments. In applications where new reference samples become available over time, adaptive multi-classifier systems (AMCSs) are desirable for updating class models. In this paper, an incremental learning strategy based on an aggregated dynamical niching particle swarm optimization (ADNPSO) algorithm is proposed to efficiently evolve heterogeneous classifier ensembles in response to new reference data. This strategy is applied to an AMCS where all parameters of a pool of fuzzy ARTMAP (FAM) neural network classifiers, each one corresponding to a PSO particle, are co-optimized such that both error rate and network size are minimized. To sustain a high level of accuracy while minimizing the computational complexity, the AMCS integrates information from multiple diverse classifiers, where learning is guided by the ADNPSO algorithm that optimizes networks according both these objectives. Moreover, FAM networks are evolved to maintain (1) genotype diversity of solutions around local optima in the optimization search space, and (2) phenotype diversity in the objective space. Using local Pareto optimality, networks are then stored in an archive to create a pool of base classifiers among which cost-effective ensembles are selected on the basis of accuracy, and both genotype and phenotype diversity. Performance of the ADNPSO strategy is compared against AMCSs where learning of FAM networks is guided through mono- and multi-objective optimization, and assessed under different incremental learning scenarios, where new data is extracted from real-world video streams for face recognition. Simulation results indicate that the proposed strategy provides a level of accuracy that is comparable to that of using mono-objective optimization, yet requires only a fraction of its resources.

References

  1. Brown, G., Wyatt, J., Harris, R., and Yao, X. (2005). Diversity creation methods: a survey and categorization. Information Fusion, 29(6):5-20.
  2. Carpenter, G. A., Grossberg, S., Markuzon, N., Reynolds, J. H., and Rosen, D. B. (1992). Fuzzy artmap: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Systems, Man, and Cybernetics C, 3(5):698-713.
  3. Coello, C., Pulido, G., and Lechuga, M. (2004). Handling multiple objectives with particle swarm optimization. IEEE Transactions on Evolutionary Computation, 8(3):256-279.
  4. Connolly, J.-F., Granger, E., and Sabourin, R. (2008). Supervised incremental learning with the fuzzy artmap neural network. In Proceedings of the Artificial Neural Networks in Pattern Recognition, pages 66-77, Paris, France.
  5. Connolly, J.-F., Granger, E., and Sabourin, R. (2012a). An adaptive classification system for video-based face recognition. Information Sciences, 192(1):50-70. Swarm Intelligence and Its Applications.
  6. Connolly, J.-F., Granger, E., and Sabourin, R. (2012b). Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition. Pattern Recognition, 45(7):2460- 2477.
  7. Elwell, R. and Polikar, R. (2011). Incremental learning of concept drift in nonstationary environments. IEEE Transactions on Neural Networks, 22(10):1517-1531.
  8. Gorodnichy, D. O. (2005). Video-based framework for face recognition in video. In Second Workshop on Face Processing in Video in Proceedings of the Conference on Computer and Robot Vision, pages 325-344, Victoria, Canada.
  9. Granger, E., Henniges, P., Oliveira, L. S., and Sabourin, R. (2007). Supervised learning of fuzzy artmap neural networks through particle swarm optimization. Journal of Pattern Recognition Research, 2(1):27-60.
  10. Jain, A. K., Ross, A., and Pankanti, S. (2006). Biometrics: A tool for information security. IEEE Transactions on Information Forensics and Security, 1(2):125-143.
  11. Kennedy, J. (2007). Some issues and practices for particle swarms. In Proceedings of the IEEE International on Swarm Intelligence, pages 162-169, Honolulu, USA.
  12. Kuncheva, L. I. (2004). Classifier ensembles for changing environments. In Proceedings of the International Workshop on Multiple Classifier Systems, pages 1-15, Cagliari, Italy.
  13. Li, R., Mersch, T. R., Wen, O. X., Kaylani, A., and Anagnostopoulos, G. C. (2010). Multi-objective memetic evolution of art-based classifiers. In Proceedings of the IEEE Congress on Evolutionary Computation, pages 1-8, Barcelona, Spain.
  14. Minku, L. L., White, A. P., and Yao, X. (2010). The impact of diversity on online ensemble learning in the presence of concept drift. IEEE Transactions on Knowledge and Data Engineering, 22(5):730-742.
  15. Nickabadi, A., Ebadzadeh, M. M., and Safabakhsh, R. (2008). DNPSO: A dynamic niching particle swarm optimizer for multi-modal optimization. In Proceedings of the IEEE Congress on Evolutionary Computation, pages 26-32, Hong Kong, China.
  16. Polikar, R., Udpa, L., Udpa, S. S., and Honavar, V. (2001). Learn++ : An incremental learning algorithm for supervised neural networks. IEEE Transactions on Systems, Man, and Cybernetics C, 31(4):497-508.
  17. Roli, F., Didaci, L., and Marcialis, G. (2008). Adaptive biometric systems that can improve with use. In Ratha, N. K. and Govindaraju, V., editors, Advances in Biometrics, pages 447-471. Springer London.
  18. Valentini, G. (2003). Ensemble methods based on biasvariance analysis. PhD thesis, University of Genova, Genova, Switzerland.
Download


Paper Citation


in Harvard Style

Connolly J., Granger E. and Sabourin R. (2013). Evolving Classifier Ensembles using Dynamic Multi-objective Swarm Intelligence . In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8565-41-9, pages 206-215. DOI: 10.5220/0004269302060215


in Bibtex Style

@conference{icpram13,
author={Jean-François Connolly and Eric Granger and Robert Sabourin},
title={Evolving Classifier Ensembles using Dynamic Multi-objective Swarm Intelligence},
booktitle={Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2013},
pages={206-215},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004269302060215},
isbn={978-989-8565-41-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Evolving Classifier Ensembles using Dynamic Multi-objective Swarm Intelligence
SN - 978-989-8565-41-9
AU - Connolly J.
AU - Granger E.
AU - Sabourin R.
PY - 2013
SP - 206
EP - 215
DO - 10.5220/0004269302060215