Dynamic Feature Selection with Wrapper Model and Ensemble Approach based on Measures of Local Relevances and Group Diversity using Genetic Algorithm

Marek Kurzynski, Pawel Trajdos, Maciej Krysmann

2015

Abstract

In the paper the novel feature selection method, using wrapper model and ensemble approach, is presented. In the proposed method features are selected dynamically, i.e. separately for each classified object. First, a set of identical one-feature classifiers using different single feature is created and next the ensemble of features (classifiers) is selected as a solution of optimization problem using genetic algorithm. As an optimality criterion, the sum of measures of features relevance and diversity of ensemble of features is adopted. Both measures are calculated using original concept of randomized reference classifier, which on average acts like classifier with evaluated feature. The performance of the proposed method was compared against six state-of- art feature selection methods using nine benchmark databases. The experimental results clearly show the effectiveness of the dynamic mode and ensemble approach in feature selection procedure.

References

  1. Bache, K. and Lichman, M. (2013). UCI machine learning repository.
  2. Bolon-Canedo, V., Sanchez-Marono, N., and AlonzoBetandos, A. (2012). A review of feature selection methods on synthetic data. Knowledge Information System, 34:483-519.
  3. Chandrashekar, G. and Sahin, F. (2014). A survey on feature selection methods. Computers and Electrical Engineering, 40:16-28.
  4. Dems?ar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1-30.
  5. Dietterich, T. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10:1895-1923.
  6. Duda, R., Hart, P., and Stork, D. (2012). Pattern Classification. Wiley Interscience, New York.
  7. Duin, R., Juszczak, P., Pekalska, E., and et al. (2007). A matlab toolbox for pattern recognition.
  8. Goldberg, D. (1989). Genetic Algorithms in Search, Optimization and Machine learning. Addison-Wesley Publishing Company, London.
  9. Gu, Q., Li, Z., and Han, J. (2012). Generalized fisher score for feature selection. CoRR, abs/1202.3725.
  10. Guyon, I. and Elisseeff, A. (2003). An introduction to variable and feature selection. Journal on Machine Learning Research, 3:1157-1182.
  11. Kuncheva, L. (2004a). Combining Pattern Classifier: Methods and Algorithms. Wiley-Interscience, London.
  12. Kuncheva, L. (2004b). Ludmila kuncheva collection.
  13. Lysiak, R., Kurzynski, M., and Woloszynski, T. (2014). Optimal selection of ensemble classifiers using measures of competence and diversity of base classifiers. Neurocomputing, 126:29-35.
  14. Saeys, Y., Abeel, T., and Van de Peer, Y. (2008). Robust feature selection using ensemble feature selection techiques. Lecture Notes in Artificial Intelligence, 5212:313-325.
  15. Saeys, Y., Inza, I., and Larranaga, P. (2007). A review of feature selection techniques in bioinformatics. Bioinformatics, 23:2507-2517.
  16. Wang, H., Khoshgoftaar, T., and Napolitano, A. (2010). A comparative study of ensemble feature selection techniques for software defect prediction. In 2010 Ninth Int. Conf. on Machine Learning and Applications, pages 135-140. IEEE Computer Society.
  17. Woloszynski, T. (2013). Classifier competence based on probabilistic modeling (ccprmod.m) at matlab central file exchange.
  18. Woloszynski, T. and Kurzynski, M. (2010). A measure of competence based on randomized reference classifier for dynamic ensemble selection. In 2010 Twentieth International Conference on Pattern Recognition,, pages 4194-4197. Int. Association on Pattern Recognition.
  19. Woloszynski, T. and Kurzynski, M. (2011). A probabilistic model of classifier competence for dynamic ensemble selection. Pattern Recognition, 44(10-11):2656-2668.
  20. Woloszynski, T., Kurzynski, M., Podsiadlo, P., and Stachowiak, G. (2012). A measure of competence based on random classification for dynamic ensemble selection. Information Fusion, 13:207-213.
  21. Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5:214-259.
  22. Zafra, A., Pechenizkiy, M., and Ventura, S. (2010). Reducing dimensionality in multiple instance learning with a filter method. Lecture Notes in Computer Science, 6077:35-44.
Download


Paper Citation


in Harvard Style

Kurzynski M., Trajdos P. and Krysmann M. (2015). Dynamic Feature Selection with Wrapper Model and Ensemble Approach based on Measures of Local Relevances and Group Diversity using Genetic Algorithm . In Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 1: ECTA, ISBN 978-989-758-157-1, pages 167-173. DOI: 10.5220/0005591401670173


in Bibtex Style

@conference{ecta15,
author={Marek Kurzynski and Pawel Trajdos and Maciej Krysmann},
title={Dynamic Feature Selection with Wrapper Model and Ensemble Approach based on Measures of Local Relevances and Group Diversity using Genetic Algorithm},
booktitle={Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 1: ECTA,},
year={2015},
pages={167-173},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005591401670173},
isbn={978-989-758-157-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 1: ECTA,
TI - Dynamic Feature Selection with Wrapper Model and Ensemble Approach based on Measures of Local Relevances and Group Diversity using Genetic Algorithm
SN - 978-989-758-157-1
AU - Kurzynski M.
AU - Trajdos P.
AU - Krysmann M.
PY - 2015
SP - 167
EP - 173
DO - 10.5220/0005591401670173