Stability of Ensemble Feature Selection on High-Dimension and Low-Sample Size Data - Influence of the Aggregation Method

David Dernoncourt, Blaise Hanczar, Jean-Daniel Zucker

2014

Abstract

Feature selection is an important step when building a classifier. However, the feature selection tends to be unstable on high-dimension and small-sample size data. This instability reduces the usefulness of selected features for knowledge discovery: if the selected feature subset is not robust, domain experts can have little trust that they are relevant. A growing number of studies deal with feature selection stability. Based on the idea that ensemble methods are commonly used to improve classifiers accuracy and stability, some works focused on the stability of ensemble feature selection methods. So far, they obtained mixed results, and as far as we know no study extensively studied how the choice of the aggregation method influences the stability of ensemble feature selection. This is what we study in this preliminary work. We first present some aggregation methods, then we study the stability of ensemble feature selection based on them, on both artificial and real data, as well as the resulting classification performance.

References

  1. Abeel, T., Helleputte, T., Van de Peer, Y., Dupont, P., and Saeys, Y. (2010). Robust biomarker identification for cancer diagnosis with ensemble feature selection methods. Bioinformatics, 26(3):392-398.
  2. Dernoncourt, D., Hanczar, B., and Zucker, J.-D. (2014). Analysis of feature selection stability on high dimension and small sample data. Computational Statistics and Data Analysis, 71(0):681 - 693.
  3. Ein-Dor, L., Zuk, O., and Domany, E. (2006). Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proceedings of the National Academy of Sciences, 103(15):5923-5928.
  4. Han, Y., Yang, Y., and Zhou, X. (2013). Co-regularized ensemble for feature selection. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI'13, pages 1380-1386. AAAI Press.
  5. Han, Y. and Yu, L. (2012). A variance reduction framework for stable feature selection. Statistical Analysis and Data Mining, 5(5):428-445.
  6. Haury, A.-C., Gestraud, P., and Vert, J.-P. (2011). The influence of feature selection methods on accuracy, stability and interpretability of molecular signatures. PLoS ONE, 6(12):e28210.
  7. Jain, A. K. and Chandrasekaran, B. (1982). 39 dimensionality and sample size considerations in pattern recognition practice. Handbook of Statistics, 2:835-855.
  8. Kuncheva, L. I. (2007). A stability index for feature selection. In Devedzic, V., editor, Artificial Intelligence and Applications, pages 421-427. IASTED/ACTA Press.
  9. Liu, H., Liu, L., and Zhang, H. (2010). Ensemble gene selection by grouping for microarray data classification. Journal of biomedical informatics, 43:81-87.
  10. Meinshausen, N. and Bhlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(4):417-473.
  11. Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1-2):1-39.
  12. Saeys, Y., Abeel, T., and Peer, Y. (2008). Robust feature selection using ensemble feature selection techniques. In Daelemans, W., Goethals, B., and Morik, K., editors, Machine Learning and Knowledge Discovery in Databases, volume 5212 of Lecture Notes in Computer Science, pages 313-325. Springer Berlin Heidelberg.
  13. Simon, R. (2003). Supervised analysis when the number of candidate features (p) greatly exceeds the number of cases (n). SIGKDD Explor. Newsl., 5(2):31-36.
  14. Somol, P., Grim, J., and Pudil, P. (2009). Criteria ensembles in feature selection. In Benediktsson, J. A., Kittler, J., and Roli, F., editors, MCS, volume 5519 of Lecture Notes in Computer Science, pages 304-313. Springer.
  15. Somol, P. and Novovic?ová, J. (2010). Evaluating stability and comparing output of feature selectors that optimize feature subset cardinality. IEEE Trans. Pattern Anal. Mach. Intell., 32(11):1921-1939.
  16. Surowiecki, J. (2004). The Wisdom of Crowds: Why the Many are Smarter Than the Few and how Collective Wisdom Shapes Business, Economies, Societies, and Nations. Doubleday.
  17. Wald, R., Khoshgoftaar, T. M., and Dittman, D. J. (2013). Ensemble gene selection versus single gene selection: Which is better? In Boonthum-Denecke, C. and Youngblood, G. M., editors, FLAIRS Conference. AAAI Press.
  18. Yang, P., Hwa Yang, Y., B. Zhou, B., and Y. Zomaya, A. (2010). A review of ensemble methods in bioinformatics. Current Bioinformatics, 5(4):296-308.
Download


Paper Citation


in Harvard Style

Dernoncourt D., Hanczar B. and Zucker J. (2014). Stability of Ensemble Feature Selection on High-Dimension and Low-Sample Size Data - Influence of the Aggregation Method . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 325-330. DOI: 10.5220/0004922203250330


in Bibtex Style

@conference{icpram14,
author={David Dernoncourt and Blaise Hanczar and Jean-Daniel Zucker},
title={Stability of Ensemble Feature Selection on High-Dimension and Low-Sample Size Data - Influence of the Aggregation Method},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={325-330},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004922203250330},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Stability of Ensemble Feature Selection on High-Dimension and Low-Sample Size Data - Influence of the Aggregation Method
SN - 978-989-758-018-5
AU - Dernoncourt D.
AU - Hanczar B.
AU - Zucker J.
PY - 2014
SP - 325
EP - 330
DO - 10.5220/0004922203250330