Automatic Feature Selection for Sleep/Wake Classification with Small Data Sets

J. Foussier, P. Fonseca, X. Long, S. Leonhardt

Abstract

This paper describes an automatic feature selection algorithm integrated into a classification framework developed to discriminate between sleep and wake states during the night. The feature selection algorithm proposed in this paper uses the Mahalanobis distance and the Spearman’s ranked-order correlation as selection criteria to restrict search in a large feature space. The algorithm was tested using a leave-one-subject-out cross-validation procedure on 15 single-night PSG recordings of healthy sleepers and then compared to the results of a standard Sequential Forward Search (SFS) algorithm. It achieved comparable performance in terms of Cohen’s kappa (k = 0.62) and the Area under the Precision-Recall curve (AUCPR = 0.59), but gave a significant computational time improvement by a factor of nearly 10. The feature selection procedure, applied on each iteration of the cross-validation, was found to be stable, consistently selecting a similar list of features. It selected an average of 10.33 features per iteration, nearly half of the 21 features selected by SFS. In addition, learning curves show that the training and testing performances converge faster than for SFS and that the final training-testing performance difference is smaller, suggesting that the new algorithm is more adequate for data sets with a small number of subjects.

References

  1. Abdullah, M. (1990). On a robust correlation coefficient. The Statistician, 39(4):455-460.
  2. Cohen, J. (1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1):37-46.
  3. Davis, J. and Goadrich, M. (2006). The relationship between Precision-Recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning ICML 06, volume 10 of ICML 7806, pages 233-240, Pittsburgh (USA). ACM Press.
  4. Devot, S., Bianchi, A. M., Naujokat, E., Mendez, M., Brauers, A., and Cerutti, S. (2007). Sleep monitoring through a textile recording system. In IEEE Engineering in Medicine and Biology Society, volume 2007, pages 2560-2563.
  5. Devot, S., Dratwa, R., and Naujokat, E. (2010). Sleep/wake detection based on cardiorespiratory signals and actigraphy. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), pages 5089-5092. IEEE.
  6. Duda, R. O., Hart, P. E., and Stork, D. G. (2001). Pattern Classification. Wiley, 2nd edition.
  7. Fawcett, T. (2004). ROC Graphs: Notes and Practical Considerations for Researchers. ReCALL, 31(HPL-2003- 4):1-38.
  8. Friedman, J. H. (2012). Regularized Discriminant Analysis. Journal of the American Statistical Association, 84(405):165-175.
  9. Haibo, H. and Garcia, E. (2009). Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, 21(9):1263-1284.
  10. Long, X., Fonseca, P., Foussier, J., Haakma, R., and Aarts, R. (2012). Using Dynamic Time Warping for Sleep and Wake Discrimination. In IEEE Engineering in Medicine and Biology Society - International Conference on Biomedical and Health Informatics (BHI), volume 25, pages 886-889, Hong Kong/Shenzhen (China).
  11. Provost, F., Fawcett, T., and Kohavi, R. (1998). The case against accuracy estimation for comparing induction algorithms. In Proceedings of the 15th International Conference on Machine Learning, volume 445. JSTOR.
  12. Rechtschaffen, A. and Bergmann, B. (1995). Sleep deprivation in the rat by the disk-over-water method. Behavioural Brain Research, 69(1-2):55-63.
  13. Redmond, S. J., de Chazal, P., O'Brien, C., Ryan, S., McNicholas, W. T., and Heneghan, C. (2007). Sleep staging using cardiorespiratory signals. Somnologie, 11(4):245-256.
  14. Whitney, A. (1971). A Direct Method of Nonparametric Measurement Selection. IEEE Transactions on Computers, C-20(9):1100-1103.
  15. Zoubek, L., Charbonnier, S., Lesecq, S., Buguet, A., and Chapotot, F. (2007). Feature selection for sleep/wake stages classification using data driven methods. Biomedical Signal Processing and Control, 2(3):171-179.
Download


Paper Citation


in Harvard Style

Foussier J., Fonseca P., Long X. and Leonhardt S. (2013). Automatic Feature Selection for Sleep/Wake Classification with Small Data Sets . In Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2013) ISBN 978-989-8565-35-8, pages 178-184. DOI: 10.5220/0004245401780184


in Bibtex Style

@conference{bioinformatics13,
author={J. Foussier and P. Fonseca and X. Long and S. Leonhardt},
title={Automatic Feature Selection for Sleep/Wake Classification with Small Data Sets},
booktitle={Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2013)},
year={2013},
pages={178-184},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004245401780184},
isbn={978-989-8565-35-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2013)
TI - Automatic Feature Selection for Sleep/Wake Classification with Small Data Sets
SN - 978-989-8565-35-8
AU - Foussier J.
AU - Fonseca P.
AU - Long X.
AU - Leonhardt S.
PY - 2013
SP - 178
EP - 184
DO - 10.5220/0004245401780184