Algorithms for Regularized Linear Discriminant Analysis

Jan Kalina, Jurjen Duintjer Tebbens

2015

Abstract

This paper is focused on regularized versions of classification analysis and their computation for high-dimensional data. A variety of regularized classification methods has been proposed and we critically discuss their computational aspects. We formulate several new algorithms for regularized linear discriminant analysis, which exploits a regularized covariance matrix estimator towards a regular target matrix. Numerical linear algebra considerations are used to propose tailor-made algorithms for specific choices of the target matrix. Further, we arrive at proposing a new classification method based on L2-regularization of group means and the pooled covariance matrix and accompany it by an efficient algorithm for its computation.

References

  1. Barlow, J., Bosner, N., and Drmac, Z. (2005). A new stable bidiagonal reduction algorithm. Linear Algebra and its Applications, 397:35-84.
  2. Chen, X., Kim, Y., and Wang, Z. (2012). Efficient minimax estimation of a class of high-dimensional sparse precision matrices. IEEE Transactions on Signal Processing, 60:2899-2912.
  3. Davies, P. (2014). Data Analysis and Approximate Models: Model Choice, Location-Scale, Analysis of Variance, Nonparametric Regression and Image Analysis. Chapman & Hall/CRC, Boca Raton.
  4. Duintjer Tebbens, J. and Schlesinger, P. (2007). Improving implementation of linear discriminant analysis for the high dimension/small sample size problem. Computational Statistics & Data Analysis, 52:423-437.
  5. Filzmoser, P. and Todorov, V. (2011). Review of robust multivariate statistical methods in high dimension. Analytica Chinica Acta, 705:2-14.
  6. Guo, Y., Hastie, T., and Tibshirani, R. (2007). Regularized discriminant analysis and its application in microarrays. Biostatistics, 8:86-100.
  7. Haff, L. (1980). Empirical bayes estimation of the multivariate normal covariance matrix. Annals of Statistics, 1980:586-597.
  8. Hastie, T., Tibshirani, R., and Friedman, J. (2008). The elements of statistical learning. Springer, New York, 2nd edition.
  9. Kalina, J. (2012). Highly robust statistical methods in medical image analysis. Biocybernetics and Biomedical Engineering, 32(2):3-16.
  10. Kalina, J. (2014). Classification analysis methods for high-dimensional genetic data. Biocybernetics and Biomedical Engineering, 34:10-18.
  11. Kalina, J. and Zvárová, J. (2013). Decision support systems in the process of improving patient safety. In E-health Technologies and Improving Patient Safety: Exploring Organizational Factors, pages 71-83. IGI Global, Hershey.
  12. Kogan, J. (2007). Introduction to clustering large and highdimensional data. Cambridge University Press, Cambridge.
  13. Pourahmadi, M. (2013). High-dimensional covariance estimation. Wiley, New York.
  14. Schäfer, J. and Strimmer, K. (2005). A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 32:1-30.
  15. Sreekumar et al., A. (2009). Metabolomic profiles delineate potential role for sarcosine in prostate cancer progression. Nature, 457:910-914.
  16. Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1:197-206.
  17. Tibshirani, R., Hastie, T., and Narasimhan, B. (2003). Class prediction by nearest shrunken centroids, with applications to dna microarrays. Statistical Science, 18:104-117.
  18. Xanthopoulos, P., Pardalos, P., and Trafalis, T. (2013). Robust data mining. Springer, New York.
Download


Paper Citation


in Harvard Style

Kalina J. and Duintjer Tebbens J. (2015). Algorithms for Regularized Linear Discriminant Analysis . In Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2015) ISBN 978-989-758-070-3, pages 128-133. DOI: 10.5220/0005234901280133


in Bibtex Style

@conference{bioinformatics15,
author={Jan Kalina and Jurjen Duintjer Tebbens},
title={Algorithms for Regularized Linear Discriminant Analysis},
booktitle={Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2015)},
year={2015},
pages={128-133},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005234901280133},
isbn={978-989-758-070-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2015)
TI - Algorithms for Regularized Linear Discriminant Analysis
SN - 978-989-758-070-3
AU - Kalina J.
AU - Duintjer Tebbens J.
PY - 2015
SP - 128
EP - 133
DO - 10.5220/0005234901280133