# Algorithms for Regularized Linear Discriminant Analysis

### Jan Kalina, Jurjen Duintjer Tebbens

#### Abstract

This paper is focused on regularized versions of classification analysis and their computation for high-dimensional data. A variety of regularized classification methods has been proposed and we critically discuss their computational aspects. We formulate several new algorithms for regularized linear discriminant analysis, which exploits a regularized covariance matrix estimator towards a regular target matrix. Numerical linear algebra considerations are used to propose tailor-made algorithms for specific choices of the target matrix. Further, we arrive at proposing a new classification method based on L2-regularization of group means and the pooled covariance matrix and accompany it by an efficient algorithm for its computation.

#### References

- Barlow, J., Bosner, N., and Drmac, Z. (2005). A new stable bidiagonal reduction algorithm. Linear Algebra and its Applications, 397:35-84.
- Chen, X., Kim, Y., and Wang, Z. (2012). Efficient minimax estimation of a class of high-dimensional sparse precision matrices. IEEE Transactions on Signal Processing, 60:2899-2912.
- Davies, P. (2014). Data Analysis and Approximate Models: Model Choice, Location-Scale, Analysis of Variance, Nonparametric Regression and Image Analysis. Chapman & Hall/CRC, Boca Raton.
- Duintjer Tebbens, J. and Schlesinger, P. (2007). Improving implementation of linear discriminant analysis for the high dimension/small sample size problem. Computational Statistics & Data Analysis, 52:423-437.
- Filzmoser, P. and Todorov, V. (2011). Review of robust multivariate statistical methods in high dimension. Analytica Chinica Acta, 705:2-14.
- Guo, Y., Hastie, T., and Tibshirani, R. (2007). Regularized discriminant analysis and its application in microarrays. Biostatistics, 8:86-100.
- Haff, L. (1980). Empirical bayes estimation of the multivariate normal covariance matrix. Annals of Statistics, 1980:586-597.
- Hastie, T., Tibshirani, R., and Friedman, J. (2008). The elements of statistical learning. Springer, New York, 2nd edition.
- Kalina, J. (2012). Highly robust statistical methods in medical image analysis. Biocybernetics and Biomedical Engineering, 32(2):3-16.
- Kalina, J. (2014). Classification analysis methods for high-dimensional genetic data. Biocybernetics and Biomedical Engineering, 34:10-18.
- Kalina, J. and Zvárová, J. (2013). Decision support systems in the process of improving patient safety. In E-health Technologies and Improving Patient Safety: Exploring Organizational Factors, pages 71-83. IGI Global, Hershey.
- Kogan, J. (2007). Introduction to clustering large and highdimensional data. Cambridge University Press, Cambridge.
- Pourahmadi, M. (2013). High-dimensional covariance estimation. Wiley, New York.
- Schäfer, J. and Strimmer, K. (2005). A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 32:1-30.
- Sreekumar et al., A. (2009). Metabolomic profiles delineate potential role for sarcosine in prostate cancer progression. Nature, 457:910-914.
- Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1:197-206.
- Tibshirani, R., Hastie, T., and Narasimhan, B. (2003). Class prediction by nearest shrunken centroids, with applications to dna microarrays. Statistical Science, 18:104-117.
- Xanthopoulos, P., Pardalos, P., and Trafalis, T. (2013). Robust data mining. Springer, New York.

#### Paper Citation

#### in Harvard Style

Kalina J. and Duintjer Tebbens J. (2015). **Algorithms for Regularized Linear Discriminant Analysis** . In *Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2015)* ISBN 978-989-758-070-3, pages 128-133. DOI: 10.5220/0005234901280133

#### in Bibtex Style

@conference{bioinformatics15,

author={Jan Kalina and Jurjen Duintjer Tebbens},

title={Algorithms for Regularized Linear Discriminant Analysis},

booktitle={Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2015)},

year={2015},

pages={128-133},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0005234901280133},

isbn={978-989-758-070-3},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the International Conference on Bioinformatics Models, Methods and Algorithms - Volume 1: BIOINFORMATICS, (BIOSTEC 2015)

TI - Algorithms for Regularized Linear Discriminant Analysis

SN - 978-989-758-070-3

AU - Kalina J.

AU - Duintjer Tebbens J.

PY - 2015

SP - 128

EP - 133

DO - 10.5220/0005234901280133