Linear Discriminant Analysis based on Fast Approximate SVD

Nassara Elhadji Ille Gado, Edith Grall-Maës, Malika Kharouf

2017

Abstract

We present an approach for performing linear discriminant analysis (LDA) in the contemporary challenging context of high dimensionality. The projection matrix of LDA is usually obtained by simultaneously maximizing the between-class covariance and minimizing the within-class covariance. However it involves matrix eigendecomposition which is computationally expensive in both time and memory requirement when the number of samples and the number of features are large. To deal with this complexity, we propose to use a recent dimension reduction method. The technique is based on fast approximate singular value decomposition (SVD) which has deep connections with low-rank approximation of the data matrix. The proposed approach, appSVD+LDA, consists of two stages. The first stage leads to a set of artificial features based on the original data. The second stage is the classical LDA. The foundation of our approach is presented and its performances in term of accuracy and computation time in comparison with some state-of-the-art techniques are provided for different real data sets.

References

  1. Achlioptas, D. (2003). Database-friendly random projections: Johnson-lindenstrauss with binary coins. Journal of computer and System Sciences, 66(4):671-687.
  2. Boutsidis, C., Zouzias, A., Mahoney, M. W., and Drineas, P. (2015). Randomized dimensionality reduction formeans clustering. IEEE Transactions on Information Theory, 61(2):1045-1062.
  3. Cardoso, Â. and Wichert, A. (2012). Iterative random projections for high-dimensional data clustering. Pattern Recognition Letters, 33(13):1749-1755.
  4. Chen, L., Man, H., and Nefian, A. V. (2005). Face recognition based on multi-class mapping of fisher scores. Pattern Recognition, 38(6):799-811.
  5. Duda, R. O., Hart, P. E., and Stork, D. G. (2012). Pattern classification. John Wiley & Sons.
  6. Friedman, J. H. (1989). Regularized discriminant analysis. Journal of the American statistical association, 84(405):165-175.
  7. Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA.
  8. Lee, Y. K., Lee, E. R., and Park, B. U. (2012). Principal component analysis in very high-dimensional spaces. Statistica Sinica, pages 933-956.
  9. Liu, H. and Chen, W.-S. (2009). A novel random projection model for linear discriminant analysis based face recognition. In 2009 International Conference on Wavelet Analysis and Pattern Recognition, pages 112-117. IEEE.
  10. Menon, A. K. and Elkan, C. (2011). Fast algorithms for approximating the singular value decomposition. ACM Transactions on Knowledge Discovery from Data (TKDD), 5(2):13.
  11. Moulin, C., Largeron, C., Ducottet, C., Géry, M., and Barat, C. (2014). Fisher linear discriminant analysis for text-image combination in multimedia information retrieval. Pattern Recognition, 47(1):260-269.
  12. Welling, M. (2005). Fisher linear discriminant analysis. Department of Computer Science, University of Toronto, 3:1-4.
  13. Ye, J. and Li, Q. (2004). Lda/qr: an efficient and effective dimension reduction algorithm and its theoretical foundation. Pattern recognition, 37(4):851-854.
  14. Ye, J., Li, Q., Xiong, H., Park, H., Janardan, R., and Kumar, V. (2005). Idr/qr: an incremental dimension reduction algorithm via qr decomposition. IEEE Transactions on Knowledge and Data Engineering, 17(9):1208- 1222.
  15. Yu, H. and Yang, J. (2001). A direct lda algorithm for highdimensional data with application to face recognition. Pattern recognition, 34(10):2067-2070.
Download


Paper Citation


in Harvard Style

Elhadji Ille Gado N., Grall-Maës E. and Kharouf M. (2017). Linear Discriminant Analysis based on Fast Approximate SVD . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 359-365. DOI: 10.5220/0006148603590365


in Bibtex Style

@conference{icpram17,
author={Nassara Elhadji Ille Gado and Edith Grall-Maës and Malika Kharouf},
title={Linear Discriminant Analysis based on Fast Approximate SVD},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2017},
pages={359-365},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006148603590365},
isbn={978-989-758-222-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Linear Discriminant Analysis based on Fast Approximate SVD
SN - 978-989-758-222-6
AU - Elhadji Ille Gado N.
AU - Grall-Maës E.
AU - Kharouf M.
PY - 2017
SP - 359
EP - 365
DO - 10.5220/0006148603590365