Optimal Conjugate Gradient Algorithm for Generalization of Linear Discriminant Analysis Based on L1 Norm

Kanishka Tyagi, Nojun Kwak, Michael Manry

2014

Abstract

This paper analyzes a linear discriminant subspace technique from an L1 point of view. We propose an efficient and optimal algorithm that addresses several major issues with prior work based on, not only the L1 based LDA algorithm but also its L2 counterpart. This includes algorithm implementation, effect of outliers and optimality of parameters used. The key idea is to use conjugate gradient to optimize the L1 cost function and to find an learning factor during the update of the weight vector in the subspace. Experimental results on UCI datasets reveal that the present method is a significant improvement over the previous work. Mathematical treatment for the proposed algorithm and calculations for learning factor are the main subject of this paper.

References

  1. Bell, A. and sejnowski, T. (1995). An informationmaximization approach to blind separation and blind deconvolution. Neural Computation, 7.
  2. Cai, X., K.Tyagi, and Manry, M. (2011). An optimal construction and training of second order rbf network for approximation and illumination invariant image segmentation.
  3. Cao, L., Chua, K., Chong, W., Lee, H., and Gu, Q. (2003). A comparison of pca, kpca and ica for dimensionality reduction in support vector machine. Neurocomputing, 55.
  4. Chong, E. and Stanislaw, Z. (2013). An introduction to optimization. Wiley, USA, 3rd edition.
  5. Claerbout, J. F. and Muir, F. (1973). Robust modeling with erratic data.
  6. Duda, R., Hart, P., and Stork, D. (2012). Pattern Classification. Wiley-interscience, USA, 2nd edition.
  7. Fisher, R. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2).
  8. Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. Academic Press, USA, 2nd edition.
  9. Haykin, S. (2009). Neural networks and learning machines. Prentice Hall, USA, 3rd edition.
  10. J. A. Scales, A. Gersztenkorn, S. T. and Lines, L. R. (1988). Robust optimization methods in geophysical inverse theory.
  11. Ji, J. (2006). Cgg method for robust inversion and its application to velocity-stack inversion.
  12. Koren, Y. and Carmel, L. (2008). Robust linear dimensionality reduction. IEEE Transactions on Visualization and Computer Graphics, 10(4):459-470.
  13. Kwak, N. (2008). Principal component analysis based on l-1 norm maximization. IEEE Trans. Pattern Analysis and Machine Intelligence, 30(9):1672-1680.
  14. Kwak, N. and Choi, C.-H. (2003). Feature extraction based on ica for binary classification problems. IEEE Trans. on Knowledge and Data Engineering, 15(6):1374- 1388.
  15. Kwon, O.-W. and Lee, T.-W. (2004). Phoneme recognition using ica-based feature extraction and transformation. Signal Processing, 84(6).
  16. Li, X., Hu, W., Wang, H., and Zhang, Z. (2010). Linear discriminant analysis using rotational invariant l1 norm. Neurocomputing, 73.
  17. Malalur, S. S. and Manry, M. T. (2010). Multiple optimal learning factors for feed-forward networks.
  18. Oh, J. and Kwak, N. (2013). Generalization of linear discriminant analysis using lp-norm. Pattern Recognition Letters, 34(6):679-685.
  19. Olavi, N. (1993). Convergence of iterations for linear equations. Birkhauser, USA, 3rd edition.
  20. Sugiyama, M. (2007). Dimensionality reduction of multimodal labeled data by fisher discriminant analysis. Journal of Machine Learning Research, 8:1027-1061.
  21. Theodoridis, S. and Koutroumbas, K. (2009). Pattern Recognition. Academic Press, USA, 4th edition.
Download


Paper Citation


in Harvard Style

Tyagi K., Kwak N. and Manry M. (2014). Optimal Conjugate Gradient Algorithm for Generalization of Linear Discriminant Analysis Based on L1 Norm . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 207-212. DOI: 10.5220/0004825402070212


in Bibtex Style

@conference{icpram14,
author={Kanishka Tyagi and Nojun Kwak and Michael Manry},
title={Optimal Conjugate Gradient Algorithm for Generalization of Linear Discriminant Analysis Based on L1 Norm},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={207-212},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004825402070212},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Optimal Conjugate Gradient Algorithm for Generalization of Linear Discriminant Analysis Based on L1 Norm
SN - 978-989-758-018-5
AU - Tyagi K.
AU - Kwak N.
AU - Manry M.
PY - 2014
SP - 207
EP - 212
DO - 10.5220/0004825402070212