# EFFICIENT CLASSIFICATION FOR LARGE-SCALE PROBLEMS BY MULTIPLE LDA SUBSPACES

### Martina Uray, Peter M. Roth, Horst Bischof

#### Abstract

In this paper we consider the limitations of Linear Discriminative Analysis (LDA) when applying it for largescale problems. Since LDA was originally developed for two-class problems the obtained transformation is sub-optimal if multiple classes are considered. In fact, the separability between the classes is reduced, which decreases the classification power. To overcome this problem several approaches including weighting strategies and mixture models were proposed. But these approaches are complex and computational expensive. Moreover, they were only tested for a small number of classes. In contrast, our approach allows to handle a huge number of classes showing excellent classification performance at low computational costs. The main idea is to split the original data into multiple sub-sets and to compute a single LDA space for each sub-set. Thus, the separability in the obtained subspaces is increased and the overall classification power is improved. Moreover, since smaller matrices have to be handled the computational complexity is reduced for both, training and classification. These benefits are demonstrated on different publicly available datasets. In particular, we consider the task of object recognition, where we can handle up to 1000 classes.

#### References

- Belhumeur, P. N., Hespanha, J. P., and Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(7):711 - 720.
- Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum Likelihood from Incomplete Data via the EM Algorithm. Royal Statistical Society, 39(1):1 - 38.
- Fisher, R. A. (1936). The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7:179- 188.
- Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. Academic Press.
- Geusebroek, J. M., Burghouts, G. J., and Smeulders, A. W. M. (2005). The Amsterdam Library of Object Images. Computer Vision, 61(1):103 - 112.
- Hunt, M. (1979). A statistical approach to metrics for word and syllable recognition. In Meeting of the Acoustical Society of American, volume 66, pages 535 - 536.
- Kim, H., Kim, D., and Bang, S. Y. (2002). Face Recognition Using LDA Mixture Model. In Proc. Intern. Conf. on Pattern Recognition, volume 2, pages 486 - 489.
- Kim, H., Kim, D., and Bang, S. Y. (2003). Extensions of LDA by PCA Mixture Model and Class-Wise Features. Pattern Recognition, 36(5):1095 - 1105.
- Kim, T.-K., Kittler, J., and Cipolla, R. (2007). Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations. IEEE Trans. on Pattern Analysis and Machine Intelligence, 29(6):1005 - 1018.
- Loog, M., Duin, R. P. W., and Haeb-Umbach, R. (2001). Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria. IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(7):762 - 766.
- MartÃnez, A. M. and Zhu, M. (2005). Where are Linear Feature Extraction Methods Applicable? IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(12):1934 - 1944.
- Nene, S. A., Nayar, S. K., and Murase, H. (1996). Columbia Object Image Library (COIL-100). Technical Report CUCS-006-96, Columbia University.
- Rao, C. R. (1948). The Utilization of Multiple Measurements in Problems of Biological Classification. Royal Statistical Society - Series B, 10(2):159 - 203.
- Strang, G. (2006). Linear Algebra and Its Applications. Brooks/Cole.
- Swets, D. L. and Weng, J. (1996). Using Discriminant Eigenfeatures for Image Retrieval. IEEE Trans. on Pattern Analysis and Machine Intelligence, 18(8):831 - 837.
- Tipping, M. E. and Bishop, C. M. (1999). Mixtures of Probabilistic Principal Component Analyzers. Neural Computation, 11(2):443 - 482.
- Zhou, D. and Yang, X. (2004). Face Recognition Using Direct-Weighted LDA. In Proc. of the Pacific Rim Int. Conference on Artificial Intelligence, pages 760 - 768.

#### Paper Citation

#### in Harvard Style

Uray M., M. Roth P. and Bischof H. (2009). **EFFICIENT CLASSIFICATION FOR LARGE-SCALE PROBLEMS BY MULTIPLE LDA SUBSPACES** . In *Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)* ISBN 978-989-8111-69-2, pages 299-306. DOI: 10.5220/0001754802990306

#### in Bibtex Style

@conference{visapp09,

author={Martina Uray and Peter M. Roth and Horst Bischof},

title={EFFICIENT CLASSIFICATION FOR LARGE-SCALE PROBLEMS BY MULTIPLE LDA SUBSPACES},

booktitle={Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)},

year={2009},

pages={299-306},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0001754802990306},

isbn={978-989-8111-69-2},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)

TI - EFFICIENT CLASSIFICATION FOR LARGE-SCALE PROBLEMS BY MULTIPLE LDA SUBSPACES

SN - 978-989-8111-69-2

AU - Uray M.

AU - M. Roth P.

AU - Bischof H.

PY - 2009

SP - 299

EP - 306

DO - 10.5220/0001754802990306