Exploiting Local Class Information in Extreme Learning Machine

Alexandros Iosifidis, Anastasios Tefas, Ioannis Pitas

2014

Abstract

In this paper we propose an algorithm for Single-hidden Layer Feedforward Neural networks training. Based on the observation that the learning process of such networks can be considered to be a non-linear mapping of the training data to a high-dimensional feature space, followed by a data projection process to a lowdimensional space where classification is performed by a linear classifier, we extend the Extreme Learning Machine (ELM) algorithm in order to exploit the local class information in its optimization process. The proposed Local Class Variance Extreme Learning Machine classifier is evaluated in facial image classification problems, where we compare its performance with that of other ELM-based classifiers. Experimental results show that the incorporation of local class information in the ELMoptimization process enhances classification performance.

References

  1. Bartlett, P. L. (1998). The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536.
  2. Belkin, M., Niyogi, P., and Sindhwani, V. (2007). Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399-2434.
  3. Devijver, P. and Kittler, J. (1982). Pattern Recognition: A Statistical Approach. Prentice-Hall.
  4. Duda, R., Hart, P., and Stork, D. (2000). Pattern Classification, 2nd ed. Wiley-Interscience.
  5. Helmy, T. and Rasheed, Z. (2009). Multi-category bioinformatics dataset classification using extreme learning machine. IEEE Evolutionary Computation.
  6. Huang, G. B., Chen, L., and Siew, C. K. (2006). Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks, 17(4):879-892.
  7. Huang, G. B., Zhou, H., Ding, X., and Zhang, R. (2012). Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 42(2):513-529.
  8. Huang, G. B., Zhu, Q. Y., and Siew, C. K. (2004). Extreme learning machine: a new learning scheme of feedforward neural networks. IEEE International Joint Conference on Neural Networks.
  9. Iosifidis, A., Tefas, A., and Pitas, I. (2013a). Active classification for human action recognition. IEEE International Conference on Image Processing.
  10. Iosifidis, A., Tefas, A., and Pitas, I. (2013b). Dynamic action recognition based on dynemes and extreme learning machine. Pattern Recognition Letters, 34:1890- 1898.
  11. Iosifidis, A., Tefas, A., and Pitas, I. (2013c). Minimum class variance extreme learning machine for human action recognition. IEEE Transactions on Circuits and Systems for Video Technology, 23(11):1968-1979.
  12. Iosifidis, A., Tefas, A., and Pitas, I. (2013d). Person identification from actions based on artificial neural networks. IEEE Symposium Series on Computational Intelligence.
  13. Iosifidis, A., Tefas, A., and Pitas, I. (2014a). Human action recognition based on bag of features and multi-view neural networks. IEEE International Conference on Image Processing.
  14. Iosifidis, A., Tefas, A., and Pitas, I. (2014b). Minimum variance extreme learning machine for human action recognition. IEEE International Conference on Acoustics, Speech and Signal Processing.
  15. Iosifidis, A., Tefas, A., and Pitas, I. (2014c). Semisupervised classification of human actions based on neural networks. IEEE International Conference on Pattern Recognition.
  16. Kanade, T., Tian, Y., and Cohn, J. (2000). Comprehensive database for facial expression analysis. IEEE International Conference on Automatic Face and Gesture Recognition.
  17. Kondor, R. and Lafferty, J. (2002). Diffusion kernels on graphs and other discrete input spaces. International Conference on Machine Learning.
  18. Lan, Y., Soh, Y. C., and Huang, G. B. (2008). Extreme learning machine based bacterial protein subcellular localization prediction. IEEE International Joint Conference on Neural Networks.
  19. Lee, K. C., Ho, J., and Kriegman, D. (2005). Acquiriing linear subspaces for face recognition under varialbe lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5):684-698.
  20. Lyons, M., Akamatsu, S., Kamachi, M., and Gyoba, J. (1998). Coding facial expressions with gabor wavelets. IEEE International Conference on Automatic Face and Gesture Recognition.
  21. Martinez, A. and Kak, A. Pca versus lda. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):228-233.
  22. Rong, H. J., Huang, G. B., and Ong, Y. S. (2008). Extreme learning machine for multi-categories classification applications. IEEE International Joint Conference on Neural Networks.
  23. Samaria, F. and Harter, A. (1994). Parameterisation of a stochastic model for human face identification. IEEE Workshop on Applications of Computer Vision.
  24. Sugiyama, M. (2007). Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. Journal of Machine Learning Research, 8:1027- 1061.
  25. Wang, Y., Cao, F., and Yuan, Y. (2011). A study on effectiveness of extreme learning machine. Neurocomputing, 74(16):2483-2490.
  26. Yan, S., Xu, D., Zhang, B., Zhang, H., Yang, Q., and Lin, S. (2007). Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Transactions on on Pattern Analysis ans Machine Intelligence, 29(1):40-50.
  27. Yin, L., Wei, X., Sun, Y., Wang, J., and Rosato, M. (2006). A 3d facial expression database for facial behavior research. IEEE International Conference on Automatic Face and Gesture Recognition.
  28. Zong, W. and Huang, G. B. (2011). Face recognition based on extreme learning machine. Neurocomputing, 74(16):2541-2551.
Download


Paper Citation


in Harvard Style

Iosifidis A., Tefas A. and Pitas I. (2014). Exploiting Local Class Information in Extreme Learning Machine . In Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014) ISBN 978-989-758-054-3, pages 49-55. DOI: 10.5220/0005038500490055


in Bibtex Style

@conference{ncta14,
author={Alexandros Iosifidis and Anastasios Tefas and Ioannis Pitas},
title={Exploiting Local Class Information in Extreme Learning Machine},
booktitle={Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)},
year={2014},
pages={49-55},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005038500490055},
isbn={978-989-758-054-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)
TI - Exploiting Local Class Information in Extreme Learning Machine
SN - 978-989-758-054-3
AU - Iosifidis A.
AU - Tefas A.
AU - Pitas I.
PY - 2014
SP - 49
EP - 55
DO - 10.5220/0005038500490055