ONLINE SEQUENTIAL LEARNING BASED ON ENHANCED EXTREME LEARNING MACHINE USING LEFT OR RIGHT PSEUDO-INVERSE

Weiwei Zong, Yuan Lan, Guang-Bin Huang

2012

Abstract

The latest development (Huang et al., 2011) has shown that better generalization performance can be obtained for extreme learning machine (ELM) by adding a positive value to the diagonal of HTH or HHT , where H is the hidden layer output matrix. This paper further extends this enhanced ELM to online sequential learning mode. An online sequential learning algorithm is proposed for SLFNs and other regularization networks, consisting of two formulas for two kinds of scenarios: when initial training data is of small scale or large scale. Performance of proposed online sequential learning algorithm is demonstrated through six benchmarking data sets for both regression and multi-class classification problems.

References

  1. Asirvadam, V. S., McLoone, S. F., and Irwin, G. W. (2002). Parallel and separable recursive levenberg-marquardt training algorithm. In 12th IEEE Workshop on Neural Networks for Signal Processing, pages 129-138. IEEE.
  2. Blake and Merz, C. J. (1998). UCI repository of machine learning databases.
  3. Boyd, S., Ghaoui, L. E., Feron, E., and Balakrishnan, V. (1994). Linear Matrix Inequalities in System and Control Theory. Society for Industrial and Applied Mathematic.
  4. Deng, W., Zheng, Q., and Chen, L. (2009). Proximal support vector machine classifiers. In IEEE Symposium on Computational Intelligence and Data Mining (CIDM 09), pages 389-395.
  5. Feng, G., Huang, G.-B., Lin, Q., and Gay, R. (2009). Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Transactions on Neural Networks, 20(8):1352-1357.
  6. Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55-67.
  7. Huang, G.-B. and Chen, L. (2007). Convex incremental extreme learning machine. Neurocomputing, 70:3056- 3062.
  8. Huang, G.-B. and Chen, L. (2008). Enhanced random search based incremental extreme learning machine. Neurocomputing, 71:3460-3468.
  9. Huang, G.-B., Chen, L., and Siew, C.-K. (2006a). Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks, 17(4):879-892.
  10. Huang, G.-B., Ding, X., and Zhou, H. (2010). Optimization method based extreme learning machine for classification. Neurocomputing, 74:155-163.
  11. Huang, G.-B., Saratchandran, P., and Sundararajan, N. (2004). An efficient sequential learning algorithm for growing and pruning rbf (gap-rbf) networks. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 34.
  12. Huang, G.-B., Saratchandran, P., and Sundararajan, N. (2005). A generalized growing and pruning rbf (ggaprbf) neural network for function approximation. IEEE Transactions on Neural Networks, 16(1):57-67.
  13. Huang, G.-B., Zhou, H., Ding, X., and Zhang, R. (2011). Extreme learning machine for regression and multiclass classification. (in press) IEEE Transactions on Systems, Man, and Cybernetics.
  14. Huang, G.-B., Zhu, Q.-Y., and Siew, C.-K. (2006b). Extreme learning machine: Theory and applications. Neurocomputing, 70(1-3):489 - 501.
  15. Liang, N.-Y., Huang, G.-B., Saratchandran, P., and Sundararajan, N. (2006). A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Transactions on Neural Networks, 17(6):1411- 1423.
  16. Ngia, L. S., Sjöberg, J., and Viberg, M. (1998). Adaptive neural nets filter using a recursive levenbergmarquardt search direction. In the 32nd Asilomar Conference on Signals, Systems and Computers, CA, USA.
  17. Rao, C. R. and Mitra, S. K. (1971). Generalized Inverse of Matrices and its Applications. John Wiley & Sons, Inc, New York.
  18. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning representations by back-propagation errors. Nature, 323:533-536.
  19. Serre, D. (2002). Matrices: Theory and applications. Springer-Verlag New York, Inc.
  20. Toh, K.-A. (2008). Deterministic neural classification. Neural Computation, 20(6):1565-1595.
Download


Paper Citation


in Harvard Style

Zong W., Lan Y. and Huang G. (2012). ONLINE SEQUENTIAL LEARNING BASED ON ENHANCED EXTREME LEARNING MACHINE USING LEFT OR RIGHT PSEUDO-INVERSE . In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8425-98-0, pages 300-305. DOI: 10.5220/0003777603000305


in Bibtex Style

@conference{icpram12,
author={Weiwei Zong and Yuan Lan and Guang-Bin Huang},
title={ONLINE SEQUENTIAL LEARNING BASED ON ENHANCED EXTREME LEARNING MACHINE USING LEFT OR RIGHT PSEUDO-INVERSE},
booktitle={Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2012},
pages={300-305},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003777603000305},
isbn={978-989-8425-98-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - ONLINE SEQUENTIAL LEARNING BASED ON ENHANCED EXTREME LEARNING MACHINE USING LEFT OR RIGHT PSEUDO-INVERSE
SN - 978-989-8425-98-0
AU - Zong W.
AU - Lan Y.
AU - Huang G.
PY - 2012
SP - 300
EP - 305
DO - 10.5220/0003777603000305