Extreme Learning Machine with Enhanced Variation of Activation Functions

Jacek Kabzinski

2016

Abstract

The main aim of this paper is to stress the fact that the sufficient variability of activation functions (AF) is important for an Extreme Learning Machine (ELM) approximation accuracy and applicability. A slight modification of the standard ELM procedure is proposed, which allows increasing the variance of each AF, without losing too much from the simplicity of random selection of parameters. The proposed modification does not increase the computational complexity of an ELM training significantly. Enhancing the variation of AFs results in reduced output weights norm, better numerical conditioning of the output weights calculation, smaller errors for the same number of the hidden neurons. The proposed approach works efficiently together with the Tikhonov regularization of ELM.

References

  1. Akusok, A., Bjork, K. M., Miche, Y., and Lendasse, A. (2015). High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications. IEEE Access, 3, 1011-1025. http://doi.org/10.1109/ ACCESS.2015.2450498
  2. Chen, Z. X., Zhu, H. Y., and Wang, Y. G. (2013). A modified extreme learning machine with sigmoidal activation functions. Neural Computing and Applications, 22(3-4), 541-550. http://doi.org/10.1007/ s00521-012-0860-2
  3. Feng, G., Lan, Y., Zhang, X., and Qian, Z. (2015). Dynamic adjustment of hidden node parameters for extreme learning machine. IEEE Transactions on Cybernetics, 45(2), 279-288. http://doi.org/10.1109/TCYB.2014.23 25594
  4. Guang-Bin Huang, and Chee-Kheong Siew. (2004). Extreme learning machine: RBF network case. In ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004. (Vol. 2, pp. 1029-1036). IEEE. http://doi.org/10.1109/ICARCV.2004.1468985
  5. Huang, G., Huang, G. Bin, Song, S., and You, K. (2015). Trends in extreme learning machines: A review. Neural Networks, 61, 32-48. http://doi.org/10.1016/j.neunet. 2014.10.001
  6. Kabzinski, J. (2015). Is Extreme Learning Machine Effective for Multisource Friction Modeling? (in Artificial Intelligence Applications and Innovations, Springer pp. 318-333). http://doi.org/10.1007/978-3- 319-23868-5_23
  7. Lin, S., Liu, X., Fang, J., and Xu, Z. (2015). Is extreme learning machine feasible? A theoretical assessment (Part II). IEEE Transactions on Neural Networks and Learning Systems, 26(1), 21-34. http://doi.org/10. 1109/TNNLS.2014.2336665
  8. Liu, X., Lin, S., Fang, J., and Xu, Z. (2015). Is extreme learning machine feasible? A theoretical assessment (Part I). IEEE Transactions on Neural Networks and Learning Systems, 26(1), 7-20. http://doi.org/10.1109/ TNNLS.2014.2335212
  9. Miche, Y., Sorjamaa, A., Bas, P., Simula, O., Jutten, C., and Lendasse, A. (2010). OP-ELM: Optimally pruned extreme learning machine. IEEE Transactions on Neural Networks, 21(1), 158-162. http://doi.org/10. 1109/TNN.2009.2036259
  10. Miche, Y., van Heeswijk, M., Bas, P., Simula, O., and Lendasse, A. (2011). TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing, 74(16), 2413-2421. http://doi.org/ 10.1016/j.neucom.2010.12.042
  11. Parviainen, E., and Riihimäki, J. (2013). A connection between extreme learning machine and neural network kernel. In Communications in Computer and Information Science (Vol. 272 CCIS, pp. 122-135).
Download


Paper Citation


in Harvard Style

Kabzinski J. (2016). Extreme Learning Machine with Enhanced Variation of Activation Functions . In Proceedings of the 8th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (IJCCI 2016) ISBN 978-989-758-201-1, pages 77-82. DOI: 10.5220/0006066200770082


in Bibtex Style

@conference{ncta16,
author={Jacek Kabzinski},
title={Extreme Learning Machine with Enhanced Variation of Activation Functions},
booktitle={Proceedings of the 8th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (IJCCI 2016)},
year={2016},
pages={77-82},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006066200770082},
isbn={978-989-758-201-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (IJCCI 2016)
TI - Extreme Learning Machine with Enhanced Variation of Activation Functions
SN - 978-989-758-201-1
AU - Kabzinski J.
PY - 2016
SP - 77
EP - 82
DO - 10.5220/0006066200770082