Bayesian Regularized Committee of Extreme Learning Machine

José M. Martínez-Martínez, Pablo Escandell-Montero, Emilio Soria-Olivas, Joan Vila-Francés, Rafael Magdalena-Benedito

2013

Abstract

Extreme Learning Machine (ELM) is an efficient learning algorithm for Single-Hidden Layer Feedforward Networks (SLFNs). Its main advantage is its computational speed due to a random initialization of the parameters of the hidden layer, and the subsequent use of Moore-Penrose’s generalized inverse in order to compute the weights of the output layer. The main inconvenient of this technique is that as some parameters are randomly assigned and remain unchanged during the training process, they can be non-optimum and the network performance may be degraded. This paper aims to reduce this problem using ELM committees. The way to combine them is to use a Bayesian linear regression due to its advantages over other approaches. Simulations on different data sets have demonstrated that this algorithm generally outperforms the original ELM algorithm.

References

  1. Alpaydin, E. (2010). Introduction to Machine Learning. MIT Press, 2nd edition.
  2. Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
  3. Berger., J. O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer.
  4. Bishop, C. M. (2007). Pattern Recognition and Machine Learning. Springer, 1st ed. 2006. corr. 2nd printing edition.
  5. Chen, T. and Martin, E. (2009). Bayesian linear regression and variable selection for spectroscopic calibration. Anal. Chim. Acta, 631(1):13-21.
  6. Congdon, P. (2006). Bayesian Statistical Modelling. Wiley.
  7. Deng, W., Zeng, Q., and Chen, L. (2009). Proc. ieee symp. comput. intell. data mining.
  8. Escandell-Montero, P., Martínez-Martínez, J. M., SoriaOlivas, E., Guimerá-Tomás, J., Martínez-Sober, M., and Serrano-L ópez, A. J. (2012). Regularized committee of extreme learning machine for regression problems. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2012. ESANN 7812.
  9. Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: Applications to nonorthogonal problems. Technometrics, 12(1):69-82.
  10. Huang, G., Zhu, Q.-Y., and Siew, C. (2006). Extreme learning machine: Theory and applications. Neurocomputing, 70:489-501.
  11. Malathi, V., Marimuthu, N., and Baskar, S. (2010). Intelligent approaches using support vector machine and extreme learning machine for transmission line protection. Neurocomputing, 73(10-12):2160 - 2167.
  12. Rao, C. R. and Mitra., S. K. (1972). Generalized Inverse of Matrices and Its Applications. Wiley.
  13. Seni, G. and Elder, J. (2010). Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions. Morgan and Claypool Publishers.
  14. Sun, Z.-L., Choi, T.-M., Au, K.-F., and Yu, Y. (2008). Sales forecasting using extreme learning machine with applications in fashion retailing. Decision Support Systems, 46(1):411 - 419.
  15. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (Series B), 58:267-288.
  16. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society B, 67:301-320.
Download


Paper Citation


in Harvard Style

M. Martínez-Martínez J., Escandell-Montero P., Soria-Olivas E., Vila-Francés J. and Magdalena-Benedito R. (2013). Bayesian Regularized Committee of Extreme Learning Machine . In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8565-41-9, pages 109-114. DOI: 10.5220/0004173901090114


in Bibtex Style

@conference{icpram13,
author={José M. Martínez-Martínez and Pablo Escandell-Montero and Emilio Soria-Olivas and Joan Vila-Francés and Rafael Magdalena-Benedito},
title={Bayesian Regularized Committee of Extreme Learning Machine},
booktitle={Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2013},
pages={109-114},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004173901090114},
isbn={978-989-8565-41-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Bayesian Regularized Committee of Extreme Learning Machine
SN - 978-989-8565-41-9
AU - M. Martínez-Martínez J.
AU - Escandell-Montero P.
AU - Soria-Olivas E.
AU - Vila-Francés J.
AU - Magdalena-Benedito R.
PY - 2013
SP - 109
EP - 114
DO - 10.5220/0004173901090114