STOCHASTIC GPU-BASED MULTITHREAD IMPLEMENTATION OF MULTIPLE BACK-PROPAGATION

Noel Lopes, Bernardete Ribeiro

2010

Abstract

Graphics Processing Units (GPUs) have evolved into a highly parallel, multi-threaded, many-core processor with enormous computational power. The GPU is especially well suited to address pattern recognition problems that can be expressed as data-parallel computations. Thus it provides a viable alternative to the use of dedicated hardware in the neural network (NN) field, where the long training times have always been a major drawback. In this paper, we propose a GPU implementation of the online (stochastic) training mode of the Multiple Back-Propagation (MBP) algorithm and compare it with corresponding standalone CPU version and with the batch training mode GPU implementation. For a fair and unbiased comparison we run the experiments with benchmarks from machine learning and pattern recognition field and we show that the GPU performance excel the CPU results in particular for high complex problems.

References

  1. Almeida, L. B. (1997). Handbook of Neural Computation, chapter C1.2 Multilayer perceptrons, pages C1.2:1- C1.2:30. IOP Publishing Ltd and Oxford University Press.
  2. Catanzaro, B., Sundaram, N., and Keutzer, K. (2008). Fast support vector machine training and classification on graphics processors. In Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pages 104-111, Helsinki, Finland.
  3. Che, S., Boyer, M., Meng, J., Tarjan, D., Sheaffer, J. W., and Skadron, K. (2008). A performance study of general-purpose applications on graphics processors using CUDA. Journal of Parallel and Distributed Computing, 68(10):1370-1380.
  4. Fahlman, S. E. and Lebiere, C. (1990). The cascadecorrelation learning architecture. In Advances in Neural Information Processing Systems 2, pages 524-532. Morgan Kaufmann.
  5. Friedman, J. H. (1991). Multivariate adaptive regression splines. The Annals of Statistics, 19(1):1-67.
  6. Jang, H., Park, A., and Jung, K. (2008). Neural network implementation using cuda and openmp. In DICTA 7808: Proceedings of the 2008 Digital Image Computing: Techniques and Applications, pages 155-161, Washington, DC, USA. IEEE Computer Society.
  7. Lopes, N. and Ribeiro, B. (2001). Hybrid learning in a multi-neural network architecture. Neural Networks, 2001. Proceedings. IJCNN 7801. International Joint Conference on Neural Networks, 4:2788-2793.
  8. Lopes, N. and Ribeiro, B. (2003). An efficient gradientbased learning algorithm applied to neural networks with selective actuation neurons. Neural, Parallel & Scientific Computations, 11(3):253-272.
  9. Lopes, N. and Ribeiro, B. (2009). GPU implementation of the multiple back-propagation algorithm. In Proceedings of Intelligent Data Engineering and Automated Learning, volume 5788 of Lecture Notes in Computer Science, pages 449-456. Springer.
  10. Schaa, D. and Kaeli, D. (2009). Exploring the multipleGPU design space. In Parallel & Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on, pages 1-12.
  11. Steinkrau, D., Simard, P. Y., and Buck, I. (2005). Using GPUs for machine learning algorithms. In ICDAR 7805: Proceedings of the Eighth International Conference on Document Analysis and Recognition, pages 1115-1119, Washington, DC, USA. IEEE Computer Society.
Download


Paper Citation


in Harvard Style

Lopes N. and Ribeiro B. (2010). STOCHASTIC GPU-BASED MULTITHREAD IMPLEMENTATION OF MULTIPLE BACK-PROPAGATION . In Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-674-021-4, pages 271-276. DOI: 10.5220/0002722102710276


in Bibtex Style

@conference{icaart10,
author={Noel Lopes and Bernardete Ribeiro},
title={STOCHASTIC GPU-BASED MULTITHREAD IMPLEMENTATION OF MULTIPLE BACK-PROPAGATION},
booktitle={Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2010},
pages={271-276},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002722102710276},
isbn={978-989-674-021-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - STOCHASTIC GPU-BASED MULTITHREAD IMPLEMENTATION OF MULTIPLE BACK-PROPAGATION
SN - 978-989-674-021-4
AU - Lopes N.
AU - Ribeiro B.
PY - 2010
SP - 271
EP - 276
DO - 10.5220/0002722102710276