Parallel Batch Pattern Training of Recirculation Neural Network

Volodymyr Turchenko, Vladimir Golovko, Anatoly Sachenko

Abstract

The development of a parallel batch pattern back propagation training algorithm of a recirculation neural network is presented in this paper. The model of a recirculation neural network and usual sequential batch pattern algorithm of its training are theoretically described. An algorithmic description of the parallel version of the batch pattern training method is presented. The parallelization efficiency of the developed parallel algorithm is investigated on the example of data compression and principal component analysis. The results of the experimental researches show that the developed parallel algorithm provides high parallelization efficiency on a parallel symmetric multiprocessor computer system. It allows applying the developed parallel software for the facilitation of scientific research of neural network-based intrusion detection system for computer networks.

References

  1. Haykin, S., 2008. Neural networks and learning machines. Prentice Hall, 936 p.
  2. Mahapatra, S., Mahapatra, R., Chatterji B., 1997. A Parallel Formulation of Back-propagation Learning on Distributed Memory Multiprocessors. Parallel Computing. Vol. 22, No. 12, pp. 1661-1675.
  3. Hanzálek, Z., 1998. A Parallel Algorithm for Gradient Training of Feed-forward Neural Networks. Parallel Computer. Vol. 24, No. 5-6, pp. 823-839.
  4. Vin, T.K., Seng, P.Z., Kuan, M.N.P., Haron, F., 2005. A Framework for Grid-based Neural Networks. Proceedings of First International Conference on Distributed Frameworks for Multimedia Applications. pp. 246-250.
  5. Krammer, L., Schikuta, E., Wanek, H., 2006. A Gridbased Neural Network Execution Service Source. Proceedings of 24th IASTED International Conference on Parallel and Distributed Computations and Networking. pp. 35-40.
  6. De Llano, R.M., Bosque, J.L., 2010. Study of Neural Net Training Methods in Parallel and Distributed Architectures. Future Generation Computer Systems. Vol. 26, Issue 2, pp. 183-190.
  7. Cernansky M., 2009. Training Recurrent Neural Network Using Multistream Extended Kalman Filter on Multicore Processor and Cuda Enabled Graphic Processor Unit. Lecture Notes in Computer Science. Volume 5768, Part I, pp. 381-390.
  8. Lotric, U., Dobnikar, A., 2009. Parallel Implementations of Recurrent Neural Network Learning. M. Kolehmainen et al. (Eds.): ICANNGA 2009, LNCS 5495. Springer-Verlag, Berlin, Heidelberg, pp. 99- 108.
  9. Turchenko, V., Grandinetti, L., 2010. Parallel Batch Pattern BP Training Algorithm of Recurrent Neural Network. Proceedings of the 14th IEEE International Conference on Intelligent Engineering Systems. Las Palmas of Gran Canaria, Spain, pp. 25-30.
  10. Turchenko, V., Grandinetti, L., Sachenko, A., 2012. Parallel Batch Pattern Training of Neural Networks on Computational Clusters. Proceedings of the 2012 International Conference on High Performance Computing & Simulation HPCS 2012. July 2 - 6, Madrid, Spain, in press.
  11. Golovko, V., Galushkin A., 2001. Neural Networks: training, models and applications, Moscow, Radiotechnika (in Russian).
  12. Bryliuk, D., Starovoitov, V., 2001. Application of Recirculation Neural Network and Principal Component Analysis for Face Recognition. The 2nd International Conference on Neural Networks and Artificial Intelligence. Minsk, BSUIR, pp.136-142.
  13. Turchenko, V., Grandinetti, L., 2009. Efficiency Research of Batch and Single Pattern MLP Parallel Training Algorithms. Proceedings 5th IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems IDAACS2009. Rende, Italy, pp. 218-224.
  14. Golovko, V., Gladyschuk, V., 1999. Recirculation Neural Network Training for Image Processing. Advanced Computer Systems. pp. 73-78.
  15. Turchenko, V., Grandinetti, L., Bosilca, G., Dongarra, J., 2010. Improvement of parallelization efficiency of batch pattern BP training algorithm using Open MPI. Elsevier Procedia Computer Science 2010. Volume 1, Issue 1, pp. 525-533.
  16. Vaitsekhovich, L., Golovko, V., 2009. Intrusion Detection in TCP/IP Networks Using Immune Systems Paradigm and Neural Network Detectors. XI International PhD Workshop OWD 2009. pp. 219-224.
  17. Komar, M., Golovko, V., Sachenko, A., Bezobrazov S., 2011. Intelligent system for detection of networking intrusion. Proceedings of the 6th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems IDAACS-2011. Prague (Czech Republic), V1, pp. 374-377.
  18. KDD Cup Competition 1999. - Information on: http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.ht ml.
  19. MPICH2, 2011. http://www.mcs.anl.gov/research/projects /mpich2/
  20. PAGaLiNNeT, 2011. http://uweb.deis.unical.it/turchenko/ research-projects/pagalinnet/
Download


Paper Citation


in Harvard Style

Turchenko V., Golovko V. and Sachenko A. (2012). Parallel Batch Pattern Training of Recirculation Neural Network . In Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ANNIIP, (ICINCO 2012) ISBN 978-989-8565-21-1, pages 644-650. DOI: 10.5220/0004150206440650


in Bibtex Style

@conference{anniip12,
author={Volodymyr Turchenko and Vladimir Golovko and Anatoly Sachenko},
title={Parallel Batch Pattern Training of Recirculation Neural Network},
booktitle={Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ANNIIP, (ICINCO 2012)},
year={2012},
pages={644-650},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004150206440650},
isbn={978-989-8565-21-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ANNIIP, (ICINCO 2012)
TI - Parallel Batch Pattern Training of Recirculation Neural Network
SN - 978-989-8565-21-1
AU - Turchenko V.
AU - Golovko V.
AU - Sachenko A.
PY - 2012
SP - 644
EP - 650
DO - 10.5220/0004150206440650