Acceleration of Backpropagation Training with Selective Momentum Term

Diego Carvalho, Areolino Neto

2019

Abstract

In many cases it is very hard to get an Artificial Neural Network (ANN) suitable for learning the solution, i.e., it cannot acquire the desired knowledge or needs an enormous number of training iterations. In order to improve the learning of ANN type Multi-Layer Perceptron (MLP), this work describes a new methodology for selecting weights, which will have the momentum term added to variation calculus of their values during each training iteration via Backpropagation (BP) algorithm. For that, the Pearson or Spearman correlation coefficients are used. Even very popular, the usage of BP algorithm has some drawbacks, among them the high convergence time is highlighted. A well-known technique used to reduce this disadvantage is the momentum term, which tries to accelerate the ANN learning keeping its stability, but when it is applied in all weights, as commonly used, with inadequate parameters, the result can be easily a failure in the training or at least an insignificant reduction of the ANN training time. The use of the Selective Momentum Term (SMT) can reduce the training time and, therefore, be also used for improving the training of deep neural networks.

Paper Citation

in Harvard Style

Carvalho D. and Neto A. (2019). Acceleration of Backpropagation Training with Selective Momentum Term.In Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-350-6, pages 443-450. DOI: 10.5220/0007272004430450

in Bibtex Style

@conference{icaart19,
author={Diego Carvalho and Areolino Neto},
title={Acceleration of Backpropagation Training with Selective Momentum Term},
booktitle={Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2019},
pages={443-450},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007272004430450},
isbn={978-989-758-350-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Acceleration of Backpropagation Training with Selective Momentum Term
SN - 978-989-758-350-6
AU - Carvalho D.
AU - Neto A.
PY - 2019
SP - 443
EP - 450
DO - 10.5220/0007272004430450