A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning

Seiya Satoh, Ryohei Nakano

2015

Abstract

In the search space of a complex-valued multilayer perceptron having J hidden units, C-MLP(J), there are singular regions, where the gradient is zero. Although singular regions cause serious stagnation of learning, there exist narrow descending paths from the regions. Based on this observation, a completely new learning method called C-SSF (complex singularity stairs following) 1.0 was proposed, which utilizes singular regions to generate starting points of C-MLP(J) search. Although C-SSF1.0 finds excellent solutions of successive C-MLPs, it takes long CPU time because the number of searches increases as J gets larger. To deal with this problem, C-SSF1.1 was proposed, a few times faster by the introduction of search pruning, but it still remained unsatisfactory. In this paper we propose a yet faster C-SSF1.3, going further with search pruning, and then evaluate the method in terms of solution quality and processing time.

References

  1. S. Amari: Natural gradient works efficiently in learning, Neural Comput., 10(2), 251/276 (1998)
  2. M.F. Amin and et al.: Wirtinger calculus based gradient descent and Levenberg-Marquardt learning algorithms in complex-valued neural networks, Proc. ICONIP, 550/559 (2011)
  3. K. Fukumizu and S. Amari: Local minima and plateaus in hierarchical structure of multilayer perceptrons, Neural Networks, 13(3), 317/327 (2000)
  4. A. Hirose: Complex-Valued Neural Networks, 2nd ed., Springer-Verlag, Berlin Heidelberg (2012)
  5. M.S. Kim and C.C. Guest: Modification of backpropagation networks for complex-valued signal processing in frequency domain, Proc. IJCNN, 3, 27/31 (1990)
  6. H. Leung and S. Haykin: The complex backpropagation algorithm, IEEE Trans. Signal Process., 39(9), 2101/2104 (1991)
  7. D.G. Luenberger: Linear and nonlinear programming, Addison-Wesley Publishing Company, Reading, Massachusetts (1984)
  8. T. Nitta: Reducibility of the complex-valued neural network, Neural Information Processing - Letters and Reviews, 2(3), 53/56 (2004)
  9. T. Nitta: Local minima in hierarchical structures of complex-valued neural networks, Neural Networks, 43, 1/7 (2013)
  10. S. Satoh and R. Nakano: Eigen vector descent and line search for multilayer perceptron, Proc. IAENG Int. Conf. on AI & Applications (ICAIA'12), 1 1/6 (2012)
  11. S. Satoh and R. Nakano: Fast and stable learning utilizing singular regions of multilayer perceptron, Neural Processing Letters, 38(2), 99/115 (2013)
  12. S. Satoh, and R. Nakano: Complex-valued multilayer perceptron search utilizing singular regions of complexvalued parameter space, Proc. ICANN, 315/322 (2014)
  13. S. Satoh, and R. Nakano: Complex-valued multilayer perceptron learning using singular regions and search pruning, Proc. IJCNN (to be published) (2015)
  14. H.J. Sussmann: Uniqueness of the weights for minimal feedforward nets with a given input-output map, Neural Networks, 5(4), 589/593 (1992)
  15. S. Suzumura and R. Nakano: Complex-valued BFGS method for complex-valued neural networks, IEICE Trans. on Information & Systems, J96-D(3), 423/431 (in Japanese) (2013)
Download


Paper Citation


in Harvard Style

Satoh S. and Nakano R. (2015). A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning . In Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (ECTA 2015) ISBN 978-989-758-157-1, pages 122-129. DOI: 10.5220/0005594801220129


in Bibtex Style

@conference{ncta15,
author={Seiya Satoh and Ryohei Nakano},
title={A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning},
booktitle={Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (ECTA 2015)},
year={2015},
pages={122-129},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005594801220129},
isbn={978-989-758-157-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (ECTA 2015)
TI - A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning
SN - 978-989-758-157-1
AU - Satoh S.
AU - Nakano R.
PY - 2015
SP - 122
EP - 129
DO - 10.5220/0005594801220129