# A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning

### Seiya Satoh, Ryohei Nakano

#### Abstract

In the search space of a complex-valued multilayer perceptron having J hidden units, C-MLP(J), there are singular regions, where the gradient is zero. Although singular regions cause serious stagnation of learning, there exist narrow descending paths from the regions. Based on this observation, a completely new learning method called C-SSF (complex singularity stairs following) 1.0 was proposed, which utilizes singular regions to generate starting points of C-MLP(J) search. Although C-SSF1.0 finds excellent solutions of successive C-MLPs, it takes long CPU time because the number of searches increases as J gets larger. To deal with this problem, C-SSF1.1 was proposed, a few times faster by the introduction of search pruning, but it still remained unsatisfactory. In this paper we propose a yet faster C-SSF1.3, going further with search pruning, and then evaluate the method in terms of solution quality and processing time.

#### References

- S. Amari: Natural gradient works efficiently in learning, Neural Comput., 10(2), 251/276 (1998)
- M.F. Amin and et al.: Wirtinger calculus based gradient descent and Levenberg-Marquardt learning algorithms in complex-valued neural networks, Proc. ICONIP, 550/559 (2011)
- K. Fukumizu and S. Amari: Local minima and plateaus in hierarchical structure of multilayer perceptrons, Neural Networks, 13(3), 317/327 (2000)
- A. Hirose: Complex-Valued Neural Networks, 2nd ed., Springer-Verlag, Berlin Heidelberg (2012)
- M.S. Kim and C.C. Guest: Modification of backpropagation networks for complex-valued signal processing in frequency domain, Proc. IJCNN, 3, 27/31 (1990)
- H. Leung and S. Haykin: The complex backpropagation algorithm, IEEE Trans. Signal Process., 39(9), 2101/2104 (1991)
- D.G. Luenberger: Linear and nonlinear programming, Addison-Wesley Publishing Company, Reading, Massachusetts (1984)
- T. Nitta: Reducibility of the complex-valued neural network, Neural Information Processing - Letters and Reviews, 2(3), 53/56 (2004)
- T. Nitta: Local minima in hierarchical structures of complex-valued neural networks, Neural Networks, 43, 1/7 (2013)
- S. Satoh and R. Nakano: Eigen vector descent and line search for multilayer perceptron, Proc. IAENG Int. Conf. on AI & Applications (ICAIA'12), 1 1/6 (2012)
- S. Satoh and R. Nakano: Fast and stable learning utilizing singular regions of multilayer perceptron, Neural Processing Letters, 38(2), 99/115 (2013)
- S. Satoh, and R. Nakano: Complex-valued multilayer perceptron search utilizing singular regions of complexvalued parameter space, Proc. ICANN, 315/322 (2014)
- S. Satoh, and R. Nakano: Complex-valued multilayer perceptron learning using singular regions and search pruning, Proc. IJCNN (to be published) (2015)
- H.J. Sussmann: Uniqueness of the weights for minimal feedforward nets with a given input-output map, Neural Networks, 5(4), 589/593 (1992)
- S. Suzumura and R. Nakano: Complex-valued BFGS method for complex-valued neural networks, IEICE Trans. on Information & Systems, J96-D(3), 423/431 (in Japanese) (2013)

#### Paper Citation

#### in Harvard Style

Satoh S. and Nakano R. (2015). **A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning** . In *Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (ECTA 2015)* ISBN 978-989-758-157-1, pages 122-129. DOI: 10.5220/0005594801220129

#### in Bibtex Style

@conference{ncta15,

author={Seiya Satoh and Ryohei Nakano},

title={A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning},

booktitle={Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (ECTA 2015)},

year={2015},

pages={122-129},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0005594801220129},

isbn={978-989-758-157-1},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 7th International Joint Conference on Computational Intelligence - Volume 3: NCTA, (ECTA 2015)

TI - A Yet Faster Version of Complex-valued Multilayer Perceptron Learning using Singular Regions and Search Pruning

SN - 978-989-758-157-1

AU - Satoh S.

AU - Nakano R.

PY - 2015

SP - 122

EP - 129

DO - 10.5220/0005594801220129