Plateau in a Polar Variable Complex-valued Neuron

Tohru Nitta

2014

Abstract

In this paper, the characteristics of the complex-valued neuron model with parameters represented by polar coordinates (called polar variable complex-valued neuron) are investigated. The main results are as reported below. The polar variable complex-valued neuron is unidentifiable: there exists a parameter that does not affect the output value of the neuron and one cannot identify its value. The plateau phenomenon can occur during learning of the polar variable complex-valued neuron: the learning error does not decrease in a period. Furthermore, it is suggested by computer simulations that a single polar variable complex-valued neuron has the following characteristics: (a) Unidentifiable parameters (singular points) degrade the learning speed. (b) A plateau can occur during learning. When the weight is attracted to the singular point, the learning tends to be stuck.

References

  1. Amari, S., Park, H., and Ozeki, T. (2006). Singularities affect dynamics of learning in neuromanifolds. Neural Computation, 18(5):1007-1065.
  2. Cousseau, F., Ozeki, T., and Amari, S. (2008). Dynamics of learning in multilayer perceptrons near singularities. IEEE Trans. Neural Networks, 19(8):1313-1328.
  3. Fukumizu, K. (1999). Generalization error of the maximum likelihood estimation of layered models. In Proc. 1999 Workshop on Information-Based Induction Sciences (IBIS'99), pages 221-226. (In Japanese).
  4. Hirose, A. (2006). Complex-valued neural networks, Series on Studies in Computational Intelligence, volume 32. Springer-Verlag.
  5. Hirose, A. (2013). Complex-valued neural networks: advances and applications. IEEE Press-Wiley.
  6. Hirose, A., Asano, Y., and Hamano, T. (2004). Modeutilizing developmental learning based on coherent neural networks. In Proc. Int. Conf. on Neural Information Processing (ICONIP2004), pages 116-121, Berlin. Springer.
  7. Hirose, A., Tabata, C., and Ishimaru, D. (2001). Coherent neural network architecture realizing a self-organizing activeness mechanism. In Baba, N., Jain, J. C., and Howlett, R. J., editors, Knowledge-Based Intelligent Information Engineering Systems & Allied Technologies, pages 576-580. IOS Press, Tokyo.
  8. Kawata, S. and Hirose, A. (2003). A coherent optical neural network that learns desirable phase values in frequency domain by using multiple optical-path differences. Opt. Lett., 28(24):2524-2526.
  9. Nitta, T. (2008). Complex-valued neural network and complex-valued back-propagation learning algorithm. In Hawkes, P. W., editor, Advances in Imaging and Electron Physics, volume 152, pages 153-221. Elsevier, Amsterdam, The Netherlands.
  10. Nitta, T. (2009). Complex-valued neural networks: utilizing high-dimensional parameters. Information Science Reference, Pennsylvania.
  11. Nitta, T. (2010). On the singularity of a single complexvalued neuron. IEICE Trans. Inf. & Syst. D, J93- D(8):1614-1621. (in Japanese).
  12. Nitta, T. (2013). Local minima in hierarchical structures of complex-valued neural networks. Neural Networks, 43:1-7.
  13. Wei, H., Zhang, J., Cousseau, F., Ozeki, T., and Amari, S. (2008). Dynamics of learning near singularities in layered networks. Neural Computation, 20:813-843.
Download


Paper Citation


in Harvard Style

Nitta T. (2014). Plateau in a Polar Variable Complex-valued Neuron . In Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-015-4, pages 526-531. DOI: 10.5220/0004887905260531


in Bibtex Style

@conference{icaart14,
author={Tohru Nitta},
title={Plateau in a Polar Variable Complex-valued Neuron},
booktitle={Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2014},
pages={526-531},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004887905260531},
isbn={978-989-758-015-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Plateau in a Polar Variable Complex-valued Neuron
SN - 978-989-758-015-4
AU - Nitta T.
PY - 2014
SP - 526
EP - 531
DO - 10.5220/0004887905260531