Evaluating Neuromodulator-controlled Stochastic Plasticity for Learning Recurrent Neural Control Networks

Christian W. Rempis, Hazem Toutounji, Frank Pasemann

Abstract

Learning recurrent neural networks as behavior controllers for robots requires measures to guide the learning towards a desired behavior. Organisms in nature solve this problem with feedback signals to assess their behavior and to refine their actions. In line with this, a neural framework is developed where the synaptic learning is controlled by artificial neuromodulators that are produced in response to (undesired) sensory signals. To test this framework and to get a base line to evaluate further approaches, we perform five classical benchmark experiments with a simple random plasticity method. We show that even with this simple plasticity method, behaviors can already be found for all experiments, even for comparably large networks with over 90 plastic synapses. The performance depends strongly on the complexity of the task and less on the chosen network topology. This suggests that controlling learning with neuromodulators is a viable approach that is promising to work also with more sophisticated plasticity methods in the future.

References

  1. Ashby, W. R. (1960). Design for a brain: The origin of adaptive behavior (2nd edition). Chapman and Hall, London UK.
  2. Buckley, C. L. (2008). A systemic analysis of the ideas immanent in neuromodulation. PhD thesis, University of Southampton.
  3. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4-6):495-506.
  4. Dürr, P., Mattiussi, C., Soltoggio, A., and Floreano, D. (2008). Evolvability of neuromodulated learning for robots. In Proc. of the 2008 ECSIS Symposium on Learning and Adaptive Behavior in Robotic Systems, pages 41-46.
  5. Fellous, J. and Linster, C. (1998). Computational models of neuromodulation. Neural Computation, 10(4):771- 805.
  6. Floreano, D. and Urzelai, J. (2001). Neural morphogenesis, synaptic plasticity, and evolution. Theory in Biosciences, 120(3):225-240.
  7. Hoinville, T. and Hénaff, P. (2004). Comparative study of two homeostatic mechanisms in evolved neural controllers for legged locomotion. In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2004, volume 3, pages 2624-2629.
  8. Husbands, P. (1998). Evolving robot behaviours with diffusing gas networks. In Proc. of Evolutionary Robotics, EvoRob'98, volume 1468 of LNCS, pages 71-86. Springer.
  9. Moioli, R., Vargas, P., and Husbands, P. (2009). A multiple hormone approach to the homeostatic control of conflicting behaviours in an autonomous mobile robot. In Proc. of IEEE Congress on Evolutionary Computation, CEC'09, pages 47-54.
  10. O'Keefe, J. and Dostrovsky, J. (1971). The hippocampus as a spatial map: Preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34(1):171- 175.
  11. Pitonakova, L. (2012). Ultrastable neuroendocrine robot controller. Adaptive Behavior, 21(1):47-63.
  12. Rempis, C. W., Thomas, V., Bachmann, F., and Pasemann, F. (2010). NERD - Neurodynamics and Evolutionary Robotics Development Kit. In SIMPAR 2010, volume 6472 of LNAI, pages 121-132. Springer.
  13. Rempis, C. W., Toutounji, H., and Pasemann, F. (2013). Controlling the learning of behaviors in the sensorimotor loop with neuromodulators in self-monitoring neural networks. In Workshop on Autonomous Learning at the IEEE International Conference on Robotics and Automation, ICRA 2013.
  14. Soltoggio, A. and Stanley, K. (2012). From modulated hebbian plasticity to simple behavior learning through noise and weight saturation. Neural Networks, 34:28- 41.
  15. Timmis, J., Neal, M., and Thorniley, J. (2009). An adaptive neuro-endocrine system for robotic systems. In Proc. of the IEEE Workshop on Robotic Intelligence in Informationally Structured Space, RIISS'09, pages 129-136.
Download


Paper Citation


in Harvard Style

W. Rempis C., Toutounji H. and Pasemann F. (2013). Evaluating Neuromodulator-controlled Stochastic Plasticity for Learning Recurrent Neural Control Networks . In Proceedings of the 5th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2013) ISBN 978-989-8565-77-8, pages 489-496. DOI: 10.5220/0004554504890496


in Bibtex Style

@conference{ncta13,
author={Christian W. Rempis and Hazem Toutounji and Frank Pasemann},
title={Evaluating Neuromodulator-controlled Stochastic Plasticity for Learning Recurrent Neural Control Networks},
booktitle={Proceedings of the 5th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2013)},
year={2013},
pages={489-496},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004554504890496},
isbn={978-989-8565-77-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2013)
TI - Evaluating Neuromodulator-controlled Stochastic Plasticity for Learning Recurrent Neural Control Networks
SN - 978-989-8565-77-8
AU - W. Rempis C.
AU - Toutounji H.
AU - Pasemann F.
PY - 2013
SP - 489
EP - 496
DO - 10.5220/0004554504890496