![](bg8.png)
5 CONCLUSIONS
We demonstrated with five typical experiments from
the field of robot learning and early evolutionary
robotics, that a simple random search on a given net-
work topology is sufficient to find many suitable so-
lutions, as long as the network changes are started
and stopped by a reasonable feedback signal. In our
case, this feedback is realized with neuromodulators
that are triggered as a reaction to the sensed behav-
ior. Because of this, and the simplicity of the im-
plementation, the learning should also work directly
on physical robots without external supervision. The
benchmarks show that the feasibility of the method
strongly depends on the experiment complexity, not
so much on the chosen network substrate. Also, tem-
porary solutions appear and get relearned when the
behavior proves ineffective in some situations. These
aspects – already available in such a simple approach
– are highly desired in the field of robot learning to
allow adaptive, self-contained robots with life-long
learning capabilities. The method, however, is not
meant to be used as a competitive learning paradigm
for real robots. Instead, one intention of the bench-
mark is to provide a minimal testbed to evaluate new
learning paradigms for recurrent neural networks in
the sensori-motor loop. These paradigms should be
better in some aspects compared to such a simple ran-
dom search to justify their usually much higher com-
plexity. For this, the benchmarks are also publicly
available at the supplementary page.
Supplementary Material can be found at:
nerd.x-bot.org/neuromodulator-benchmarks
ACKNOWLEDGEMENTS
This work was partially funded by DFG-grant PA
480/7-1. We thank Josef Behr and Florian Ziegler for
testing and refining the simulation model and for their
contributions to the NERD toolkit.
REFERENCES
Ashby, W. R. (1960). Design for a brain: The origin of
adaptive behavior (2nd edition). Chapman and Hall,
London UK.
Buckley, C. L. (2008). A systemic analysis of the ideas im-
manent in neuromodulation. PhD thesis, University of
Southampton.
Doya, K. (2002). Metalearning and neuromodulation. Neu-
ral Networks, 15(4-6):495–506.
D
¨
urr, P., Mattiussi, C., Soltoggio, A., and Floreano, D.
(2008). Evolvability of neuromodulated learning for
robots. In Proc. of the 2008 ECSIS Symposium on
Learning and Adaptive Behavior in Robotic Systems,
pages 41–46.
Fellous, J. and Linster, C. (1998). Computational models
of neuromodulation. Neural Computation, 10(4):771–
805.
Floreano, D. and Urzelai, J. (2001). Neural morphogene-
sis, synaptic plasticity, and evolution. Theory in Bio-
sciences, 120(3):225–240.
Hoinville, T. and H
´
enaff, P. (2004). Comparative study of
two homeostatic mechanisms in evolved neural con-
trollers for legged locomotion. In Proc. of IEEE/RSJ
International Conference on Intelligent Robots and
Systems, IROS 2004, volume 3, pages 2624–2629.
Husbands, P. (1998). Evolving robot behaviours with
diffusing gas networks. In Proc. of Evolutionary
Robotics, EvoRob’98, volume 1468 of LNCS, pages
71–86. Springer.
Moioli, R., Vargas, P., and Husbands, P. (2009). A multiple
hormone approach to the homeostatic control of con-
flicting behaviours in an autonomous mobile robot. In
Proc. of IEEE Congress on Evolutionary Computa-
tion, CEC’09, pages 47–54.
O’Keefe, J. and Dostrovsky, J. (1971). The hippocampus as
a spatial map: Preliminary evidence from unit activity
in the freely-moving rat. Brain Research, 34(1):171–
175.
Pitonakova, L. (2012). Ultrastable neuroendocrine robot
controller. Adaptive Behavior, 21(1):47–63.
Rempis, C. W., Thomas, V., Bachmann, F., and Pasemann,
F. (2010). NERD - Neurodynamics and Evolutionary
Robotics Development Kit. In SIMPAR 2010, volume
6472 of LNAI, pages 121–132. Springer.
Rempis, C. W., Toutounji, H., and Pasemann, F. (2013).
Controlling the learning of behaviors in the sensori-
motor loop with neuromodulators in self-monitoring
neural networks. In Workshop on Autonomous Learn-
ing at the IEEE International Conference on Robotics
and Automation, ICRA 2013.
Soltoggio, A. and Stanley, K. (2012). From modulated
hebbian plasticity to simple behavior learning through
noise and weight saturation. Neural Networks, 34:28–
41.
Timmis, J., Neal, M., and Thorniley, J. (2009). An adap-
tive neuro-endocrine system for robotic systems. In
Proc. of the IEEE Workshop on Robotic Intelligence
in Informationally Structured Space, RIISS’09, pages
129–136.
IJCCI2013-InternationalJointConferenceonComputationalIntelligence
496