DYNAMICAL INVARIANTS FOR CPG CONTROL IN

AUTONOMOUS ROBOTS

Fernando Herrero-Carr

on, Francisco de Borja Rodr

ıguez and Pablo Varona

Grupo de Neurociencia Computacional, Escuela Polit

ecnica Superior, Universidad Aut

onoma de Madrid

Calle Francisco Tom

as y Valiente 11, 28049, Madrid, Spain

Keywords:

Bio-inspired robotics, Central pattern generators.

Abstract:

Several studies have shown the usefulness of central pattern generator circuits to control autonomous rhythmic

motion in robots. The traditional approach is building CPGs from nonlinear oscillators, adjusting a connec-

tivity matrix and its weights to achieve the desired function. Compared to existing living CPGs, this approach

seems still somewhat limited in resources. Living CPGs have a large number of available mechanisms to ac-

complish their task. The main function of a CPG is ensuring that some constraints regarding rhythmic activity

are always kept, surmounting any disturbances from the external environment. We call this constraints the

“dynamical invariant” of a CPG. Understanding the underlying biological mechanisms would take the design

of robotic CPGs a step further. It would allow us to begin the design with a set of invariants to be preserved.

The presence of these invariants will guarantee that, in response to unexpected conditions, an effective motor

program will emerge that will perform the expected function, without the need of anticipating every possible

scenario. In this paper we discuss how some bio-inspired elements contribute to building up these invariants.

1 INTRODUCTION

Biology has been a source of inspiration for robot

constructors for some time now. In particular, many

works have built Central Pattern Generator circuits to

generate rhythmic motion in a robot (see (Ijspeert,

2008) for a review). Usually CPGs are regarded as

a black box dynamical system capable of generating

rhythmic patterns, in a way that is ﬂexible enough

to accommodate external modulation and/or perturba-

tions, but robust enough to stay close to an effective

pattern.

Living CPGs have developed to perform very spe-

ciﬁc tasks involving rhythmic activity: controlling

swimming, chewing, breathing, heart beating, walk-

ing, etc (Grillner, 2006). But, not every individual

of one same species presents the same physiological

structures: due to differences in size, weight, or even

history (illnesses, lesions, etc.) CPGs must be able to

adapt to different environments (Bucher et al., 2005).

Therefore, we can conclude that underlying CPG cir-

cuits there are mechanisms that allow for a great de-

gree of ﬂexibility while preserving its core function.

The core function of a CPG can be expressed as

a set of dynamical invariants that must be preserved.

Recent research (Reyes et al., 2008) has clearly shown

that the pyloric CPG of the lobster presents sub-

stantial differences from animal to animal, yet ro-

bustly preserves the relationship between bursting pe-

riod and phase lag between neurons. Even when one

single CPG is forced by replacing one synapse with

an artiﬁcially modiﬁed synapse, the CPG is shown to

have some mechanism by which this relationship is

preserved.

Traditional techniques to build CPG control in

robots try to solve motor control tasks by mimicking

known biological systems, in particular circuit topol-

ogy and the presence of oscillators. This approach

provides limited capability to deal with unforeseen

scenarios. The concept of the dynamical invariant al-

lows for a new design strategy in which the motor

control can be achieved in a way that is not speci-

ﬁed a priori, but emerges from the set of biologically

inspired principles. Having a general set of invari-

ant preserving mechanisms allows for more freedom

in the process of design, providing more ﬂexible and

general solutions to existing problems in robot con-

trol.

441

Herrero-Carrón F., de Borja Rodríguez F. and Varona P. (2010).

DYNAMICAL INVARIANTS FOR CPG CONTROL IN AUTONOMOUS ROBOTS.

In Proceedings of the 7th International Conference on Informatics in Control, Automation and Robotics, pages 441-445

DOI: 10.5220/0003004304410445

 SciTePress

2 INGREDIENTS FOR

DYNAMICAL INVARIANT

PRESERVING CPGS

In this section we discuss scientiﬁc results on the

mechanisms known to contribute to the essential ﬂex-

ibility and robustness of CPGs, which may be directly

related to the implementation of dynamical invariants.

2.1 Topologies

CPGs are known for their ﬂexibility: they generate a

robust rhythmic activity with a recognizable pattern,

but they can be modulated by external input. The ﬁ-

nal shape of the rhythm produced by one CPG will

depend on many factors: intrinsic properties of in-

dividual neurons and synapses, strength and sign of

the couplings and network topology, all contribute to

the function of the circuit. It is known (Huerta et al.,

2001; Stiesberg et al., 2007) that non-open topologies

maximize the quality of the rhythm produced in terms

of ﬂexibility and regularity. A non-open topology is

that in which every neuron receives at least one con-

nection from another CPG member, in contrast to an

“open” topology, where at least one neuron does not

receive synapses from any other CPG member

In the design of an artiﬁcial CPG for robot control,

it is necessary to take this into account. All neurons

within the circuit must receive feedback from the rest

of the circuit, so that knowledge about how the CPG

is performing is distributed to all units. Of course, not

all neurons will process this information in the same

way.

Different works, both theoretical and experimen-

tal, have studied the role of synapses in the synchro-

nization of neural oscillators. However, this study

cannot be decoupled from the properties of the units

being connected. Depending on the intrinsic charac-

teristics of neurons, synaptic activity will have dif-

ferent effects. In fact, different CPGs may adopt

different combinations of neural and synaptic prop-

erties to achieve the same goal (Prinz et al., 2004).

The result is that the CPG designer has a wider va-

riety of mechanisms to choose from. For instance,

it is widely accepted that in-phase synchronization

can be achieved through excitatory interaction, but

it can also be achieved by inhibitory interaction with

the appropriate conditions (Wang and Rinzel, 1992).

And conversely, anti-phase synchronization is usu-

ally considered to happen under inhibition (Wang and

Rinzel, 1992; Rowat and Selverston, 1997) but it can

be found to happen under excitatory connections as

well (Kopell and Somers, 1995).

Usually phase relationship between modules of a

CPG is an important dynamical invariant that must

be preserved. There exist some important theoretical

studies on the mechanisms for phase locking between

oscillators. A general study based on a phase model

can be found in (Kopell and Ermentrout, 2000). How-

ever, it poses strong restrictions upon the coupling

between oscillators. More realistic approaches that

take into account synaptic dynamics for predicting the

type of synchronization among bursting neurons can

be found in (Oprisan et al., 2004) and (Elson et al.,

2002).

When building complex artiﬁcial CPGs, it is dif-

ﬁcult to predict how the coupling of neurons within

a given topology will affect synchronization. Being

able to predict stable phase-locking regimes as in the

above mentioned works could help us design complex

CPGs, with rich dynamics and guarantee that the re-

sulting phase and frequency relationships will be the

desired ones.

2.2 Starting, Stopping and Maintenance

of Rhythm

Few studies in robotics are concerned with how CPGs

are started and stopped. There exist various mecha-

nisms underlying the starting, stopping and sustaining

of rhythm in living CPGs.

In cases like breathing or heart beating, the group

of neurons responsible for the maintenance of rhythm

cannot afford to stop. There are other cases, however,

in which a set of neurons needs to be activated and,

after some time of activity, deactivated.

In the ﬁrst case, special neurons called “pace-

maker” neurons are responsible for setting the pace

at which the rest of the group will oscillate (Selver-

ston et al., 2009). Thanks to biophysical sub cellular

mechanisms, they can produce oscillations in isola-

tion, without external input. Through synaptic con-

nections, the activity of pacemakers is “distributed”

to other members of the groups.

Pacemaker neurons can usually adapt their burst

frequency over a large range. By means of external

input and feedback from other neurons in the network,

the pacemaker group will be able to decide if the cir-

cuit is working properly and, if not, set the appropri-

ate frequency. Some CPGs show redundancy mecha-

nisms in order to maintain the correct rhythm in case

of failure. Several neuron models are capable of en-

dogenous oscillation as seen in living CPG neurons:

many variations of the bio physically detailed model

by Hodgkin and Huxley (Hodgkin and Huxley, 1952);

one classical model for bursting activity (Hindmarsh

and Rose, 1984) and other recent models with less

ICINCO 2010 - 7th International Conference on Informatics in Control, Automation and Robotics

442

computational requirements (Rulkov, 2002; Aguirre

et al., 2005).

In the second case, initiation and termination are

usually caused by external forces. A silent group

of neurons may be “recruited” by an external exci-

tatory force. Such is the case in (Arshavsky et al.,

1998), where neuron group IN 12 is recruited by neu-

ron group IN 8 and inhibited by neuron group IN 7.

This a very good example in which neurons that do

not generate a rhythm autonomously contribute to the

proper functioning of the CPG.

Interestingly, this CPG presents another important

feature. Depending on the intensity of the swimming,

an early group of neurons is recruited for weak swim-

ming and if more powerful strokes are needed, a neu-

ron group with higher activation threshold will also

be recruited. This is the delayed group of neurons.

Another mechanism found in some neurons is

known as “post-inhibitory rebound”. This feature is

widely observed, in particular again in (Arshavsky

et al., 1998). A neuron with this feature will remain

silent while inhibited. If inhibition is suddenly re-

leased, the neuron will respond with a spike or burst

of activity.

It is believed that the selection of motor programs

may be subject to an inhibitory mechanism (Grillner

et al., 2005). Each possible motor program is kept

under inhibition, until upper control centers decide to

activate it. At this point, inhibition is released. By

the mechanism of post-inhibitory rebound, a motor

program could immediately start upon lifting of inhi-

bition. Note that this is not equivalent to activation

by excitation. The difference is that in a normal state,

the circuit responsible of the motor program will not

spontaneously activate because of noise, since inhibi-

tion will keep it forcibly silent.

The mechanism for termination of activity will de-

pend on the goal of the CPG. If neurons in the CPG

are not capable of endogenous oscillation, they may

just go silent by themselves after activity has been

elicited by one of the mechanisms mentioned earlier.

If the activity needs to be sustained, then mutual ex-

citatory connections within a group of neurons will

keep them ﬁring. This is known as a “pool” of neu-

rons.

So the repertoire for how activity is elicited, main-

tained and stopped is really ample. However, each

mechanism has different subtleties, depending on the

underlying cellular characteristics, on whether inhibi-

tion or excitation should be preferred, etc.

2.3 Motor Command Coding

Individual CPG neurons display a mainly bursting ac-

tivity. Motor commands are encoded in some aspect

of the neuron’s activity, for instance in the frequency

of the spikes, or on the precise timing between them.

How information is extracted from this activity is not

trivial (Brezina et al., 2000), but simple mechanisms

can be used in robots.

The fact that rhythmic motor commands are en-

coded using bursting neurons has an advantage over

the classical view of CPGs in robotics: the mecha-

nisms that encode motor commands and those used

for synchronization are decoupled. In the traditional

view, one non-linear oscillator represents a whole

CPG. Then, one variable of the oscillator is used as

output to the controlled joint and to other oscillators

that need to be synchronized. Bursting neurons have

the ability to ﬂexibly adapt the timing between bursts

and still produce robustly reproducible bursting pat-

terns. With this decoupling, the system gains simul-

taneously in robustness and ﬂexibility. In addition,

some bursting neurons have the ability to encode in-

formation relative to their identity and their context

and send messages to neighbouring neurons, as we

discuss in the following section.

2.4 Neural Signatures

Recent experiments have shown that CPG individual

cells have neural signatures that consist of neuron spe-

ciﬁc spike timings in their bursting activity (Szucs

et al., 2003). Model simulations indicate that neural

signatures that identify each cell can play a functional

role in the activity of CPG circuits (Latorre et al.,

2006). These signatures coexist with the information

encoded in the slow wave rhythm of the CPG which

results in a neural signal with multiple simultaneous

codes. Readers of this signal (muscles and other neu-

rons) can take advantage of the multiple simultaneous

codes and process them one by one, or simultaneously

in order to perform different tasks. The sender and the

content of the signals can also be used separately to

discriminate the information received by a neuron by

distinctly processing the input as a function of these

codes. These mechanisms can contribute to build dy-

namical invariants through a self-organizing strategy

that includes non supervised learning as a function of

local discrimination. Artiﬁcial CPG networks built

with neurons that display neural signatures allow for

a new set of learning rules that include not only the

modiﬁcation of the connections, but also the parame-

ters that affect the local discrimination.

DYNAMICAL INVARIANTS FOR CPG CONTROL IN AUTONOMOUS ROBOTS

443

2.5 Homeostasis: Self-regulation

Mechanisms

CPG research has also shown the presence of

many homeostatic mechanisms to self-regulate and

to deal with unexpected circumstances at the cellu-

lar and neural network levels and in multiple time

scales (Marder and Goaillard, 2006). Dynamical in-

variants that work in short time scales can be built

with the above mentioned mechanisms that involve

neuron and synapse dynamics and speciﬁc topolo-

gies. However other types of invariants working in

longer time scales can use mechanisms of adapta-

tion and learning (including sub cellular plasticity)

to generate rhythms even in the absence of the in-

put that sustains this rhythm under normal circum-

stances (Thoby-Brisson and Simmers, 1998). Long

scale dynamical invariants arise from self-regulatory

mechanism that involve tightly regulated synaptic and

intrinsic properties. Models could implement this self

regulation through speciﬁc synaptic and sub cellular

learning paradigms.

Implementing already known sub cellular and net-

work learning mechanisms (Marder and Prinz, 2002),

CPGs may easily accommodate the degradation pro-

cess of a working robot. If a joint loses torque, if

pieces begin to wear off, a CPG implementing these

mechanisms could adjust its function to reﬂect the

natural evolution of the mechanical parts. Even if

some part suddenly stops working or is disconnected

from the main body of the robot, the CPG could still

keep its original function.

3 CONCLUSIONS

Central Pattern Generator neural circuits are well

known for their ability to generate and sustain rhyth-

mic activity. And they do so in a robust manner, tol-

erant to perturbations, but with ﬂexibility, being able

to adjust their working regime to the requirements of

the environment.

We have presented recent results that extend the

idea of a CPG. Results from which the robotics ﬁeld

will surely beneﬁt. To begin with, we have intro-

duced a new design idea, termed dynamical invari-

ant. The key aspect of designing an artiﬁcial CPG

should be specifying what restrictions must be kept

under any condition. So, the presence of one or sev-

eral dynamical invariants will guarantee the effective-

ness of motor control in unknown or changing envi-

ronments. Living CPGs use different strategies to im-

plement their invariants, we have reviewed some of

them in this paper.

First, we have reported on studies that conclude

that those topologies that maximize rhythm quality

are non-open topologies. That is, every neuron in the

circuit receives at least one input from the rest of the

circuit. Following this, we review those mechanisms

that are responsible for rhythm start, stop and suste-

nance. Few works known to the authors have con-

cerned themselves with the issue of initiation and ter-

mination of CPGs in robotics. We believe that the

works mentioned here will be of great inspiration to

robot designers. Then, we discuss the mechanism by

which living motor neurons are believed to code mo-

tor commands. Studying it, robotic CPGs can gain

greater ﬂexibility and robustness by decoupling syn-

chronization and coding mechanisms. Next, a re-

cently discovered property of bursting neurons is pre-

sented: neural signatures. This mechanism increases

the computing capabilities of a CPG. It allows neu-

rons to code context speciﬁc information so that re-

ceivers of different neural messages may discriminate

their input according to the identity of the sender, cir-

cumstances on which a message occurs, etc. We be-

lieve that this result will widen the view of CPGs from

simple pattern generators to information processing

capable circuits. And ﬁnally, we present reports on

the ability of long term adaptation of CPGs. Several

results indicate that CPGs can satisfy their dynamical

invariants despite differences from animal to animal,

or development of the animal from young to adult or

changes in function and/or size due to illnesses and

injuries. Understanding the mechanisms underlying

this ability will allow designers to construct CPGs that

will adapt over a range of “robot families” and to the

consequences of long time operation like loss of per-

formance and degradation of the parts.

To conclude, we believe that biology has still

many secrets to reveal. We have pointed out key

issues that robotics has not yet explored, but with

promising potential. We are beginning to imple-

ment some of these ideas in our own robots (Herrero-

Carr

on et al., 2010), and look forward to the next gen-

eration of CPGs.

ACKNOWLEDGEMENTS

Work supported by MICINN BFU2009-08473,

CAM S-SEM-0255-2006 and TIN 2007-65989. Fer-

nando Herrero-Carr

on with an FPU-UAM grant.

ICINCO 2010 - 7th International Conference on Informatics in Control, Automation and Robotics

444

REFERENCES

Aguirre, C., Campos, D., Pascual, P., and Serrano, E.

(2005). A model of spiking-bursting neuronal be-

havior using a piecewise linear two-dimensional map.

Computational Intelligence and Bioinspired Systems,

pages 130–135.

Arshavsky, Y. I., Deliagina, T. G., Orlovsky, G. N., Panchin,

Y. V., Popova, L. B., and Sadreyev, R. I. (1998). Anal-

ysis of the central pattern generator for swimming in

the mollusk clione. Annals of the New York Academy

of Sciences, 860(1):51–69.

Brezina, V., Orekhova, I. V., and Weiss, K. R. (2000). The

neuromuscular transform: The dynamic, nonlinear

link between motor neuron ﬁring patterns and mus-

cle contraction in rhythmic behaviors. J Neurophysiol,

83(1):207–231.

Bucher, D., Prinz, A. A., and Marder, E. (2005). Animal-

to-animal variability in motor pattern production in

adults and during growth. J. Neurosci., 25(7):1611–

1619.

Elson, R. C., Selverston, A. I., Abarbanel, H. D., and Ra-

binovich, M. I. (2002). Inhibitory synchronization of

bursting in biological neurons: Dependence on synap-

tic time constant. J Neurophysiol, 88(3):1166–1176.

Grillner, S. (2006). Biological pattern generation: The cel-

lular and computational logic of networks in motion.

Neuron, 52(5).

Grillner, S., Hellgren, J., M

enard, A., Saitoh, K., and Wik-

str

om, M. A. (2005). Mechanisms for selection of

basic motor programs–roles for the striatum and pal-

lidum. Trends Neurosci, 28(7):364–370.

Herrero-Carr

on, F., Rodr

ıguez, F. d. B., and Varona, P.

(2010). Bio-inspired design strategies for cpg control

in modular robotics. Submitted.

Hindmarsh, J. L. and Rose, R. M. (1984). A model of

neuronal bursting using three coupled ﬁrst order dif-

ferential equations. Proceedings Of The Royal So-

ciety Of London. Series B, Containing Papers Of a

Biological Character. Royal Society (Great Britain),

221(1222):87–102.

Hodgkin, A. L. and Huxley, A. F. (1952). A quantitative

description of membrane current and its application to

conduction and excitation in nerve. The Journal of

physiology, 117(4):500–544.

Huerta, R., Varona, P., Rabinovich, M. I., and Abarbanel,

H. D. (2001). Topology selection by chaotic neurons

of a pyloric central pattern generator. Biological Cy-

bernetics, 84(1):L1–L8.

Ijspeert, A. (2008). Central pattern generators for locomo-

tion control in animals and robots: A review. Neural

Networks, 21(4):642–653.

Kopell, N. and Ermentrout, G. (2000). Mechanisms of

phase-locking and frequency control in pairs of cou-

pled neural oscillators.

Kopell, N. and Somers, D. (1995). Anti-phase solutions in

relaxation oscillators coupled through excitatory inter-

actions. Journal of Mathematical Biology, 33(3):261–

280.

Latorre, R., Rodr

ıguez, F., and Varona, P. (2006). Neural

signatures: Multiple coding in spiking–bursting cells.

Biological Cybernetics, 95(2):169–183.

Marder, E. and Goaillard, J.-M. (2006). Variability, com-

pensation and homeostasis in neuron and network

function. Nature Reviews Neuroscience, 7(7):563–

574.

Marder, E. and Prinz, A. A. (2002). Modeling stabil-

ity in neuron and network function: the role of ac-

tivity in homeostasis. BioEssays : news and re-

views in molecular, cellular and developmental biol-

ogy., 24(12):1145–1154.

Oprisan, S. A., Prinz, A. A., and Canavier, C. C. (2004).

Phase resetting and phase locking in hybrid circuits

of one model and one biological neuron. Biophys J,

87(4):2283–2298.

Prinz, A. A., Bucher, D., and Marder, E. (2004). Simi-

lar network activity from disparate circuit parameters.

Nature Neuroscience, 7(12):1345–1352.

Reyes, M., Huerta, R., Rabinovich, M., and Selverston, A.

(2008). Artiﬁcial synaptic modiﬁcation reveals a dy-

namical invariant in the pyloric cpg. European Jour-

nal of Applied Physiology, 102(6):667–675.

Rowat, P. F. and Selverston, A. I. (1997). Oscillatory mech-

anisms in pairs of neurons connected with fast in-

hibitory synapses. Journal of Computational Neuro-

science, 4(2):103–127.

Rulkov, N. F. (2002). Modeling of spiking-bursting neural

behavior using two-dimensional map. Physical Re-

view E, 65(4):041922+.

Selverston, A. B., Sz

ucs, A., Huerta, R., Pinto, R. D., and

Reyes, M. (2009). Neural mechanisms underlying the

generation of the lobster gastric mill motor pattern.

Frontiers in Neural Circuits.

Stiesberg, G. R., Reyes, M. B., Varona, P., Pinto, R. D., and

Huerta, R. (2007). Connection topology selection in

central pattern generators by maximizing the gain of

information. Neural Computation, 19(4):974–993.

Szucs, A., Pinto, R. D., Rabinovich, M. I., Abarbanel,

H. D., and Selverston, A. I. (2003). Synaptic mod-

ulation of the interspike interval signatures of burst-

ing pyloric neurons. Journal of neurophysiology,

89(3):1363–1377.

Thoby-Brisson, M. and Simmers, J. (1998). Neuro-

modulatory inputs maintain expression of a lobster

motor pattern-generating network in a modulation-

dependent state: Evidence from long-term decentral-

ization in vitro. J. Neurosci., 18(6):2212–2225.

Wang, X.-J. and Rinzel, J. (1992). Alternating and syn-

chronous rhythms in reciprocally inhibitory model

neurons. Neural Comput., 4(1):84–97.

DYNAMICAL INVARIANTS FOR CPG CONTROL IN AUTONOMOUS ROBOTS

445