![](bg1.png)
ITERATIVE LINEAR QUADRATIC REGULATOR DESIGN
FOR NONLINEAR BIOLOGICAL MOVEMENT SYSTEMS
Weiwei Li
Department of Mechanical and Aerospace Engineering, University of California San Diego
9500 Gilman Dr, La Jolla, CA 92093-0411
Emanuel Todorov
Department of Cognitive Science, University of California San Diego
9500 Gilman Dr, La Jolla, CA 92093-0515
Keywords:
ILQR, Optimal control, Nonlinear system.
Abstract:
This paper presents an Iterative Linear Quadratic Regulator (ILQR) method for locally-optimal feedback con-
trol of nonlinear dynamical systems. The method is applied to a musculo-skeletal arm model with 10 state
dimensions and 6 controls, and is used to compute energy-optimal reaching movements. Numerical compar-
isons with three existing methods demonstrate that the new method converges substantially faster and finds
slightly better solutions.
1 INTRODUCTION
Optimal control theory has received a great deal of
attention since the late 1950s, and has found appli-
cations in many fields of science and engineering. It
has also provided the most fruitful general framework
for constructing models of biological movement (Y
et al., 1989; Harris and Wolpert, 1998; Lahdhiri and
Elmaraghy, 1999; Todorov and Jordan, 2002). In the
field of motor control, optimality principles not only
yield accurate descriptions of observed phenomena,
but are well justified a priori. This is because the sen-
sorimotor system is the product of optimization pro-
cesses (i.e. evolution, development, learning, adapta-
tion) which continuously improve behavioral perfor-
mance.
Producing even the simplest movement involves an
enormous amount of information processing. When
we move our hand to a target, there are infinitely many
possible paths and velocity profiles that the multi-
joint arm could take, and furthermore each trajec-
tory can be generated by an infinite variety of muscle
activation patterns (since we have many more mus-
cles than joints). Biomechanical redundancy makes
the motor system very flexible, but at the same time
requires a very well designed controller that can
choose intelligently among the many possible alter-
natives. Optimal control theory provides a principled
approach to this problem – it postulates that the move-
ment patterns being chosen are the ones optimal for
the task at hand.
The majority of existing optimality models in mo-
tor control have been formulated in open-loop. How-
ever, the most remarkable property of biological
movements (in comparison with synthetic ones) is
that they can accomplish complex high-level goals in
the presence of large internal fluctuations, noise, de-
lays, and unpredictable changes in the environment.
This is only possible through an elaborate feedback
control scheme. Indeed, focus has recently shifted
towards stochastic optimal feedback control models.
This approach has already clarified a number of long-
standing issues related to the control of redundant
biomechanical systems (Todorov and Jordan, 2002).
In their present form, however, such models have a
serious limitation – they rely on the Linear-Quadratic-
Gaussian formalism, while in reality biomechanical
systems are strongly non-linear. The goal of the
present paper is to develop a new method, and com-
pare its performance to existing methods (Todorov
and Li, 2003) on a challenging biomechanical control
problem. The new method uses iterative linearization
of the nonlinear system around a nominal trajectory,
and computes a locally optimal feedback control law
via a modified LQR technique. This control law is
then applied to the linearized system, and the result is
used to improve the nominal trajectory incrementally.
It has convergence of quasi-Newton method.
The paper is organized as follows. The new ILQR
method is derived in Section 2. In section 3 we
present a realistic biomechanical model of the human
arm moving in the horizontal plane, as well as two
222
Li W. and Todorov E. (2004).
ITERATIVE LINEAR QUADRATIC REGULATOR DESIGN FOR NONLINEAR BIOLOGICAL MOVEMENT SYSTEMS.
In Proceedings of the First International Conference on Informatics in Control, Automation and Robotics, pages 222-229
DOI: 10.5220/0001143902220229
Copyright
c
SciTePress