Performance Shaping through Cost Cumulants and Neural
Networks-based Series Expansion
Bei Kang, Chukwuemeka Aduba and Chang-Hee Won
Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA 19122, U.S.A.
Keywords:
Statistical Optimal Control, Cumulant Minimization, Neural Networks, Cost Cumulants, Performance
Shaping.
Abstract:
The performance shaping method is addressed as a statistical optimal control problem. In statistical control,
we shape the distribution of the cost function by minimizing n-th order cost cumulants. The n-th cost cumu-
lant, Hamilton-Jacobi-Bellman (HJB) equation is derived as the necessary condition for the optimality. The
proposed method provides an approach to control a higher order cost cumulant for stochastic systems, and
generalizes the traditional linear-quadratic-Gaussian and Risk-Sensitive control methods. This allows the cost
performance shaping via the cost cumulants. Moreover, the solution of general n-th cost cumulant control is
provided by numerically solving the HJB equations using neural network method. The results of this paper
are demonstrated through a satellite attitude control example.
1 INTRODUCTION
We shape the performance of the system through the
statistical properties of the cost function. In linear-
quadratic-Gaussian (LQG) control, the performance
is optimized by minimizing the mean of the cost func-
tion (Fleming and Rishel, 1975). In statistical opti-
mal control, we minimize any cost cumulant to im-
prove the performance of the system. So far the first
mean (LQG) and denumerable sum of all the cumu-
lants (risk-sensitive) of the cost function are inves-
tigated (Lim and Zhou, 2001). However, there are
other statistical parameters that we can vary to shape
the performance. This is achieved by minimizing n-th
cost cumulants. The study of cost control cumulant
was initiated by (Sain, 1966). The authors extended
the theory of cost cumulant control to third and fourth
cumulants for a nonlinear system with nonquadratic
cost and derived the corresponding HJB equations
(Won et al., 2010). HJB equation was derive, but
the solution was not determined. In fact, most HJB
equations do not have analytical solutions except for
the special cases of linear systems with quadratic cost
functions. Thus, numerical approximate methods are
needed to solve HJB equation. For the first two cu-
mulant case, we solved the HJB equation using neu-
ral networks in (Kang and Won, 2010). In this paper,
we extend this result to n-th cumulants. This is not a
simple extension of the results in (Won et al., 2010).
There, we developed a procedure to solve higher or-
der cost cumulant problem using the results of the mo-
ments. In this paper, we use induction to derive n-th
cumulant HJB equation, which was not a trivial task.
This n-th cumulant HJB equation corresponds to the
performance shaping idea. Then we solve this HJB
equation using a neural network method.
A power series expansion to approximate the
value function for an infinite-time horizon determin-
istic system was given in (Alberkht, 1961). Apply-
ing Galerkin approximate method to solve the gen-
eralized Hamilton-Jacobi-Bellman (GHJB) was given
in (Beard et al., 1997). (Chen et al., 2007) proposed
using neural network methods to solve the optimal
control problem of a nonlinear finite time system. If
we define the weighting and the basis functions as
polynomials, the neural network is in fact equivalent
to power series expansion in that we use coefficients
of the power expansion as the weights in neural net-
works. However, neural network method has the po-
tential to be more than simple power series expansion
by adding additional layers of neurons. In this work,
we extend the system dynamics of (Chen et al., 2007)
to stochastic systems. Then we solve the HJB equa-
tions for the n-th cost cumulant control problem using
neural network approximation method.
Section 2 states the problem and defines the nota-
tions used in this paper. Section 3 develops the HJB
equations for the n-th cost cumulant control of the
196
Kang B., Aduba C. and Won C..
Performance Shaping through Cost Cumulants and Neural Networks-based Series Expansion.
DOI: 10.5220/0004032101960201
In Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2012), pages 196-201
ISBN: 978-989-8565-21-1
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)