Authors:
Simon Gottschalk
1
;
Matthias Gerdts
1
and
Mattia Piccinini
2
Affiliations:
1
Institute for Applied Mathematics and Scientific Computing, Department of Aerospace Engineering, Universität der Bundeswehr München, Werner-Heisenberg-Weg 39, D-85577 Neubiberg, Germany
;
2
Department of Industrial Engineering, University of Trento, via Calepina 14, 38123 Trento, Italy
Keyword(s):
Reinforcement Learning, Optimal Control, Collision Avoidance.
Abstract:
In this manuscript, we consider obstacle avoidance tasks in trajectory planning and control. The challenges of these tasks lie in the nonconvex pure state constraints that make optimal control problems (OCPs) difficult to solve. Reinforcement Learning (RL) provides a simpler approach to dealing with obstacle constraints, because a feedback function only needs to be established. Nevertheless, it turns out that often we get a long lasting training phase and we need a large amount of data to obtain appropriate solutions. One reason is that RL, in general, does not take into account a model of the underlying dynamics. Instead, this technique relies solely on information from the data. To address these drawbacks, we establish a hybrid and hierarchical method in this manuscript. While classical optimal control techniques handle system dynamics, RL focuses on collision avoidance. The final trained controller is able to control the dynamical system in real time. Even if the complexity of a d
ynamical system is too high for fast computations or if the training phase needs to be accelerated, we show a remedy by introducing a surrogate model. Finally, the overall approach is applied to steer a car on a racing track performing dynamic overtaking maneuvers with other moving cars.
(More)