Reinforcement Learning and Optimal Control: A Hybrid Collision Avoidance Approach

Simon Gottschalk, Matthias Gerdts, Mattia Piccinini

2024

Abstract

In this manuscript, we consider obstacle avoidance tasks in trajectory planning and control. The challenges of these tasks lie in the nonconvex pure state constraints that make optimal control problems (OCPs) difficult to solve. Reinforcement Learning (RL) provides a simpler approach to dealing with obstacle constraints, because a feedback function only needs to be established. Nevertheless, it turns out that often we get a long lasting training phase and we need a large amount of data to obtain appropriate solutions. One reason is that RL, in general, does not take into account a model of the underlying dynamics. Instead, this technique relies solely on information from the data. To address these drawbacks, we establish a hybrid and hierarchical method in this manuscript. While classical optimal control techniques handle system dynamics, RL focuses on collision avoidance. The final trained controller is able to control the dynamical system in real time. Even if the complexity of a dynamical system is too high for fast computations or if the training phase needs to be accelerated, we show a remedy by introducing a surrogate model. Finally, the overall approach is applied to steer a car on a racing track performing dynamic overtaking maneuvers with other moving cars.

Paper Citation

in Harvard Style

Gottschalk S., Gerdts M. and Piccinini M. (2024). Reinforcement Learning and Optimal Control: A Hybrid Collision Avoidance Approach. In Proceedings of the 10th International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS; ISBN 978-989-758-703-0, SciTePress, pages 76-87. DOI: 10.5220/0012569800003702

in Bibtex Style

@conference{vehits24,
author={Simon Gottschalk and Matthias Gerdts and Mattia Piccinini},
title={Reinforcement Learning and Optimal Control: A Hybrid Collision Avoidance Approach},
booktitle={Proceedings of the 10th International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS},
year={2024},
pages={76-87},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012569800003702},
isbn={978-989-758-703-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS
TI - Reinforcement Learning and Optimal Control: A Hybrid Collision Avoidance Approach
SN - 978-989-758-703-0
AU - Gottschalk S.
AU - Gerdts M.
AU - Piccinini M.
PY - 2024
SP - 76
EP - 87
DO - 10.5220/0012569800003702
PB - SciTePress