Hybrid Genetic Programming and Deep Reinforcement Learning for Low-Complexity Robot Arm Trajectory Planning

Quentin Vacher, Nicolas Beuve, Paul Allaire, Thibaut Marty, Mickaël Dardaillon, Karol Desnos

2024

Abstract

Robot arm control is a technological challenge where an algorithm needs to learn a deep understanding of spatial navigation. In particular, spatial navigation requires learning the relationship between the motor joint angular positions and the Cartesian coordinates of the robot. Trajectory planning is an even more complex challenge, where the algorithm must create a trajectory between two coordinates that does not cause a collision. State-of-the-art algorithms capable of solving trajectory planning are based on deep Reinforcement Learning (RL). These algorithms achieve high accuracy but suffer from high computational complexity. This paper proposes to use a genetic RL algorithm, the Tangled Program Graphs (TPGs), to solve trajectory planning. Using a genetic process, the TPGs generate a graph of programs with low inference complexity. On a first trajectory planning problem, the algorithm used achieves performance close to the state-of-the-art, but with a 100 less execution time and a 20× smaller model size. On a second and more difficult problem, the TPGs are not able to learn with efficiency. We propose a hybrid solution that mixes the TPGs and a state-of-the-art deep RL algorithm, the Soft Actor-Critic (SAC). This solution achieves better performance than the state-of-the-art for both problems, with 6 to 20 times faster execution times.

Paper Citation

in Harvard Style

Vacher Q., Beuve N., Allaire P., Marty T., Dardaillon M. and Desnos K. (2024). Hybrid Genetic Programming and Deep Reinforcement Learning for Low-Complexity Robot Arm Trajectory Planning. In Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: ECTA; ISBN 978-989-758-721-4, SciTePress, pages 139-150. DOI: 10.5220/0013012500003837

in Bibtex Style

@conference{ecta24,
author={Quentin Vacher and Nicolas Beuve and Paul Allaire and Thibaut Marty and Mickaël Dardaillon and Karol Desnos},
title={Hybrid Genetic Programming and Deep Reinforcement Learning for Low-Complexity Robot Arm Trajectory Planning},
booktitle={Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: ECTA},
year={2024},
pages={139-150},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013012500003837},
isbn={978-989-758-721-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: ECTA
TI - Hybrid Genetic Programming and Deep Reinforcement Learning for Low-Complexity Robot Arm Trajectory Planning
SN - 978-989-758-721-4
AU - Vacher Q.
AU - Beuve N.
AU - Allaire P.
AU - Marty T.
AU - Dardaillon M.
AU - Desnos K.
PY - 2024
SP - 139
EP - 150
DO - 10.5220/0013012500003837
PB - SciTePress