Upside-Down Reinforcement Learning for More Interpretable Optimal Control

Juan Cardenas-Cartagena, Juan Cardenas-Cartagena, Massimiliano Falzari, Massimiliano Falzari, Marco Zullich, Matthia Sabatelli

2025

Abstract

Model-Free Reinforcement Learning (RL) algorithms either learn how to map states to expected rewards or search for policies that can maximize a certain performance function. Model-Based algorithms instead, aim to learn an approximation of the underlying model of the RL environment and then use it in combination with planning algorithms. Upside-Down Reinforcement Learning (UDRL) is a novel learning paradigm that aims to learn how to predict actions from states and desired commands. This task is formulated as a Supervised Learning problem and has successfully been tackled by Neural Networks (NNs). In this paper, we investigate whether function approximation algorithms other than NNs can also be used within a UDRL framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs with the significant benefit of resulting in policies that are inherently more interpretable than NNs, therefore paving the way for more transparent, safe, and robust RL.

Paper Citation

in Harvard Style

Cardenas-Cartagena J., Falzari M., Zullich M. and Sabatelli M. (2025). Upside-Down Reinforcement Learning for More Interpretable Optimal Control. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: IAI; ISBN 978-989-758-737-5, SciTePress, pages 859-869. DOI: 10.5220/0013299400003890

in Bibtex Style

@conference{iai25,
author={Juan Cardenas-Cartagena and Massimiliano Falzari and Marco Zullich and Matthia Sabatelli},
title={Upside-Down Reinforcement Learning for More Interpretable Optimal Control},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: IAI},
year={2025},
pages={859-869},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013299400003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: IAI
TI - Upside-Down Reinforcement Learning for More Interpretable Optimal Control
SN - 978-989-758-737-5
AU - Cardenas-Cartagena J.
AU - Falzari M.
AU - Zullich M.
AU - Sabatelli M.
PY - 2025
SP - 859
EP - 869
DO - 10.5220/0013299400003890
PB - SciTePress