Quantum Reinforcement Learning for Solving a Stochastic Frozen Lake Environment and the Impact of Quantum Architecture Choices

Theodora-Augustina Drăgan, Maureen Monnet, Christian Mendl, Christian Mendl, Jeanette Lorenz

2023

Abstract

Quantum reinforcement learning (QRL) models augment classical reinforcement learning schemes with quantum-enhanced kernels. Different proposals on how to construct such models empirically show a promising performance. In particular, these models might offer a reduced parameter count and shorter times to reach a solution than classical models. It is however presently unclear how these quantum-enhanced kernels as subroutines within a reinforcement learning pipeline need to be constructed to indeed result in an improved performance in comparison to classical models. In this work we exactly address this question. First, we propose a hybrid quantum-classical reinforcement learning model that solves a slippery stochastic frozen lake, an environment considerably more difficult than the deterministic frozen lake. Secondly, different quantum architectures are studied as options for this hybrid quantum-classical reinforcement learning model, all of them well-motivated by the literature. They all show very promising performances with respect to similar classical variants. We further characterize these choices by metrics that are relevant to benchmark the power of quantum circuits, such as the entanglement capability, the expressibility, and the information density of the circuits. However, we find that these typical metrics do not directly predict the performance of a QRL model.

Paper Citation

in Harvard Style

Drăgan T., Monnet M., Mendl C. and Lorenz J. (2023). Quantum Reinforcement Learning for Solving a Stochastic Frozen Lake Environment and the Impact of Quantum Architecture Choices. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-623-1, pages 199-210. DOI: 10.5220/0011673400003393

in Bibtex Style

@conference{icaart23,
author={Theodora-Augustina Drăgan and Maureen Monnet and Christian Mendl and Jeanette Lorenz},
title={Quantum Reinforcement Learning for Solving a Stochastic Frozen Lake Environment and the Impact of Quantum Architecture Choices},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2023},
pages={199-210},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011673400003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Quantum Reinforcement Learning for Solving a Stochastic Frozen Lake Environment and the Impact of Quantum Architecture Choices
SN - 978-989-758-623-1
AU - Drăgan T.
AU - Monnet M.
AU - Mendl C.
AU - Lorenz J.
PY - 2023
SP - 199
EP - 210
DO - 10.5220/0011673400003393