Towards Multi-agent Reinforcement Learning using Quantum Boltzmann Machines

Tobias Müller, Christoph Roch, Kyrill Schmid, Philipp Altmann

2022

Abstract

Reinforcement learning has driven impressive advances in machine learning. Simultaneously, quantum-enhanced machine learning algorithms using quantum annealing underlie heavy developments. Recently, a multi-agent reinforcement learning (MARL) architecture combining both paradigms has been proposed. This novel algorithm, which utilizes Quantum Boltzmann Machines (QBMs) for Q-value approximation has outperformed regular deep reinforcement learning in terms of time-steps needed to converge. However, this algorithm was restricted to single-agent and small 2x2 multi-agent grid domains. In this work, we propose an extension to the original concept in order to solve more challenging problems. Similar to classic DQNs, we add an experience replay buffer and use different networks for approximating the target and policy values. The experimental results show that learning becomes more stable and enables agents to find optimal policies in grid-domains with higher complexity. Additionally, we assess how parameter sharing influences the agents’ behavior in multi-agent domains. Quantum sampling proves to be a promising method for reinforcement learning tasks, but is currently limited by the Quantum Processing Unit (QPU) size and therefore by the size of the input and Boltzmann machine.

Paper Citation

in Harvard Style

Müller T., Roch C., Schmid K. and Altmann P. (2022). Towards Multi-agent Reinforcement Learning using Quantum Boltzmann Machines. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-547-0, pages 121-130. DOI: 10.5220/0010762100003116

in Bibtex Style

@conference{icaart22,
author={Tobias Müller and Christoph Roch and Kyrill Schmid and Philipp Altmann},
title={Towards Multi-agent Reinforcement Learning using Quantum Boltzmann Machines},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2022},
pages={121-130},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010762100003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Towards Multi-agent Reinforcement Learning using Quantum Boltzmann Machines
SN - 978-989-758-547-0
AU - Müller T.
AU - Roch C.
AU - Schmid K.
AU - Altmann P.
PY - 2022
SP - 121
EP - 130
DO - 10.5220/0010762100003116