PIMAEX: Multi-Agent Exploration Through Peer Incentivization

Michael Kölle, Johannes Tochtermann, Julian Schönberger, Gerhard Stenzel, Philipp Altmann, Claudia Linnhoff-Popien

2025

Abstract

While exploration in single-agent reinforcement learning has been studied extensively in recent years, consid-erably less work has focused on its counterpart in multi-agent reinforcement learning. To address this issue, this work proposes a peer-incentivized reward function inspired by previous research on intrinsic curiosity and influence-based rewards. The PIMAEX reward, short for Peer-Incentivized Multi-Agent Exploration, aims to improve exploration in the multi-agent setting by encouraging agents to exert influence over each other to increase the likelihood of encountering novel states. We evaluate the PIMAEX reward in conjunction with PIMAEX-Communication, a multi-agent training algorithm that employs a communication channel for agents to influence one another. The evaluation is conducted in the Consume/Explore environment, a partially observable environment with deceptive rewards, specifically designed to challenge the exploration vs. exploitation dilemma and the credit-assignment problem. The results empirically demonstrate that agents using the PI-MAEX reward with PIMAEX-Communication outperform those that do not.

Paper Citation

in Harvard Style

Kölle M., Tochtermann J., Schönberger J., Stenzel G., Altmann P. and Linnhoff-Popien C. (2025). PIMAEX: Multi-Agent Exploration Through Peer Incentivization. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 572-579. DOI: 10.5220/0013260000003890

in Bibtex Style

@conference{icaart25,
author={Michael Kölle and Johannes Tochtermann and Julian Schönberger and Gerhard Stenzel and Philipp Altmann and Claudia Linnhoff-Popien},
title={PIMAEX: Multi-Agent Exploration Through Peer Incentivization},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART},
year={2025},
pages={572-579},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013260000003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART
TI - PIMAEX: Multi-Agent Exploration Through Peer Incentivization
SN - 978-989-758-737-5
AU - Kölle M.
AU - Tochtermann J.
AU - Schönberger J.
AU - Stenzel G.
AU - Altmann P.
AU - Linnhoff-Popien C.
PY - 2025
SP - 572
EP - 579
DO - 10.5220/0013260000003890
PB - SciTePress