Action-Based Intrinsic Reward Design for Cooperative Behavior Acquisition in Multi-Agent Reinforcement Learning

Iori Takeuchi, Keiki Takadama, Keiki Takadama

2025

Abstract

In recent years, research has been conducted in multi-agent reinforcement learning that aims at efficient agent exploration in complex environments by using intrinsic rewards. However, such intrinsic rewards may inhibit the learning of behaviors necessary for acquiring cooperative behavior, and may not be able to solve the task of the environment. In this paper, we propose two types of internal reward designs to promote agents’ learning of cooperative behaviors in multi-agent reinforcement learning. One is to use the average of the values of the actions selected by all agents to promote the learning of actions necessary for cooperative behavior but difficult to increase in value. The other is to provide an individual intrinsic reward when the value of the action selected by each agent is lower than the average of the values of all the actions at the time, aiming to escape from the local solution. The results of the experiment with StarCraft II scenario 6h vs 8z showed that by adding the proposed intrinsic reward to the intrinsic reward that encourages agents to explore unexplored areas, cooperative behavior can be obtained in more cases than before.

Paper Citation

in Harvard Style

Takeuchi I. and Takadama K. (2025). Action-Based Intrinsic Reward Design for Cooperative Behavior Acquisition in Multi-Agent Reinforcement Learning. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 632-639. DOI: 10.5220/0013320300003890

in Bibtex Style

@conference{icaart25,
author={Iori Takeuchi and Keiki Takadama},
title={Action-Based Intrinsic Reward Design for Cooperative Behavior Acquisition in Multi-Agent Reinforcement Learning},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART},
year={2025},
pages={632-639},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013320300003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART
TI - Action-Based Intrinsic Reward Design for Cooperative Behavior Acquisition in Multi-Agent Reinforcement Learning
SN - 978-989-758-737-5
AU - Takeuchi I.
AU - Takadama K.
PY - 2025
SP - 632
EP - 639
DO - 10.5220/0013320300003890
PB - SciTePress