Construction of Football Agents by Inverse Reinforcement Learning Using Relative Positional Information Among Players

Daiki Wakabayashi, Tomoaki Yamazaki, Kouzou Ohara

2025

Abstract

Recent advancements in reinforcement learning have made it possible to develop football agents that autonomously emulate the behavior of human players. However, it is still challenging for existing methods to successfully replicate realistic player behaviors. In fact, agents exhibit behaviors like clustering around the ball or shooting prematurely. One cause of this problem lies in reward functions that always assign large rewards to certain actions, such as scoring a goal, regardless of the situation, which bias agents towards high-reward actions. In this study, we incorporate the relative positional reward and the positional weight for shooting into the reward function used for reinforcement learning. The relative positional reward, derived from the positions of players, the ball, and the goal, is estimated using inverse reinforcement learning on a dataset of real football games. The positional weight for shooting is similarly based on actual shooting positions observed in these games. Through experiments on a dataset derived from real football games, we demonstrate that the relative positional reward helps align the agents’ behaviors more closely with those of human players.

Paper Citation

in Harvard Style

Wakabayashi D., Yamazaki T. and Ohara K. (2025). Construction of Football Agents by Inverse Reinforcement Learning Using Relative Positional Information Among Players. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 208-217. DOI: 10.5220/0013322900003890

in Bibtex Style

@conference{icaart25,
author={Daiki Wakabayashi and Tomoaki Yamazaki and Kouzou Ohara},
title={Construction of Football Agents by Inverse Reinforcement Learning Using Relative Positional Information Among Players},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART},
year={2025},
pages={208-217},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013322900003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART
TI - Construction of Football Agents by Inverse Reinforcement Learning Using Relative Positional Information Among Players
SN - 978-989-758-737-5
AU - Wakabayashi D.
AU - Yamazaki T.
AU - Ohara K.
PY - 2025
SP - 208
EP - 217
DO - 10.5220/0013322900003890
PB - SciTePress