Opponent Modelling in the Game of Tron using Reinforcement Learning

Stefan J. L. Knegt, Madalina M. Drugan, Marco A. Wiering

2018

Abstract

In this paper we propose the use of vision grids as state representation to learn to play the game Tron using neural networks and reinforcement learning. This approach speeds up learning by significantly reducing the number of unique states. Furthermore, we introduce a novel opponent modelling technique, which is used to predict the opponent’s next move. The learned model of the opponent is subsequently used in Monte-Carlo roll-outs, in which the game is simulated n-steps ahead in order to determine the expected value of conducting a certain action. Finally, we compare the performance using two different activation functions in the multi-layer perceptron, namely the sigmoid and exponential linear unit (Elu). The results show that the Elu activation function outperforms the sigmoid activation function in most cases. Furthermore, vision grids significantly increase learning speed and in most cases this also increases the agent’s performance compared to when the full grid is used as state representation. Finally, the opponent modelling technique allows the agent to learn a predictive model of the opponent’s actions, which in combination with Monte-Carlo roll-outs significantly increases the agent’s performance.

Paper Citation

in Harvard Style

Knegt S., M. Drugan M. and A. Wiering M. (2018). Opponent Modelling in the Game of Tron using Reinforcement Learning.In Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-275-2, pages 29-40. DOI: 10.5220/0006536300290040

in Bibtex Style

@conference{icaart18,
author={Stefan J. L. Knegt and Madalina M. Drugan and Marco A. Wiering},
title={Opponent Modelling in the Game of Tron using Reinforcement Learning},
booktitle={Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2018},
pages={29-40},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006536300290040},
isbn={978-989-758-275-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Opponent Modelling in the Game of Tron using Reinforcement Learning
SN - 978-989-758-275-2
AU - Knegt S.
AU - M. Drugan M.
AU - A. Wiering M.
PY - 2018
SP - 29
EP - 40
DO - 10.5220/0006536300290040