Authors:
Nil Stolt Ansó
1
;
Anton O. Wiehe
1
;
Madalina M. Drugan
2
and
Marco A. Wiering
1
Affiliations:
1
Bernoulli Institute, Department of Artificial Intelligence, University of Groningen, Nijenborgh 9, Groningen and Netherlands
;
2
ITLearns.Online, Utrecht and Netherlands
Keyword(s):
Deep Reinforcement Learning, Computer Games, State Representation, Artificial Neural Networks.
Related
Ontology
Subjects/Areas/Topics:
Agents
;
Artificial Intelligence
;
Autonomous Systems
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Evolutionary Computing
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Symbolic Systems
;
Theory and Methods
Abstract:
The online game Agar.io has become massively popular on the internet due to its intuitive game design and its ability to instantly match players with others around the world. The game has a continuous input and action space and allows diverse agents with complex strategies to compete against each other. In this paper we focus on the pellet eating task in the game, in which an agent has to learn to optimize its navigation strategy to grow maximally in size within a specific time period. This work first investigates how different state representations affect the learning process of a Q-learning algorithm combined with artificial neural networks which are used for representing the Q-function. The representations examined range from raw pixel values to extracted handcrafted feature vision grids. Secondly, the effects of using different resolutions for the representations are examined. Finally, we compare the performance of different value function network architectures. The architectures
examined are two convolutional Deep Q-networks (DQN) of varying depth and one multilayer perceptron. The results show that the use of handcrafted feature vision grids significantly outperforms the direct use of raw pixel input. Furthermore, lower resolutions of 42×42 lead to better performances than larger resolutions of 84 × 84.
(More)