Unsupervised Learning of State Representation using Balanced View Spatial Deep InfoMax: Evaluation on Atari Games

Menore Mengistu, Menore Mengistu, Getachew Alemu, Pierre Chevaillier, Pierre De Loor

2022

Abstract

In this paper, we present an unsupervised state representation learning of spatio-temporally evolving sequences of autonomous agents’ observations. Our method uses contrastive learning through mutual information (MI) maximization between a sample and the views derived through selection of pixels from the sample and other randomly selected negative samples. Our method employs balancing MI by finding the optimal ratios of positive-to-negative pixels in these derived (constructed) views. We performed several experiments and determined the optimal ratios of positive-to-negative signals to balance the MI between a given sample and the constructed views. The newly introduced method is named as Balanced View Spatial Deep InfoMax (BVS-DIM). We evaluated our method on Atari games and performed comparisons with the state-of-the-art unsupervised state representation learning baseline method. We show that our solution enables to successfully learn state representations from sparsely sampled or randomly shuffled observations. Our BVS-DIM method also marginally enhances the representation powers of encoders to capture high-level latent factors of the agents’ observations when compared with the baseline method.

Paper Citation

in Harvard Style

Mengistu M., Alemu G., Chevaillier P. and De Loor P. (2022). Unsupervised Learning of State Representation using Balanced View Spatial Deep InfoMax: Evaluation on Atari Games. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-547-0, pages 110-119. DOI: 10.5220/0010785000003116

in Bibtex Style

@conference{icaart22,
author={Menore Mengistu and Getachew Alemu and Pierre Chevaillier and Pierre De Loor},
title={Unsupervised Learning of State Representation using Balanced View Spatial Deep InfoMax: Evaluation on Atari Games},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2022},
pages={110-119},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010785000003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Unsupervised Learning of State Representation using Balanced View Spatial Deep InfoMax: Evaluation on Atari Games
SN - 978-989-758-547-0
AU - Mengistu M.
AU - Alemu G.
AU - Chevaillier P.
AU - De Loor P.
PY - 2022
SP - 110
EP - 119
DO - 10.5220/0010785000003116