GAN-based Intrinsic Exploration for Sample Efficient Reinforcement Learning

Doğay Kamar, Nazím Üre, Nazím Üre, Gözde Ünal, Gözde Ünal

2022

Abstract

In this study, we address the problem of efficient exploration in reinforcement learning. Most common exploration approaches depend on random action selection, however these approaches do not work well in environments with sparse or no rewards. We propose Generative Adversarial Network-based Intrinsic Reward Module that learns the distribution of the observed states and sends an intrinsic reward that is computed as high for states that are out of distribution, in order to lead agent to unexplored states. We evaluate our approach in Super Mario Bros for a no reward setting and in Montezuma’s Revenge for a sparse reward setting and show that our approach is indeed capable of exploring efficiently. We discuss a few weaknesses and conclude by discussing future works.

Paper Citation

in Harvard Style

Kamar D., Üre N. and Ünal G. (2022). GAN-based Intrinsic Exploration for Sample Efficient Reinforcement Learning. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-547-0, pages 264-272. DOI: 10.5220/0010825500003116

in Bibtex Style

@conference{icaart22,
author={Doğay Kamar and Nazím Üre and Gözde Ünal},
title={GAN-based Intrinsic Exploration for Sample Efficient Reinforcement Learning},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2022},
pages={264-272},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010825500003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - GAN-based Intrinsic Exploration for Sample Efficient Reinforcement Learning
SN - 978-989-758-547-0
AU - Kamar D.
AU - Üre N.
AU - Ünal G.
PY - 2022
SP - 264
EP - 272
DO - 10.5220/0010825500003116