Interpolated Experience Replay for Continuous Environments

Wenzel Pilar von Pilchau, Anthony Stein, Jörg Hähner

2022

Abstract

The concept of Experience Replay is a crucial element in Deep Reinforcement Learning algorithms of the DQN family. The basic approach reuses stored experiences to, amongst other reasons, overcome the problem of catastrophic forgetting and as a result stabilize learning. However, only experiences that the learner observed in the past are used for updates. We anticipate that these experiences posses additional valuable information about the underlying problem that just needs to be extracted in the right way. To achieve this, we present the Interpolated Experience Replay technique that leverages stored experiences to create new, synthetic ones by means of interpolation. A previous proposed concept for discrete-state environments is extended to work in continuous problem spaces. We evaluate our approach on the MountainCar benchmark environment and demonstrate its promising potential.

Paper Citation

in Harvard Style

Pilar von Pilchau W., Stein A. and Hähner J. (2022). Interpolated Experience Replay for Continuous Environments. In Proceedings of the 14th International Joint Conference on Computational Intelligence (IJCCI 2022) - Volume 1: NCTA; ISBN 978-989-758-611-8, SciTePress, pages 237-248. DOI: 10.5220/0011326900003332

in Bibtex Style

@conference{ncta22,
author={Wenzel Pilar von Pilchau and Anthony Stein and Jörg Hähner},
title={Interpolated Experience Replay for Continuous Environments},
booktitle={Proceedings of the 14th International Joint Conference on Computational Intelligence (IJCCI 2022) - Volume 1: NCTA},
year={2022},
pages={237-248},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011326900003332},
isbn={978-989-758-611-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computational Intelligence (IJCCI 2022) - Volume 1: NCTA
TI - Interpolated Experience Replay for Continuous Environments
SN - 978-989-758-611-8
AU - Pilar von Pilchau W.
AU - Stein A.
AU - Hähner J.
PY - 2022
SP - 237
EP - 248
DO - 10.5220/0011326900003332
PB - SciTePress