Enabling Markovian Representations under Imperfect Information

Francesco Belardinelli, Francesco Belardinelli, Borja G. León, Vadim Malvone

2022

Abstract

Markovian systems are widely used in reinforcement learning (RL), when the successful completion of a task depends exclusively on the last interaction between an autonomous agent and its environment. Unfortunately, real-world instructions are typically complex and often better described as non-Markovian. In this paper we present an extension method that allows solving partially-observable non-Markovian reward decision processes (PONMRDPs) by solving equivalent Markovian models. This potentially facilitates Markovian-based state-of-the-art techniques, including RL, to find optimal behaviours for problems best described as PONMRDP. We provide formal optimality guarantees of our extension methods together with a counterexample illustrating that naive extensions from existing techniques in fully-observable environments cannot provide such guarantees.

Paper Citation

in Harvard Style

Belardinelli F., G. León B. and Malvone V. (2022). Enabling Markovian Representations under Imperfect Information. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-547-0, pages 450-457. DOI: 10.5220/0010882200003116

in Bibtex Style

@conference{icaart22,
author={Francesco Belardinelli and Borja G. León and Vadim Malvone},
title={Enabling Markovian Representations under Imperfect Information},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2022},
pages={450-457},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010882200003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Enabling Markovian Representations under Imperfect Information
SN - 978-989-758-547-0
AU - Belardinelli F.
AU - G. León B.
AU - Malvone V.
PY - 2022
SP - 450
EP - 457
DO - 10.5220/0010882200003116