Quantifying Multimodality in World Models

Andreas Sedlmeier, Michael Kölle, Robert Müller, Leo Baudrexel, Claudia Linnhoff-Popien

2022

Abstract

Model-based Deep Reinforcement Learning (RL) assumes the availability of a model of an environment’s underlying transition dynamics. This model can be used to predict future effects of an agent’s possible actions. When no such model is available, it is possible to learn an approximation of the real environment, e.g. by using generative neural networks, sometimes also called World Models. As most real-world environments are stochastic in nature and the transition dynamics are oftentimes multimodal, it is important to use a modelling technique that is able to reflect this multimodal uncertainty. In order to safely deploy such learning systems in the real world, especially in an industrial context, it is paramount to consider these uncertainties. In this work, we analyze existing and propose new metrics for the detection and quantification of multimodal uncertainty in RL based World Models. The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way, which is a prerequisite for deploying RL systems in real-world settings.

Paper Citation

in Harvard Style

Sedlmeier A., Kölle M., Müller R., Baudrexel L. and Linnhoff-Popien C. (2022). Quantifying Multimodality in World Models. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-547-0, pages 367-374. DOI: 10.5220/0010898500003116

in Bibtex Style

@conference{icaart22,
author={Andreas Sedlmeier and Michael Kölle and Robert Müller and Leo Baudrexel and Claudia Linnhoff-Popien},
title={Quantifying Multimodality in World Models},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2022},
pages={367-374},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010898500003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Quantifying Multimodality in World Models
SN - 978-989-758-547-0
AU - Sedlmeier A.
AU - Kölle M.
AU - Müller R.
AU - Baudrexel L.
AU - Linnhoff-Popien C.
PY - 2022
SP - 367
EP - 374
DO - 10.5220/0010898500003116