Optimizing Sensor Redundancy in Sequential Decision-Making Problems

Jonas Nüßlein, Maximilian Zorn, Fabian Ritz, Jonas Stein, Gerhard Stenzel, Julian Schönberger, Thomas Gabor, Claudia Linnhoff-Popien

2025

Abstract

Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards. In real-world, i.e. not simulated, environments, sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions. A significant challenge in deploying RL policies in real-world scenarios is handling sensor dropouts, which can result from hardware malfunctions, physical damage, or environmental factors like dust on a camera lens. A common strategy to mitigate this issue is to use backup sensors, though this comes with added costs. This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold, C. Our approach uses a second-order approximation of expected returns and includes penalties for exceeding cost constraints. The approach is evaluated across eight OpenAI Gym environments and a custom Unity-based robotic environment (RobotArmGrasping). Empirical results demonstrate that our quadratic program effectively approximates real expected returns, facilitating the identification of optimal sensor configurations.

Paper Citation

in Harvard Style

Nüßlein J., Zorn M., Ritz F., Stein J., Stenzel G., Schönberger J., Gabor T. and Linnhoff-Popien C. (2025). Optimizing Sensor Redundancy in Sequential Decision-Making Problems. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 245-252. DOI: 10.5220/0013086700003890

in Bibtex Style

@conference{icaart25,
author={Jonas Nüßlein and Maximilian Zorn and Fabian Ritz and Jonas Stein and Gerhard Stenzel and Julian Schönberger and Thomas Gabor and Claudia Linnhoff-Popien},
title={Optimizing Sensor Redundancy in Sequential Decision-Making Problems},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART},
year={2025},
pages={245-252},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013086700003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART
TI - Optimizing Sensor Redundancy in Sequential Decision-Making Problems
SN - 978-989-758-737-5
AU - Nüßlein J.
AU - Zorn M.
AU - Ritz F.
AU - Stein J.
AU - Stenzel G.
AU - Schönberger J.
AU - Gabor T.
AU - Linnhoff-Popien C.
PY - 2025
SP - 245
EP - 252
DO - 10.5220/0013086700003890
PB - SciTePress