Using Shapley Additive Explanations to Explain a Deep Reinforcement Learning Agent Controlling a Turtlebot3 for Autonomous Navigation

Sindre Remman, Anastasios Lekkas

2024

Abstract

We employ Shapley Additive Explanations (SHAP) to interpret a DRL agent’s decisions, trained using the Soft-Actor Critic algorithm for controlling a TurtleBot3 via LiDAR in a simulated environment. To leverage spatial correlations between laser data points, we use a neural network with a convolutional first layer to extract features, followed by feedforward layers to choose actions based on these extracted features. We use the Gazebo simulator and Robotic Operating System (ROS) to simulate and control the TurtleBot3, and we implement visualization of the calculated SHAP values using rviz, coloring the LiDAR states based on their SHAP values. Our contributions are as follows: (1) To our knowledge, this is the first research paper using the SHAP method to explain the decision-making of a DRL agent controlling a mobile robot using LiDAR data states. (2) We introduce a visualization approach by clustering LiDAR data points using Density-based spatial clustering of applications with noise (DBSCAN) and visualizing the average SHAP values for each cluster to improve interpretability. Our results show that although the agent often makes decisions based on human-interpretable information, such as an obstacle on the left necessitating a right turn and vice versa, the agent has also learned to use information that is not human-interpretable. We hypothesize and discuss if this indicates the policy is overfitted to the map used for gathering data.

Paper Citation

in Harvard Style

Remman S. and Lekkas A. (2024). Using Shapley Additive Explanations to Explain a Deep Reinforcement Learning Agent Controlling a Turtlebot3 for Autonomous Navigation. In Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO; ISBN 978-989-758-717-7, SciTePress, pages 334-340. DOI: 10.5220/0012950200003822

in Bibtex Style

@conference{icinco24,
author={Sindre Remman and Anastasios Lekkas},
title={Using Shapley Additive Explanations to Explain a Deep Reinforcement Learning Agent Controlling a Turtlebot3 for Autonomous Navigation},
booktitle={Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO},
year={2024},
pages={334-340},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012950200003822},
isbn={978-989-758-717-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO
TI - Using Shapley Additive Explanations to Explain a Deep Reinforcement Learning Agent Controlling a Turtlebot3 for Autonomous Navigation
SN - 978-989-758-717-7
AU - Remman S.
AU - Lekkas A.
PY - 2024
SP - 334
EP - 340
DO - 10.5220/0012950200003822
PB - SciTePress