Visual Counterfactual Explanations Using Semantic Part Locations

Florence Böttger, Tim Cech, Willy Scheibel, Jürgen Döllner

2023

Abstract

As machine learning models are becoming more widespread and see use in high-stake decisions, the explainability of these decisions is getting more relevant. One approach for explainability are counterfactual explanations, which are defined as changes to a data point such that it appears as a different class. Their close connection to the original dataset aids their explainability. However, existing methods of creating counterfacual explanations often rely on other machine learning models, which adds an additional layer of opacity to the explanations. We propose additions to an established pipeline for creating visual counterfacual explanations by using an inherently explainable algorithm that does not rely on external models. Using annotated semantic part locations, we replace parts of the counterfactual creation process. We evaluate the approach on the CUB-200-2011 dataset. Our approach outperforms the previous results: we improve (1) the average number of edits by 0.1 edits, (2) the keypoint accuracy of editing within any semantic parts of the image by an average of at least 7 percentage points, and (3) the keypoint accuracy of editing the same semantic parts by at least 17 percentage points.

Paper Citation

in Harvard Style

Böttger F., Cech T., Scheibel W. and Döllner J. (2023). Visual Counterfactual Explanations Using Semantic Part Locations. In Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-671-2, SciTePress, pages 63-74. DOI: 10.5220/0012179000003598

in Bibtex Style

@conference{kdir23,
author={Florence Böttger and Tim Cech and Willy Scheibel and Jürgen Döllner},
title={Visual Counterfactual Explanations Using Semantic Part Locations},
booktitle={Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2023},
pages={63-74},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012179000003598},
isbn={978-989-758-671-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Visual Counterfactual Explanations Using Semantic Part Locations
SN - 978-989-758-671-2
AU - Böttger F.
AU - Cech T.
AU - Scheibel W.
AU - Döllner J.
PY - 2023
SP - 63
EP - 74
DO - 10.5220/0012179000003598
PB - SciTePress