From Explanations to Segmentation: Using Explainable AI for Image Segmentation

Clemens Seibold, Johannes Künzel, Anna Hilsmann, Peter Eisert, Peter Eisert

2022

Abstract

The new era of image segmentation leveraging the power of Deep Neural Nets (DNNs) comes with a price tag: to train a neural network for pixel-wise segmentation, a large amount of training samples has to be manually labeled on pixel-precision. In this work, we address this by following an indirect solution. We build upon the advances of the Explainable AI (XAI) community and extract a pixel-wise binary segmentation from the output of the Layer-wise Relevance Propagation (LRP) explaining the decision of a classification network. We show that we achieve similar results compared to an established U-Net segmentation architecture, while the generation of the training data is significantly simplified. The proposed method can be trained in a weakly supervised fashion, as the training samples must be only labeled on image-level, at the same time enabling the output of a segmentation mask. This makes it especially applicable to a wider range of real applications where tedious pixel-level labelling is often not possible.

Paper Citation

in Harvard Style

Seibold C., Künzel J., Hilsmann A. and Eisert P. (2022). From Explanations to Segmentation: Using Explainable AI for Image Segmentation. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 616-626. DOI: 10.5220/0010893600003124

in Bibtex Style

@conference{visapp22,
author={Clemens Seibold and Johannes Künzel and Anna Hilsmann and Peter Eisert},
title={From Explanations to Segmentation: Using Explainable AI for Image Segmentation},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={616-626},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010893600003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - From Explanations to Segmentation: Using Explainable AI for Image Segmentation
SN - 978-989-758-555-5
AU - Seibold C.
AU - Künzel J.
AU - Hilsmann A.
AU - Eisert P.
PY - 2022
SP - 616
EP - 626
DO - 10.5220/0010893600003124
PB - SciTePress