Feature-level Approach for the Evaluation of Text Classification Models

Vanessa Bracamonte, Seira Hidano, Toru Nakamura, Shinsaku Kiyomoto

2022

Abstract

Visualization of explanations of text classification models is important for their evaluation. The evaluation of these models is mostly based on visualization techniques that apply to a datapoint level. Although a feature-level evaluation is possible with current visualization libraries, existing approaches do not yet implement ways for an evaluator to visualize how a text classification model behaves for features of interest for the whole data or a subset of it. In this paper, we describe and evaluate a simple feature-level approach that leverages existing interpretability methods and visualization techniques to provide evaluators information on the importance of specific features in the behavior of a text classification model. We conduct case studies of two types of text classification models: a movie review sentiment classification model and a comment toxicity model. The results show that a feature-level explanation visualization approach can help identify problems with the models.

Paper Citation

in Harvard Style

Bracamonte V., Hidano S., Nakamura T. and Kiyomoto S. (2022). Feature-level Approach for the Evaluation of Text Classification Models. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 3: IVAPP; ISBN 978-989-758-555-5, SciTePress, pages 164-170. DOI: 10.5220/0010846900003124

in Bibtex Style

@conference{ivapp22,
author={Vanessa Bracamonte and Seira Hidano and Toru Nakamura and Shinsaku Kiyomoto},
title={Feature-level Approach for the Evaluation of Text Classification Models},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 3: IVAPP},
year={2022},
pages={164-170},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010846900003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 3: IVAPP
TI - Feature-level Approach for the Evaluation of Text Classification Models
SN - 978-989-758-555-5
AU - Bracamonte V.
AU - Hidano S.
AU - Nakamura T.
AU - Kiyomoto S.
PY - 2022
SP - 164
EP - 170
DO - 10.5220/0010846900003124
PB - SciTePress