Data Augmentation for Reliability and Fairness in Counselling Quality Classification

Vivek Kumar, Vivek Kumar, Simone Balloccu, Simone Balloccu, Zixiu Wu, Zixiu Wu, Ehud Reiter, Rim Helaoui, Diego Recupero, Daniele Riboni

2022

Abstract

The mental health domain poses serious challenges to the validity of existing Natural Language Processing (NLP) approaches. Scarce and unbalanced data limits models’ reliability and fairness, therefore hampering real-world application. In this work, we address these challenges by using our recently released Anno-MI dataset, containing professionally annotated transcriptions in motivational interviewing (MI). To do so, we inspect the effects of data augmentation on classical machine (CML) and deep learning (DL) approaches for counselling quality classification. First, we adopt augmentation to balance the target label in order to improve the classifiers’ reliability. Next, we conduct the bias and fairness analysis by choosing the therapy topic as the sensitive variable. Finally, we implement a fairness-aware augmentation technique, showing how topic-wise bias can be mitigated by augmenting the target label with respect to the sensitive variable.Our work is the first step towards increasing reliability and reducing the bias of classification models, as well as dealing with data scarcity and imbalance in mental health.

Paper Citation

in Harvard Style

Kumar V., Balloccu S., Wu Z., Reiter E., Helaoui R., Recupero D. and Riboni D. (2022). Data Augmentation for Reliability and Fairness in Counselling Quality Classification. In Proceedings of the 1st Workshop on Scarce Data in Artificial Intelligence for Healthcare - Volume 1: SDAIH, ISBN 978-989-758-629-3, SciTePress, pages 23-28. DOI: 10.5220/0011531400003523

in Bibtex Style

@conference{sdaih22,
author={Vivek Kumar and Simone Balloccu and Zixiu Wu and Ehud Reiter and Rim Helaoui and Diego Recupero and Daniele Riboni},
title={Data Augmentation for Reliability and Fairness in Counselling Quality Classification},
booktitle={Proceedings of the 1st Workshop on Scarce Data in Artificial Intelligence for Healthcare - Volume 1: SDAIH,},
year={2022},
pages={23-28},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011531400003523},
isbn={978-989-758-629-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st Workshop on Scarce Data in Artificial Intelligence for Healthcare - Volume 1: SDAIH,
TI - Data Augmentation for Reliability and Fairness in Counselling Quality Classification
SN - 978-989-758-629-3
AU - Kumar V.
AU - Balloccu S.
AU - Wu Z.
AU - Reiter E.
AU - Helaoui R.
AU - Recupero D.
AU - Riboni D.
PY - 2022
SP - 23
EP - 28
DO - 10.5220/0011531400003523
PB - SciTePress