Debiasing Sentence Embedders Through Contrastive Word Pairs

Philip Kenneweg, Sarah Schröder, Alexander Schulz, Barbara Hammer

2023

Abstract

Over the last years, various sentence embedders have been an integral part in the success of current machine learning approaches to Natural Language Processing (NLP). Unfortunately, multiple sources have shown that the bias, inherent in the datasets upon which these embedding methods are trained, is learned by them. A variety of different approaches to remove biases in embeddings exists in the literature. Most of these approaches are applicable to word embeddings and in fewer cases to sentence embeddings. It is problematic that most debiasing approaches are directly transferred from word embeddings, therefore these approaches fail to take into account the nonlinear nature of sentence embedders and the embeddings they produce. It has been shown in literature that bias information is still present if sentence embeddings are debiased using such methods. In this contribution, we explore an approach to remove linear and nonlinear bias information for NLP solutions, without impacting downstream performance. We compare our approach to common debiasing methods on classical bias metrics and on bias metrics which take nonlinear information into account.

Paper Citation

in Harvard Style

Kenneweg P., Schröder S., Schulz A. and Hammer B. (2023). Debiasing Sentence Embedders Through Contrastive Word Pairs. In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-626-2, pages 205-212. DOI: 10.5220/0011615300003411

in Bibtex Style

@conference{icpram23,
author={Philip Kenneweg and Sarah Schröder and Alexander Schulz and Barbara Hammer},
title={Debiasing Sentence Embedders Through Contrastive Word Pairs},
booktitle={Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2023},
pages={205-212},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011615300003411},
isbn={978-989-758-626-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Debiasing Sentence Embedders Through Contrastive Word Pairs
SN - 978-989-758-626-2
AU - Kenneweg P.
AU - Schröder S.
AU - Schulz A.
AU - Hammer B.
PY - 2023
SP - 205
EP - 212
DO - 10.5220/0011615300003411