Comparative Performance Analysis of Active Learning Strategies for the Entity Recognition Task

Philipp Kohl, Yoka Krämer, Claudia Fohry, Bodo Kraft

2024

Abstract

Supervised learning requires a lot of annotated data, which makes the annotation process time-consuming and expensive. Active Learning (AL) offers a promising solution by reducing the number of labeled data needed while maintaining model performance. This work focuses on the application of supervised learning and AL for (named) entity recognition, which is a subdiscipline of Natural Language Processing (NLP). Despite the potential of AL in this area, there is still a limited understanding of the performance of different approaches. We address this gap by conducting a comparative performance analysis with diverse, carefully selected corpora and AL strategies. Thereby, we establish a standardized evaluation setting to ensure reproducibility and consistency across experiments. With our analysis, we discover scenarios where AL provides performance improvements and others where its benefits are limited. In particular, we find that strategies including historical information from the learning process and maximizing entity information yield the most significant improvements. Our findings can guide researchers and practitioners in optimizing their annotation efforts.

Paper Citation

in Harvard Style

Kohl P., Krämer Y., Fohry C. and Kraft B. (2024). Comparative Performance Analysis of Active Learning Strategies for the Entity Recognition Task. In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-716-0, SciTePress, pages 480-488. DOI: 10.5220/0013068200003838

in Bibtex Style

@conference{kdir24,
author={Philipp Kohl and Yoka Krämer and Claudia Fohry and Bodo Kraft},
title={Comparative Performance Analysis of Active Learning Strategies for the Entity Recognition Task},
booktitle={Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2024},
pages={480-488},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013068200003838},
isbn={978-989-758-716-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Comparative Performance Analysis of Active Learning Strategies for the Entity Recognition Task
SN - 978-989-758-716-0
AU - Kohl P.
AU - Krämer Y.
AU - Fohry C.
AU - Kraft B.
PY - 2024
SP - 480
EP - 488
DO - 10.5220/0013068200003838
PB - SciTePress