Towards More Reliable Text Classification on Edge Devices via a Human-in-the-Loop

Jakob Andersen, Olaf Zukunft

2022

Abstract

Reliably classifying huge amounts of textual data is a primary objective of many machine learning applications. However, state-of-the-art text classifiers require extensive computational resources, which limit their applicability in real-world scenarios. In order to improve the application of lightweight classifiers on edge devices, e.g. personal work stations, we adapt the Human-in-the-Loop paradigm to improve the accuracy of classifiers without re-training by manually validating and correcting parts of the classification outcome. This paper performs a series of experiments to empirically assess the performance of the uncertainty-based Human-in-the-Loop classification of nine lightweight machine learning classifiers on four real-world classification tasks using pre-trained SBERT encodings as text features. Since time efficiency is crucial for interactive machine learning pipelines, we further compare the training and inference time to enable rapid interactions. Our results indicate that lightweight classifiers with a human in the loop can reach strong accuracies, e.g. improving a classifier’s F1-Score from 90.19 to 97% when 22.62% of a dataset is classified manually. In addition, we show that SBERT based classifiers are time efficient and can be re-trained in < 4 seconds using a Logistic Regression model.

Paper Citation

in Harvard Style

Andersen J. and Zukunft O. (2022). Towards More Reliable Text Classification on Edge Devices via a Human-in-the-Loop. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-547-0, pages 636-646. DOI: 10.5220/0010980600003116

in Bibtex Style

@conference{icaart22,
author={Jakob Andersen and Olaf Zukunft},
title={Towards More Reliable Text Classification on Edge Devices via a Human-in-the-Loop},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2022},
pages={636-646},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010980600003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Towards More Reliable Text Classification on Edge Devices via a Human-in-the-Loop
SN - 978-989-758-547-0
AU - Andersen J.
AU - Zukunft O.
PY - 2022
SP - 636
EP - 646
DO - 10.5220/0010980600003116