A Comparison of Automatic Labelling Approaches for Sentiment Analysis

Sumana Biswas, Karen Young, Josephine Griffith

2022

Abstract

Labelling a large quantity of social media data for the task of supervised machine learning is not only time-consuming but also difficult and expensive. On the other hand, the accuracy of supervised machine learning models is strongly related to the quality of the labelled data on which they train, and automatic sentiment labelling techniques could reduce the time and cost of human labelling. We have compared three automatic sentiment labelling techniques: TextBlob, Vader, and Afinn to assign sentiments to tweets without any human assistance. We compare three scenarios: one uses training and testing datasets with existing ground truth labels; the second experiment uses automatic labels as training and testing datasets; and the third experiment uses three automatic labelling techniques to label the training dataset and uses the ground truth labels for testing. The experiments were evaluated on two Twitter datasets: SemEval-2013 (DS-1) and SemEval-2016 (DS-2). Results show that the Afinn labelling technique obtains the highest accuracy of 80.17% (DS-1) and 80.05% (DS-2) using a BiLSTM deep learning model. These findings imply that automatic text labelling could provide significant benefits, and suggest a feasible alternative to the time and cost of human labelling efforts.

Paper Citation

in Harvard Style

Biswas S., Young K. and Griffith J. (2022). A Comparison of Automatic Labelling Approaches for Sentiment Analysis. In Proceedings of the 11th International Conference on Data Science, Technology and Applications - Volume 1: DATA, ISBN 978-989-758-583-8, pages 312-319. DOI: 10.5220/0011265900003269

in Bibtex Style

@conference{data22,
author={Sumana Biswas and Karen Young and Josephine Griffith},
title={A Comparison of Automatic Labelling Approaches for Sentiment Analysis},
booktitle={Proceedings of the 11th International Conference on Data Science, Technology and Applications - Volume 1: DATA,},
year={2022},
pages={312-319},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011265900003269},
isbn={978-989-758-583-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Data Science, Technology and Applications - Volume 1: DATA,
TI - A Comparison of Automatic Labelling Approaches for Sentiment Analysis
SN - 978-989-758-583-8
AU - Biswas S.
AU - Young K.
AU - Griffith J.
PY - 2022
SP - 312
EP - 319
DO - 10.5220/0011265900003269