An Advanced BERT LayerSum Model for Sentiment Classification of COVID-19 Tweets

Areeba Umair, Elio Masciari, Elio Masciari

2023

Abstract

The new coronavirus that triggered the global pandemic COVID-19 has had a profound effect on attitudes among people all around the world. People and communities have experienced a wide range of feelings and attitudes as a result of the pandemic. There was a great deal of apprehension following the original COVID-19 epidemic. People were worried about getting the infection or spreading it to their loved ones. These worries were heightened by the disease’s unknown nature and quick dissemination. This paper proposes a novel model for sentiment analysis of tweets related to the COVID-19 pandemic. The proposed model leverages BERT as a base model and improves the last four layers of BERT for the sentiment analysis task. The embeddings of the last four layers of BERT are stacked and then summed, and the obtained embeddings are concatenated with the classification token [CLS]. The goal of the study is twofold: we categorize tweets into positive, negative, and neutral sentiments and we classify the user sentiment. The paper highlights the importance of sentiment analysis in tracking public opinion and sentiment towards the COVID-19 pandemic and demonstrates the effectiveness of the proposed model in accurately classifying the sentiment of tweets related to COVID-19. The proposed model is evaluated and compared with four widely used models: KNN, SVM, Naı̈ve Bayes, and BERT, on a dataset of tweets labeled as positive, negative, or neutral. The results show that our proposed model achieved the highest accuracy, precision, and recall for negative sentiment classification compared to other models, indicating its effectiveness in sentiment analysis. The proposed model can be used for analyzing sentiment in order to provide valuable insights for decision-making processes.

Paper Citation

in Harvard Style

Umair A. and Masciari E. (2023). An Advanced BERT LayerSum Model for Sentiment Classification of COVID-19 Tweets. In Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA; ISBN 978-989-758-664-4, SciTePress, pages 144-151. DOI: 10.5220/0012128900003541

in Bibtex Style

@conference{data23,
author={Areeba Umair and Elio Masciari},
title={An Advanced BERT LayerSum Model for Sentiment Classification of COVID-19 Tweets},
booktitle={Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA},
year={2023},
pages={144-151},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012128900003541},
isbn={978-989-758-664-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA
TI - An Advanced BERT LayerSum Model for Sentiment Classification of COVID-19 Tweets
SN - 978-989-758-664-4
AU - Umair A.
AU - Masciari E.
PY - 2023
SP - 144
EP - 151
DO - 10.5220/0012128900003541
PB - SciTePress