An Advanced BERT LayerSum Model for Sentiment Classiﬁcation of

COVID-19 Tweets

Areeba Umair

and Elio Masciari

1,2

Department of Electrical Engineering and Information Technology, University of Naples,

Federico II, Via Claudio, Naples, 80125, Campania, Italy

ICAR-CNR, Rende, Italy

Keywords:

Sentiment Analysis, COVID-19, BERT Model, Artiﬁcial Intelligence.

Abstract:

The new coronavirus that triggered the global pandemic COVID-19 has had a profound effect on attitudes

among people all around the world. People and communities have experienced a wide range of feelings and

attitudes as a result of the pandemic. There was a great deal of apprehension following the original COVID-19

epidemic. People were worried about getting the infection or spreading it to their loved ones. These worries

were heightened by the disease’s unknown nature and quick dissemination. This paper proposes a novel model

for sentiment analysis of tweets related to the COVID-19 pandemic. The proposed model leverages BERT as a

base model and improves the last four layers of BERT for the sentiment analysis task. The embeddings of the

last four layers of BERT are stacked and then summed, and the obtained embeddings are concatenated with

the classiﬁcation token [CLS]. The goal of the study is twofold: we categorize tweets into positive, negative,

and neutral sentiments and we classify the user sentiment. The paper highlights the importance of sentiment

analysis in tracking public opinion and sentiment towards the COVID-19 pandemic and demonstrates the

effectiveness of the proposed model in accurately classifying the sentiment of tweets related to COVID-19.

The proposed model is evaluated and compared with four widely used models: KNN, SVM, Na

ıve Bayes,

and BERT, on a dataset of tweets labeled as positive, negative, or neutral. The results show that our proposed

model achieved the highest accuracy, precision, and recall for negative sentiment classiﬁcation compared to

other models, indicating its effectiveness in sentiment analysis. The proposed model can be used for analyzing

sentiment in order to provide valuable insights for decision-making processes.

1 INTRODUCTION

COVID-19, declared as pandemic by WHO (world

health organization), is one of the most severe pan-

demic, we faced. It has not only caused health issues

all over the globe but it created feelings of fear, anx-

iety among people. Many strategies such as social

distancing, face mask, usage of sanitizers, COVID-19

infection testing kits, bans on gathering, travel restric-

tions, closures of schools, workplaces, distant learn-

ing etc. were adopted by many states and resulted

in a positive outcome for the control of COVID-19

uner et al., 2020). Because of these constraints,

world is facing a serious shutdown and now it de-

mands a speedy recovery from all the negative im-

pact caused by COVID-19. With the rapid spread of

the virus and the resulting changes to our daily lives,

there has been a signiﬁcant shift in people’s emotions

and attitudes towards various aspects of life (Umair

and Masciari, 2023).

The COVID-19 pandemic has had a profound ef-

fect on many facets of society and the global commu-

nity. The following are some of the main COVID-19

effects:

• Health Impact: COVID-19 has been blamed for

millions of fatalities, leading to a considerable

loss of life worldwide. For some people who have

recovered from the acute sickness, it has also led

to long-term health issues.

• Economic Impact: As a result of the pandemic’s

impact on the world economy, numerous ﬁrms

have been forced to shut down or reduce their op-

erations. People and families all throughout the

world have been impacted by job losses, declin-

ing earnings, and ﬁnancial uncertainty. Although

governments and central banks have taken a num-

ber of steps to lessen the impact on the economy,

it will likely take some time for things to get back

to normal.

144

Umair, A. and Masciari, E.

An Advanced BERT LayerSum Model for Sentiment Classiﬁcation of COVID-19 Tweets.

DOI: 10.5220/0012128900003541

In Proceedings of the 12th International Conference on Data Science, Technology and Applications (DATA 2023) , pages 144-151

ISBN: 978-989-758-664-4; ISSN: 2184-285X

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

• Millions of students’ educations have been dis-

rupted by the pandemic’s widespread school and

university closures. Although there are now al-

ternatives for remote learning, many students still

face difﬁculties due to unequal access to resources

and technology.

• Tourism and Travel Bans: To stop the virus from

spreading, numerous nations imposed travel bans,

closed their borders, and instituted quarantine

procedures. Travel both domestically and inter-

nationally has signiﬁcantly decreased, having a

negative inﬂuence on the tourism business. Sig-

niﬁcant losses have been experienced by airlines,

hotels, and businesses that depend on tourism.

• Effects on Mental Health: The epidemic has been

hard on people’s mental health because of things

like loneliness, fear, anxiety, grief, and uncer-

tainty. There has been a major negative inﬂuence

on mental health, which has increased instances

of depression, anxiety disorders, and other mental

health diseases.

• Workplace Culture Shifts: The epidemic has ac-

celerated tendencies in digital transformation and

remote work. Due to the widespread adoption of

remote work options, workplace culture and pro-

cedures have changed. This change has both posi-

tive and bad effects, including more ﬂexibility but

also difﬁculties juggling job and family obliga-

tions and social isolation.

• Healthcare Systems: With hospitals being overrun

by COVID-19 patients, the impact on healthcare

systems has been tremendous. The pandemic has

exposed vulnerabilities and underlined the need

for preparedness and investment in healthcare sys-

tems. Resources, manpower, and infrastructure

have all been stretched.

As a matter of fact social media platforms have

been increasingly used by population, thus creating

user generated data on daily basis. Indeed, people

share their thoughts, ideas or express their feelings

about any item or service on internet in the form of so-

cial media posts or tweets (Yue et al., 2019). Users ex-

press their positive or negative sentiments about any

particular item or product or service. Sentiment anal-

ysis can be used to identify these sentiments and cat-

egorize them to obtain the user’s attitude and opinion

(Manguri et al., 2020). Sentiment analysis involves

the use of natural language processing and machine

learning techniques to analyze and classify the emo-

tions and opinions expressed in text data, such as so-

cial media posts, news articles, and customer reviews.

During COVID-19, sentiment analysis has been

used to track public opinion and sentiment on the

virus, government policies, healthcare systems (Ullah

et al., 2023), and various other related topics (Nemes

and Kiss, 2021), (Masciari, 2007). Some common

themes that have emerged from sentiment analysis

during COVID-19 include fear and anxiety, frustra-

tion with lockdown measures and restrictions, empa-

thy for healthcare workers, and gratitude for essential

workers (Umair et al., 2021). Additionally, sentiment

analysis has been used to monitor the effectiveness of

communication strategies, such as public health mes-

saging, and to identify misinformation and conspir-

acy theories (Alamoodi et al., 2021), (Fazzinga et al.,

2013).

In this paper, we propose a novel model for sen-

timent analysis. Our proposed model uses BERT as

a base model. We exploited the last four layers of

BERT for performing the sentiment analysis task. The

embeddings of last four layers of BERT are stacked

and then their sum is computed. Then, the resultant

embeddings are concatenated with the classiﬁcation

token (CLS in what follows) in order to perform sen-

timent classiﬁcation.

The rest of the paper is structured as follows: sec-

tion 2 explains the related work in the domain of senti-

ment analysis during COVID-19. Section 3 describes

the overall methodology used in analysing the senti-

ments of people during COVID-19 as well as the pro-

posed algorithm. Section 4 provides explanation on

experiments and describes the results of the experi-

ments. Moreover, section 5 concludes the study.

2 RELATED WORK

COVID-19 has caused many adverse effects on the

people’s living and many researchers analysed the

people’s feeling about COVID-19 from different view

points (Singh et al., 2021). Many people became frus-

trated with the disturbance the pandemic produced

in their daily lives as it lingered. In certain cir-

cumstances, restrictions on travel, social events, and

companies caused resentment and rage. There ex-

ists different tools, methods and techniques for per-

forming the sentiment analysis. Researchers used dif-

ferent machine learning methods for extracting the

sentiments of people about palliatives, which were

distributed in COVID-19 days using tweets (Adamu

et al., 2021). The mental of people specially students

was very disturbed after COVID’s destruction and the

analysis of student’s Mental condition by using differ-

ent method of NLP and Twitter data (Agarwal et al.,

2021) has been widely investigated. Similarly, in (Das

and Dutta, 2020) the authors worked on sentiment

analysis of people during lockdown using different

An Advanced BERT LayerSum Model for Sentiment Classiﬁcation of COVID-19 Tweets

145

techniques of plotting. Some the researchers also con-

ducted surveys for gathering the data and tried to un-

derstand the peoples’ thoughts and behaviours (Flint

et al., 2021). In (Jelodar et al., 2020), the authors

used Latent Dirichlet Allocation (LDA) to identify

the dominant topics or issues faced by people during

COVID. The restrictions imposed by governments or

stats on restaurants motivated the authors to perform

research on online restaurants reviews using differ-

ent techniques of sentiments analysis in (Luo et al.,

2020). Moreover, in (Lwin et al., 2020) the analysis

of different condition of restlessness in society (Lwin

et al., 2020) using Twitter data is presented, while in

(Praveen et al., 2021) the authors discuss the attitude

of Indian citizens. In (H. Manguri et al., 2020; Raheja

and Asthana, 2021), authors used TextBlob to esti-

mate polarity during COVID using Twitter data. As a

matter of fact, after COVID-19 vaccine release, hesi-

tancy to get vaccinated was a great obstacle in control-

ling COVID-19 spread as many people are not willing

to get themselves vaccinated. Many work addressed

this issue in order to understand what people think

about vaccines by using Facebook and Twitter data to

extract people’s feeling about vaccines using several

artiﬁcial intelligence methods(Hussain et al., 2021).

In (Bao et al., 2022), the authors suggested creat-

ing opinion trees for aspect-based sentiment analysis.

The goal of the Opinion Tree Generation is to locate

every sentimental component in a review sentence

and portray it as a semantic tree. A complete repre-

sentation of the sentence structure, including aspect

terms, opinion words, and semantic links, is given

by the opinion tree. This method may show a more

thorough aspect-level semantic structure, leading to

more accurate sentiment element extraction. Modi-

ﬁed BERT for target-oriented multimodal sentiment

categorization has been suggested in (Yu and Jiang,

2019). The BERT architecture, on which the pro-

posed model, TomBERT, is based, is frequently used

to acquire contextualized word representations using

its pre-trained model parameters from a huge corpus.

They created a target attention mechanism, inspired

by the self-attention mechanism, to further enhance

their model. This mechanism automatically recog-

nizes when opinion targets and visuals are aligned. In

(Kwan and Lim, 2021), a system for examining opin-

ions and conversations regarding COVID-19 among

the general public using Twitter activity is suggested.

The objective is to demonstrate TweetCOVID’s capa-

bilities, a system that uses tweets that are available to

the public to study COVID-19’s effects on the popu-

lation. A wide range of features are available through

the system, including data gathering and processing,

sentiment and emotion analysis, subject modeling,

controversy tracking, and visualization.

3 METHODOLOGY

We propose a novel sentiment analysis model for

COVID-19 by leveraging BERT model. For this

purpose we used publicly available Twitter data.

Our framework for sentiment analysis on COVID-19

tweets is reported in Figure 1 and is composed of three

stages.

• Stage One: We perform dataset collection and

pre-processing.

• Stage Two: We extract sentiment labels for each

tweet.

• Stage Three: We classify the tweets into positive,

negative and neutral tweets.

3.1 Data Collection and Pre-Processing

Data collection involves identifying the relevant data

sources, extracting data from them, and storing it in

a format suitable for further steps, while data pre-

processing involves cleaning and transforming the

data to ensure that they accurate, complete, and in

a suitable format for analysis. This may involve re-

moving duplicates, ﬁlling in eventual missing values,

transforming variables, and standardizing the data

(Fan et al., 2021). Proper data collection and pre-

processing are critical for ensuring the accuracy and

reliability of the analysis.

3.2 Finding Polarity Using TextBlob()

TextBlob() is a Python library used for processing tex-

tual data. It has a built-in sentiment analysis tool

that can be used to ﬁnd the polarity of a given text.

The polarity score ranges from -1 (most negative) to

1 (most positive) (Hiremath and Patil, 2022). In or-

der to use TextBlob() for ﬁnding the polarity of a

text, the following steps have been performed: 1) In-

stall TextBlob using pip install textblob; 2) Import

TextBlob from textblob module; 3) Create a TextBlob

object by passing the text as an argument; 4) Use the

sentiment property of the TextBlob object to get the

polarity score.

3.3 BERT LayerSum Model

Our proposed sentiment analysis model is shown in

Figure 2. The BERT model is used as a base model

DATA 2023 - 12th International Conference on Data Science, Technology and Applications

146

Figure 1: Proposed Framework of Sentiment Analysis

Model.

and then its last four layers are stacked and summed

using the Python functions. After this pre-processing

step the resulting CLS is concatenated with the ob-

tained embeddings and the classiﬁcation is performed

by assigning words to positive, negative and neutral

classes.

Figure 2: Proposed Sentiment Analysis Model.

3.3.1 BERT Model

BERT (Bidirectional Encoder Representations from

Transformers) is a powerful pre-trained language

model that has been shown to be highly effective in

a variety of natural language processing (NLP) tasks,

including sentiment analysis. In sentiment analysis,

BERT can be ﬁne-tuned on a labeled dataset of text

and sentiment labels to learn to predict the sentiment

of new, unseen text.

To use BERT for sentiment classiﬁcation, the

model is ﬁrst ﬁne-tuned on a labeled dataset of text

and sentiment labels. During ﬁne-tuning, BERT

learns to identify the patterns and features in the text

that are most predictive of the sentiment labels. Once

the model has been ﬁne-tuned, it can be used to pre-

dict the sentiment of new, unseen text by encoding the

text using BERT’s pre-trained language model, and

passing the encoded representation through a classiﬁ-

cation layer that maps the encoded representation to a

sentiment label.

One of the advantages of BERT is its ability to

capture the contextual meaning of words and phrases

in a sentence, as it uses a bidirectional transformer-

based architecture that reads and processes the entire

An Advanced BERT LayerSum Model for Sentiment Classiﬁcation of COVID-19 Tweets

147

sentence at once. This allow us to better understand

the sentiment of a sentence by taking into account

the relationships between the words and the context

in which they are used.

3.3.2 LayerSum Model

The hidden layers of BERT model can be exploited to

extract the sentiment from the text. The research em-

phasizes the possibilities of using the BERT model’s

hidden layers for sentiment analysis. Being a potent

pre-trained language model, BERT has powerful hid-

den layers that store rich contextual data that may be

used to extract sentiment from textual input.

Researchers can learn more about the sentiment

conveyed in the text by examining the BERT’s hid-

den layers. These concealed layers hold abstractions

of the input text at several levels, capturing both re-

gional and global contextual data. According to the

article, it is possible to ﬁnd patterns and features re-

lated to sentiment by looking at the BERT’s hidden

layers. These patterns can be used to forecast a text’s

mood, such as whether it will be positive, negative, or

neutral.

The proposed Algorithm 1 is shown in the follow-

ing.

Algorithm 1: BERT-LayerSum.

1: 1: Negative Tweets

2: 2: Positive Tweets

3: 3: Neutral Tweets

4: BERT LayerSum(): Sum of Last Four Layers

5: D: Dataset

6: Input D

7: Steps:

8: n= BERT Layers

9: for i=n-3 to n do

10: Sum()

11: end for

12: for i = 1 to size of (D) do

13: Final Embeddings ← Concatenation(CLS(D

BERT

LayerSum(D

)

14: Tweet Classiﬁcation ← Classiﬁer(Final Embeddings)

15: end for

16: Output: Tweet Classiﬁcation (Class wise probabilities)

We have described an approach that stacks the em-

beddings of the last four BERT model layers. Bidi-

rectional Encoder Representations from Transform-

ers, also known as BERT, is a well-known pre-trained

language model that has produced cutting-edge out-

comes on a number of natural language processing

tasks.

An enhanced representation of the input text is

produced by stacking the embeddings from the last

four layers. This may help the model better grasp the

content and capture more contextual information.

We used the sum function on the stacked embed-

dings after acquiring them. By adding the embed-

dings’ values, one may be able to highlight signiﬁcant

elements or patterns in the text.

We then combined the generated embeddings with

the BERT model’s CLS (classiﬁcation) token. When

utilizing BERT, a special token called the CLS token

is appended at the start of the input sequence. It con-

tains data related to the overall classiﬁcation or pre-

diction task.

The classiﬁer then used the concatenated embed-

dings to estimate class-wise probabilities by using

this representation. The concatenated embeddings are

sent into the classiﬁer, which then generates probabil-

ities for each class, indicating how likely it is that the

input belongs to that class.

By using this method, we can apply the pre-

trained representations of BERT to a classiﬁcation job

and potentially enhance the model’s performance.

4 EXPERIMENTS AND RESULTS

In order to evaluate our proposed model, we per-

formed experiments with the state-of-the-art (SOTA).

The SOTA models used for experiments are:

• BERT model: A cutting-edge pre-trained lan-

guage model, the BERT (Bidirectional Encoder

Representations from Transformers) model was

released by Google AI in 2018. It has made

major contributions to the ﬁeld of natural lan-

guage processing (NLP) and excelled in a num-

ber of NLP tasks. The Transformer architecture,

a sort of neural network that makes use of self-

attentional mechanisms to identify dependencies

between various words in a sequence, serves as

the foundation for BERT. BERT introduces the

idea of bidirectionality by taking into account

context from both the left and the right sides of

a word, in contrast to typical models that process

text in a left-to-right or right-to-left fashion.

• K Nearest Neighbour: K-nearest neighbors

(KNN) is a straightforward but efﬁcient tech-

nique used in machine learning for both classiﬁ-

cation and regression tasks. It is a non-parametric,

instance-based learning technique that is predi-

cated on the notion of data point similarity.

• Support Vector Machine: A supervised machine

learning approach called a Support Vector Ma-

chine (SVM) is utilized for both classiﬁcation and

regression tasks. Although it may be expanded to

handle multi-class classiﬁcation problems as well,

it is particularly good at handling binary classiﬁ-

DATA 2023 - 12th International Conference on Data Science, Technology and Applications

148

cation difﬁculties. The fundamental goal of SVM

is to identify a hyperplane in a high-dimensional

feature space that has the maximum separation be-

tween data points from different classes. Given

that they are the crucial data points that establish

the decision boundary, these hyperplanes are re-

ferred to as ”support vectors.”

• Naive Bayes: Naive Bayes is a simple but efﬁcient

classiﬁcation algorithm built on the Bayes theo-

rem and the assumption of feature independence.

It is frequently employed in applications involv-

ing category or discrete features, such as text clas-

siﬁcation.

To compare the performance of our proposed

model with the state-of-the-art, we used standard

evaluation matrices. The evaluation matrices reports:

a) Accuracy: the percentage of correctly classiﬁed in-

stances out of the total number of instances; b) Pre-

cision: the percentage of correctly classiﬁed positive

instances out of the total number of instances classi-

ﬁed as positive; c) Recall: the percentage of correctly

classiﬁed positive instances out of the total number of

positive instances in the dataset.

4.1 Discussion on Results

The results of positive sentiment classiﬁcation are re-

ported in Figure 3. It can be seen from the plot that

our model outperforms all state-of-the-art models by

achieving maximum accuracy, maximum precision,

and maximum recall. The proposed model achieved

the highest accuracy of 0.82, which means that it was

able to correctly classify 82% of the tweets. The pre-

cision and recall values for the proposed model are

also high at 0.88 and 0.89, respectively. This indicates

that the model not only classiﬁed the tweets correctly

but also avoided mis-classifying tweets as positive or

negative.

Comparing our model to other models, SVM

achieved the second highest accuracy, but the pre-

cision and recall values were relatively lower com-

pared to the proposed model. Classical BERT also

exhibits a relatively high accuracy while KNN and

ıve Bayes show lower accuracy and precision val-

ues compared to the other models.

Figure 4 shows the comparison of our model with

the state-of-the-art using accuracy, precision and re-

call for negative sentiment classiﬁcation. The re-

sults shows that for the negative sentiment classiﬁca-

tion, our model still outperforms all other models by

achieving highest accuracy, precision and recall.

Based on the information provided in Figure 5, it

is easy to see that our model outperforms the state-of-

the-art models in terms of accuracy, precision, and re-

Figure 3: Results of Positive sentiment classiﬁcation.

Figure 4: Results of Negative sentiment classiﬁcation.

call for neutral sentiment classiﬁcation. The rationale

for achieving better performance with our model can

be understood by considering that we leverage BERT

as a base model and the exploitation of the last four

layers of BERT for performing the sentiment analysis

task guarantee a ﬁne-tuning that is not performed by

competing approaches.

Figure 5: Results of Neutral sentiment classiﬁcation.

More in details, KNN, Naive Bayes, and SVM,

show lower performance compared to the proposed

model (BERT-based) both in terms of accuracy, pre-

cision, and recall. Main motivations for these results

are:

1. Limited feature representation: These models

might not be able to capture the complex rela-

tionships and patterns in the data as effectively as

An Advanced BERT LayerSum Model for Sentiment Classiﬁcation of COVID-19 Tweets

149

BERT, which is a pre-trained language model and

can handle large amounts of data and complex re-

lationships between words.

2. Model complexity: KNN, Naive Bayes, and SVM

are relatively simpler models compared to BERT,

and they might not be able to handle the com-

plexity of the sentiment analysis task. In contrast,

BERT has a deeper and more complex architec-

ture, allowing it to capture more nuances in the

data.

3. Hyper-parameter tuning: The performance of

these models might improve with better hyper-

parameter tuning, such as selecting optimal values

for k in KNN, or choosing an appropriate kernel

function in SVM. However, this can be a time-

consuming and computationally expensive pro-

cess.

4. Sensitivity to feature scaling: KNN relies on

distance-based similarity measures, which can be

affected by the scale of features. If the features

used in the sentiment analysis task are not prop-

erly scaled or normalized, it can lead to subopti-

mal performance of the KNN algorithm.

5. Curse of dimensionality: KNN performance

can degrade when dealing with high-dimensional

data. As the number of dimensions (features) in-

creases, the available data becomes sparse in the

high-dimensional space, and the nearest neigh-

bors may not accurately represent the underlying

patterns.

6. Independence assumption in Na

ıve Bayes: Na

ıve

Bayes assumes independence between features,

meaning that the presence or absence of one fea-

ture is independent of the presence or absence of

other features. This assumption might not hold

true for sentiment analysis, as the sentiment ex-

pressed in a text often depends on the combination

and context of multiple words and phrases. Viola-

tions of this independence assumption can lead to

lower accuracy.

7. Lack of complex pattern capture: KNN and Na

ıve

Bayes are relatively simple models compared to

more advanced deep learning models like BERT

or SVM. They may struggle to capture complex

patterns and dependencies present in natural lan-

guage data, which can result in lower accuracy in

sentiment analysis tasks.

5 CONCLUSIONS AND FUTURE

WORK

COVID-19 pandemic has signiﬁcantly impacted peo-

ple’s emotions and attitudes towards various aspects

of life. With the growing popularity of social me-

dia, sentiment analysis has become an essential tool

for tracking public opinion and sentiment towards

the virus, government policies, healthcare systems,

and related topics. In this study, we proposed

BERT LayerSum, a novel model, for sentiment anal-

ysis using BERT as a base model. Our model aims to

categorize tweets into positive, negative, and neutral

sentiments and classify them using our proposed ar-

chitecture. This paper provides insights into people’s

attitudes towards the COVID-19 pandemic, which

can help policymakers in making informed decisions.

Overall, BERT LayerSum has been shown to be a

highly effective approach for sentiment classiﬁcation,

achieving state-of-the-art results on Twitter dataset re-

lated to COVID-19. The paper’s ﬁndings underline

the importance of using the BERT model’s hidden

layers to extract sentiment from text. This strategy

can improve sentiment analysis skills and support a

number of applications, including opinion mining, so-

cial media monitoring, and customer feedback analy-

sis.

Future sentiment analysis research employing

BERT and its hidden layers may take the follow-

ing directions: Model Interpretability: Investigat-

ing techniques for understanding the BERT’s under-

lying layers and comprehending the particular char-

acteristics or language signals that support sentiment

analysis. This can involve employing strategies like

saliency mapping, attention visualization, or probing

methods to learn more about the inner workings of the

model. Examining methods to ﬁne-tune BERT for

sentiment analysis tasks in particular domains or tar-

get languages. Transfer Learning and Domain Adap-

tation. The performance of BERT might be enhanced,

and issues with domain-speciﬁc sentiment analysis

could be resolved, by adapting it to various domains

or languages. Contextual Characteristics: Senti-

ment analysis that takes into account the temporal and

contextual characteristics of a situation by incorporat-

ing knowledge of the conversation, document struc-

ture, or sentiment evolution through time. Bias and

Fairness in Sentiment Analysis: By locating and

minimizing biases in the training data or the model

itself, biases in sentiment analysis models can be ad-

dressed. Investigating debiasing techniques or creat-

ing fair sentiment analysis models that treat various

groups fairly and prevent the perpetuation of biased

trends are examples of this.

DATA 2023 - 12th International Conference on Data Science, Technology and Applications

150

REFERENCES

Adamu, H., Lutﬁ, S. L., Malim, N. H. A. H., Hassan, R.,

Di Vaio, A., and Mohamed, A. S. A. (2021). Fram-

ing twitter public sentiment on nigerian government

covid-19 palliatives distribution using machine learn-

ing. Sustainability, 13(6):3497.

Agarwal, A., Agarwal, B., Harjule, P., and Agarwal, A.

(2021). Mental health analysis of students in major

cities of india during covid-19. Predictive and preven-

tive measures for Covid-19 pandemic, pages 51–67.

Alamoodi, A. H., Zaidan, B. B., Zaidan, A. A., Albahri,

O. S., Mohammed, K. I., Malik, R. Q., Almahdi,

E. M., Chyad, M. A., Tareq, Z., Albahri, A. S.,

et al. (2021). Sentiment analysis and its applica-

tions in ﬁghting covid-19 and infectious diseases: A

systematic review. Expert systems with applications,

167:114155.

Bao, X., Wang, Z., Jiang, X., Xiao, R., and Li, S. (2022).

Aspect-based sentiment analysis with opinion tree

generation. IJCAI 2022, pages 4044–4050.

Das, S. and Dutta, A. (2020). Characterizing public emo-

tions and sentiments in COVID-19 environment: A

case study of India. J. Hum. Behav. Soc. Environ.,

31(1-4):1–14.

Fan, C., Chen, M., Wang, X., Wang, J., and Huang, B.

(2021). A review on data preprocessing techniques to-

ward efﬁcient and reliable knowledge discovery from

building operational data. Frontiers in Energy Re-

search, 9:652801.

Fazzinga, B., Flesca, S., Furfaro, F., and Masciari, E.

(2013). Rﬁd-data compression for supporting aggre-

gate queries. ACM Transactions on Database Systems

(TODS), 38(2):1–45.

Flint, S. W., Piotrkowicz, A., and Watts, K. (2021). Use of

Artiﬁcial Intelligence to understand adults’ thoughts

and behaviours relating to COVID-19. Perspect. Pub-

lic Health, XX(X):1–8.

uner, H. R., Hasano

glu,

I., and Aktas¸, F. (2020). Covid-

19: Prevention and control measures in community.

Turkish Journal of medical sciences, 50(9):571–577.

H. Manguri, K., N. Ramadhan, R., and R. Mohammed

Amin, P. (2020). Twitter Sentiment Analysis on

Worldwide COVID-19 Outbreaks. Kurdistan J. Appl.

Res., pages 54–65.

Hiremath, B. N. and Patil, M. M. (2022). Enhancing op-

timized personalized therapy in clinical decision sup-

port system using natural language processing. Jour-

nal of King Saud University-Computer and Informa-

tion Sciences, 34(6):2840–2848.

Hussain, A., Tahir, A., Hussain, Z., Sheikh, Z., Gogate, M.,

Dashtipour, K., Ali, A., and Sheikh, A. (2021). Arti-

ﬁcial intelligence-enabled analysis of public attitudes

on facebook and twitter toward COVID-19 vaccines

in the United Kingdom and the United States: Obser-

vational study. J. Med. Internet Res., 23(4):1–10.

Jelodar, H., Wang, Y., Orji, R., and Huang, H. (2020).

Deep sentiment classiﬁcation and topic discovery on

novel coronavirus or COVID-19 online discussions:

NLP using LSTM recurrent neural network approach.

arXiv, 24(10):2733–2742.

Kwan, J. S.-L. and Lim, K. H. (2021). Tweetcovid: a sys-

tem for analyzing public sentiments and discussions

about covid-19 via twitter activities. In 26th Inter-

national Conference on Intelligent User Interfaces-

Companion, pages 58–60.

Luo, W., Majumder, M. S., Liu, D., Poirier, C., Mandl,

K. D., Lipsitch, M., and Santillana, M. (2020). The

role of absolute humidity on transmission rates of the

covid-19 outbreak. MedRxiv, pages 2020–02.

Lwin, M. O., Lu, J., Sheldenkar, A., Schulz, P. J., Shin,

W., Gupta, R., and Yang, Y. (2020). Global senti-

ments surrounding the COVID-19 pandemic on Twit-

ter: Analysis of Twitter trends. JMIR Public Heal.

Surveill., 6(2):1–4.

Manguri, K. H., Ramadhan, R. N., and Amin, P. R. M.

(2020). Twitter sentiment analysis on worldwide

covid-19 outbreaks. Kurdistan Journal of Applied Re-

search, pages 54–65.

Masciari, E. (2007). Rﬁd data management for effective ob-

jects tracking. In Proceedings of the 2007 ACM sym-

posium on Applied computing, pages 457–461.

Nemes, L. and Kiss, A. (2021). Social media sentiment

analysis based on covid-19. Journal of Information

and Telecommunication, 5(1):1–15.

Praveen, S. V., Ittamalla, R., and Deepak, G. (2021). Ana-

lyzing Indian general public’s perspective on anxiety,

stress and trauma during Covid-19 - A machine learn-

ing study of 840,000 tweets. Diabetes Metab. Syndr.

Clin. Res. Rev., 15(3):667–671.

Raheja, S. and Asthana, A. (2021). Sentimental analysis of

twitter comments on COVID-19. Proc. Conﬂu. 2021

11th Int. Conf. Cloud Comput. Data Sci. Eng., pages

704–708.

Singh, M., Jakhar, A. K., and Pandey, S. (2021). Senti-

ment analysis on the impact of coronavirus in social

life using the BERT model. Soc. Netw. Anal. Min.,

11(1):1–11.

Ullah, M. H., Gelli, G., and Verde, F. (2023). Visible

light backscattering with applications to the internet

of things: State-of-the-art, challenges, and opportuni-

ties. Internet of Things, page 100768.

Umair, A. and Masciari, E. (2023). Sentimental and spa-

tial analysis of covid-19 vaccines tweets. Journal of

Intelligent Information Systems, 60(1):1–21.

Umair, A., Masciari, E., and Habib Ullah, M. H. (2021).

Sentimental analysis applications and approaches dur-

ing covid-19: a survey. In Proceedings of the 25th

International Database Engineering & Applications

Symposium, pages 304–308.

Yu, J. and Jiang, J. (2019). Adapting bert for target-oriented

multimodal sentiment classiﬁcation. IJCAI.

Yue, L., Chen, W., Li, X., Zuo, W., and Yin, M. (2019). A

survey of sentiment analysis in social media. Knowl-

edge and Information Systems, 60:617–663.

An Advanced BERT LayerSum Model for Sentiment Classiﬁcation of COVID-19 Tweets

151