Arabic Sentiment Analysis based on Neural Network Models:

Overview and Comparison

Youssra Zahidi

Yacine El Younoussi

and Yassine Al-Amrani

Information System and Software Engineering Laboratory, Abdelmalek Essaadi University, Morocco

Information Technology and Modeling Systems (TIMS) research team, Abdelmalek Essaadi University, Morocco

Keywords: Arabic Sentiment Analysis (ASA), Deep Learning (DL), Neural Network (NN) Models.

Abstract: Sentiment Analysis (SA) or opinion mining tries to select the sentiment orientation (positive, neutral, or

negative) of a text. Arabic Sentiment analysis (ASA) is complicated, it is considered a challenging task

compared to other foreign languages by dint of the complications of Arabic at the level of morphology,

orthography, its ambiguity, the lack of resources, and various dialects. Deep learning (DL) is a kind of

machine learning (ML) and contains several Neural Network (NN) models. The purpose of our work is to

debate the issue of DL models that is very important in the ASA domain also provide a comparative analysis

of the most valuable and famous NN models that gain salient results in this field, namely: ANN, CNN, RNN,

and LSTM. We found through this deep evaluation that the NN models: CNN and LSTM that is a type of

RNN have numerous benefits in the ASA field.

1 INTRODUCTION

A wide range of social media platforms users

expresses their opinions and feelings about various

topics. A vast number of the comments is not

unstructured in a pre-defined and significant manner.

That is why there is a high need to classify the data.

SA Systems is an excellent way to solve this task,

where the objective is to predict the sentiment

conveyed in some text.

As part of our ASA issue, we try to show our

comparative evaluation of many existing NN Models

to conclude a general decision on the most important

ones in the ASA domain.

The various kinds of neural networks in DL are

transforming the method we interact with several

modern tasks. These various NN models’ kinds are

very important in the DL revolution, powering

applications such as Sentiment Analysis, especially in

the Arabic language. Due to the several Arabic

dialects together with its complex structure and the

very high lack of its resources, the Arabic language is

complicated compared to the English language, for

example. All these challenges, make the ASA

research domain very difficult. Selecting the most

suitable collection of NN Models that satisfies our

requirements best is arduous work. For this reason,

we base this deep evaluation on multiple good

aspects.

This deep evaluation of the NN models is very

important. It would enable ASA researchers to select

the most appropriate group of NN Models in their

projects to make good and correct decisions.

The following parts of this work are organized as

follows: Section 2 gives the difference between ML

and DL. Section 3 outlines NN Models for ASA with

a comparative analysis of these NN Models. In

Section 4, the comparison between NN Models is

discussed in detail, and this work is concluded with

definitive ideas in Section 5.

2 DIFFERENCE BETWEEN

MACHINE LEARNING AND

DEEP LEARNING

DL is a technique of ML that relies on a NN with

various deep layers to process the data. therefore, it

learns complicated features from features that are

very simple as it proceeds to start by lower to higher

layers utilizing weights for each link and real-number

activations for each neuron.

ML tries to extract new knowledge from various

and several data that are preprocessed and loaded in

Zahidi, Y., El Younoussi, Y. and Al-Amrani, Y.

Arabic Sentiment Analysis based on Neural Network Models: Overview and Comparison.

DOI: 10.5220/0010728700003101

In Proceedings of the 2nd International Conference on Big Data, Modelling and Machine Learning (BML 2021), pages 77-80

ISBN: 978-989-758-559-3

the system.The rules are formulated by the users and

destined for the machine, and it relies on them.

Occasionally, users might interfere to treat and

correct the errors. On the other hand, DL is a little

different:

Table 1: Comparison between ML and DL.

3 NEURAL NETWORK MODELS

FOR ARABIC SENTIMENT

ANALYSIS

A large variety of Neural Network algorithms have

been utilized to resolve the sentiment analysis task. In

this work, we concentrate on three NN models that are

considered the most valuable and achieved numerous

benefits and outstanding results in the ASA field.

3.1 Artificial Neural Network

ANN is a set of several neurons or perceptrons at each

layer. Besides, it is named a Feed-Forward NN

because inputs are treated only in the forward

direction. This model is considered the most

straightforward variant of NN, it passes information

in one direction only, through several input nodes,

until it makes it to the node of output. The network

may have hidden node layers or may not, making

their functioning more interpretable.

3.2 Convolutional Neural Network

CNN is considered the most valuable and famous

model used in various fields, especially in ASA. This

computational model has one or multiple

convolutional layers that can be either entirely pooled

or connected and employs a variation of multilayer

perceptrons.

3.3 Recurrent Neural Network

RNN is more complex compared to other NN models.

This type of neural network model didn't transmit the

information in only one direction. This is how the

RNN model is said to learn predicting the layer

outcome. Each model's node acts as a memory cell,

continuing the operations computation and

implementation. If the prediction of the network is

incorrect, then during backpropagation, the system

self-learns and keeps running towards the correct

prediction. RNN contains numerous kinds. But there

is a specific type of RNN, capable of learning

lengthy-time period dependencies that gained

significant results.

 LSTM: is a unique type of RNN network

that is very effective in dealing with learning

long-term dependencies and long sequence

data. LSTM is widely used today for various

tasks, especially in ASA.

The following tables: Table 3 and Table 4, highlights

the numerous criteria to evaluate these NN Models.

We can deduce through these tables that each Neural

Network Model has its characteristics.

Table 2: Multiple Criteria of

Long Short-Term Memory Model.

Characteristics Advanta

es Disadvanta

es Works in ASA

LSTM

Model

LSTM relies on the concept of

gates and has three:

 Input gate: controls the

amount of incoming

information.

 Forget gate: controls the

information flow from the

previous memory state.

 Output gate: controls the

amount of outgoing

information

 They are more

robust to the

challenge of short

memory than

‘Vanilla’ RNNs.



They can model

long-term sequence

dependencies.

 The memory needed is

higher than the one of

‘Vanilla’ RNNs due to

multiple memory cells.

 They raise the

computing perplexity

compared to the RNN

model with the

introduction of more

parameters to learn.

(Ombabi et

al.)(Heikal et

al.)(Elfaik and

Nfaoui) (Abu

Kwaik et

al.)(Alayba et

al.)(Abdullah et

al.) (Albayati et

al.)(Al Omari et

al.)(Wahdan et al.)

ML DL

Huge data amounts

Short datasets, as long as

they are of good quality

Computation-heavy Not always

A draw precise

conclusion from raw data

Accurately pre-processed

data

Take a very long time to

train

Can take a reduced time

to train

Incapability to know what

are the specific features

that the neurons

exemplify.

The clarity of the logic

behind the decision of

machine

It can be utilized in

unforeseen manners

For resolving a particular

problem, the algorithm is

created.

BML 2021 - INTERNATIONAL CONFERENCE ON BIG DATA, MODELLING AND MACHINE LEARNING (BML’21)

Table 3: comparison of the most useful NN Models based on various characteristics.

Table 4: comparison of the most useful NN Models based on multiple criteria.

ANN RNN CNN

Advantages

 Containing a distributed

memory.

 Working with

incomplete knowledge.

 The capability to save

information on the

entire network.

 It is having fault

tolerance.

 Is even applied to expand the efficient

neighbourhood of pixels.

 Remembers each information through

time.

 Because of the feature to remember

preceding inputs, this model is

influential in time series prediction

only. This is called: LSTM.

 Automatically discovers the

significant features in the absence

of any human direction.

 Weight sharing.

Disadvantages

 Unexplained the

network behaviour.

 Hardware dependence.

 Determination of proper

network structure.

 Training an RNN is a tough matter.

 Gradient exploding and vanishing

challenges.

 It cannot treat sequences that are too

long if utilizing hyperbolic tangent or

rectified linear unit activation

functions.

 Is incapable of being spatially

never changing for the input data.

 Doesn’t encode the object

orientation and position.

 Lots of training data are needed.

ASA works (Moraes et al.)(Zahidi et

al.)

(Wahdan et al.)(Jerbi et al.) (Ombabi et al.) (Heikal et al.)

(Abu Kwaik et al.) (Alayba et al.)

(Abdullah et al.) (Al Omari et al.)

(Wahdan et al.) (Al-Azani and El-

Alf

(

Omara et al.

)

(

Dahou et al.

)

4 RESULTS AND DISCUSSION

In our literature review, we relied on the most famous

and valuable NN models: ANN, CNN, RNN, and

LSTM. We can utilize each of them in different ways.

We attempted to present a global view of them to help

you create a great choice for your issue:

 ANNs are less potent than RNNs and CNNs.

 RNNs contain less feature compatibility in

comparison to CNNs.

 CNNs are more valuable than RNNs and

ANNs.

 Temporal data relies on RNNs (also named

sequential data).

 CNNs are destined to use lower pre-processing

amounts.

 Unlike feed-forward NNs, RNNs can utilize

their internal memory for treating arbitrary

inputs sequences.

 RNNs can deal with input/output lengths

arbitrarily.

 CNNs pick up fixed sizes inputs and produce

outputs of fixed size.

In conclusion of this part, these NN models are all

advantageous in the ASA domain. However,

according to our deep evaluation, the literature, and

various significant works in the ASA field, we deduce

that: CNN and LSTM that is a type of RNN Neural

Network models have gained popularity by showing

impressive results in the field of ASA, compared to

other existing Neural Network models.

5 CONCLUSIONS

In this work, we have defined and explored the most

useful NN models in ASA. Moreover, we have

discussed in detail the characteristics of each model

ANN RNN CNN

Parameter Sharin

NoYesYes

Data T

e Tabular Data, Text Data Se

uence data Ima

e Data

Fixed Length input Yes No Yes

Spatial Relationship

No No No

lodin

and Vanishin

Gradient Yes Yes Yes

Recurrent Connections No Yes No

Arabic Sentiment Analysis based on Neural Network Models: Overview and Comparison

in the domain of ASA. Hence, the ASA research's

future opportunities contain applying these NN

Models in the ASA research field. This ASA future

work will be focusing on creating a technique to

perform ASA and will be relying on the techniques of

word embedding.

REFERENCES

Abdullah, Malak, et al. “SEDAT: Sentiment and Emotion

Detection in Arabic Text Using CNN-LSTM Deep

Learning.” Proceedings - 17th IEEE International

Conference on Machine Learning and Applications,

ICMLA 2018, Institute of Electrical and Electronics

Engineers Inc., 2019, pp. 835–40,

doi:10.1109/ICMLA.2018.00134.

Abu Kwaik, Kathrein, et al. “LSTM-CNN Deep Learning

Model for Sentiment Analysis of Dialectal Arabic.”

Communications in Computer and Information

Science, vol. 1108, Springer, 2019, pp. 108–21,

doi:10.1007/978-3-030-32959-4_8.

Al-Azani, Sadam, and El Sayed M. El-Alfy. “Hybrid Deep

Learning for Sentiment Polarity Determination of

Arabic Microblogs.” Lecture Notes in Computer

Science (Including Subseries Lecture Notes in Artificial

Intelligence and Lecture Notes in Bioinformatics), vol.

10635 LNCS, Springer Verlag, 2017, pp. 491–500,

doi:10.1007/978-3-319-70096-0_51.

Al Omari, Marwan, et al. “Hybrid CNNs-LSTM Deep

Analyzer for Arabic Opinion Mining.” 2019 6th

International Conference on Social Networks Analysis,

Management and Security, SNAMS 2019, Institute of

Electrical and Electronics Engineers Inc., 2019, pp.

364–68, doi:10.1109/SNAMS.2019.8931819.

Alayba, Abdulaziz M., et al. “A Combined CNN and LSTM

Model for Arabic Sentiment Analysis.” Lecture Notes

in Computer Science (Including Subseries Lecture

Notes in Artificial Intelligence and Lecture Notes in

Bioinformatics), vol. 11015 LNCS, Springer Verlag,

2018, pp. 179–91, doi:10.1007/978-3-319-99740-7_12.

Albayati, Abdulhakeem Q., et al. Arabic Sentiment Analysis

(ASA) Using Deep Learning Approach. 2020,

doi:10.31026/j.eng.2020.06.07.

Dahou, Abdelghani, et al. “Arabic Sentiment Classification

Using Convolutional Neural Network and Differential

Evolution Algorithm.” Computational Intelligence and

Neuroscience, vol. 2019, 2019,

doi:10.1155/2019/2537689.

Elfaik, Hanane, and El Habib Nfaoui. “Deep Bidirectional

LSTM Network Learning-Based Sentiment Analysis

for Arabic Text.” Journal of Intelligent Systems, vol.

30, no. 1, De Gruyter Open Ltd, Jan. 2021, pp. 395–

412, doi:10.1515/jisys-2020-0021.

Heikal, Maha, et al. “Sentiment Analysis of Arabic Tweets

Using Deep Learning.” Procedia Computer Science,

vol. 142, Elsevier B.V., 2018, pp. 114–22,

doi:10.1016/j.procs.2018.10.466.

Jerbi, Mohamed Amine, et al. “Sentiment Analysis of

Code-Switched Tunisian Dialect: Exploring RNN-

Based Techniques.” Communications in Computer and

Information Science, vol. 1108, Springer, 2019, pp.

122–31, doi:10.1007/978-3-030-32959-4_9.

Moraes, Rodrigo, et al. “Document-Level Sentiment

Classification: An Empirical Comparison between

SVM and ANN.” Expert Systems with Applications,

vol. 40, no. 2, Elsevier Ltd, Feb. 2013, pp. 621–33,

doi:10.1016/j.eswa.2012.07.059.

Omara, Eslam, et al. “Deep Convolutional Network for

Arabic Sentiment Analysis.” 2018 Proceedings of the

Japan-Africa Conference on Electronics,

Communications, and Computations, JAC-ECC 2018,

Institute of Electrical and Electronics Engineers Inc.,

2019, pp. 155–59, doi:10.1109/JEC-

ECC.2018.8679558.

Ombabi, Abubakr H., et al. “Deep Learning CNN–LSTM

Framework for Arabic Sentiment Analysis Using

Textual Information Shared in Social Networks.”

Social Network Analysis and Mining, vol. 10, no. 1,

Springer, Dec. 2020, p. 53, doi:10.1007/s13278-020-

00668-1.

Wahdan, Ahlam, et al. “A Systematic Review of Text

Classification Research Based on Deep Learning

Models in Arabic Language.” International Journal of

Electrical and Computer Engineering (IJECE), vol. 10,

no. 6, Dec. 2020, doi:10.11591/IJECE.V10I6.PP%P.

Zahidi, Youssra, et al. “A Powerful Comparison of Deep

Learning Frameworks for Arabic Sentiment Analysis.”

International Journal of Electrical and Computer

Engineering, vol. 11, no. 1, 2021, pp. 745–52,

doi:10.11591/ijece.v11i1.pp745-752.

BML 2021 - INTERNATIONAL CONFERENCE ON BIG DATA, MODELLING AND MACHINE LEARNING (BML’21)