A Survey of Social Emotion Prediction Methods

Abdullah Alsaedi

, Phillip Brooker

, Floriana Grasso

and Stuart Thomason

Department of Computer Science, University of Liverpool, U.K.

Department of Sociology, Social Policy and Criminology, University of Liverpool, U.K.

Keywords:

Social Emotion Prediction Methods, Social Emotion, Reader’s Emotion.

Abstract:

Emotions are an important factor that affects our communication. Considerable research has been done to

detect and classify emotion in text. However, most deal with emotion from the writer’s perspective. Social

emotion is the emotion of the reader when exposed to the text. With the increased use of social media, many

works are performed for social emotion prediction. In this paper, we attempt to provide a survey of social

emotion prediction methods. To the best of our knowledge, this is the ﬁrst work to survey the literature of

social emotion, review methods, and used techniques, compare the methods, and highlight their limitations.

1 INTRODUCTION

Emotion analysis in Natural Language Processing is

an active research area that seeks to automatically

detect emotion expressions in text, typically ground-

ing on pre-deﬁned, psychology based emotion models

(Alswaidan and Menai, 2020; Strapparava and Mihal-

cea, 2008). It extends research on sentiment analy-

sis, by providing an understanding of the text deeper

than the shallow classiﬁcation in positive, negative,

and neutral valence (Yadollahi et al., 2017).

The vast majority of work so far has focussed

on identifying the emotions of the writer of the text,

that is the expressed emotion as an indication of the

writer’s feelings or state of mind, often driven by

applications in market analysis and analysis of con-

sumer reviews. However, more recently a trend has

emerged which focusses on studying the emotion that

the text provokes in the readers, with many applica-

tions in social sciences and again marketing, for in-

stance tracking generated sympathetic responses to

potentially emotional events, or predicting the emo-

tional effect of advertisements. Research has fo-

cussed on the distinction among viewpoints, for in-

stance those of the reader’s as opposed to the writer’s

(Lin et al., 2008), or the perspective of the text itself

as a third point of view (Buechel and Hahn, 2017).

The term Social emotion has recently emerged

to indicate the aggregation of readers’ emotional re-

sponses, as collected by mining the social web for

comments, blogging, reactions etc. (Bao et al., 2012;

Rao et al., 2016), and represented either as a distribu-

tion over emotions that quantify readers’ rankings, or

as a single dominant emotion (Lin and Chen, 2008).

Social emotion prediction is the task of identify-

ing the social emotion that is likely to be provoked by

a text, where the text is usually objective, or at least

not emotionally loaded (Li et al., 2017a). The task is

challenging as readers’ emotions are likely to be af-

fected by their background and personal experiences,

which are unknown and not necessarily declared. As

an emergent research topic, the ﬁeld is also lack-

ing from the beneﬁt of numerous labelled datasets,

which are crucial for good quality supervised learn-

ing approaches (Alswaidan and Menai, 2020). This

is especially true for English

, while there are several

datasets for Chinese, thanks to news portals features

enabling readers to use various emotion labels to ex-

press their feelings.

In this paper we provide a survey of social emo-

tion prediction methods and systems, by classifying

surveyed work into three main categories on the basis

of the main strategy used for prediction.

2 SOCIAL EMOTION

PREDICTION METHODS

Existing social emotion prediction works can be clas-

siﬁed into three categories depending on whether they

A notable exception is EMOBANK (Buechel and

Hahn, 2017) a bi-perspectival large-scale 10K English

dataset annotated with writer’s as well as reader’s emotion.

Alsaedi, A., Brooker, P., Grasso, F. and Thomason, S.

A Survey of Social Emotion Prediction Methods.

DOI: 10.5220/0010546902230230

In Proceedings of the 10th International Conference on Data Science, Technology and Applications (DATA 2021), pages 223-230

ISBN: 978-989-758-521-0

223

are based on words, topics, or deep learning.

Word-based methods are based on the assumption

that all words, even neutral ones, can be associated

with a likelihood to provoke emotions (Strapparava

and Mihalcea, 2007). These can be effective, but per-

formance is affected negatively by sentiment ambigu-

ity of words in different contexts and noisy words.

Topic-based methods link emotions to broader

topics/events rather than single terms (Bao et al.,

2009), and use the machinery of topic modelling (Blei

and McAuliffe, 2007) to introduce an intermediate

emotion layer. The main criticism to these methods

is that they do not consider the order of words and

cannot encode the sequence information in text.

More recent methods are based on techniques

coming from deep learning, such as deep neural net-

works and word-embeddings, used to overcome, for

instance, the problem of sparsity associated with topic

based methods (Li et al., 2017b).

In remaining of this section, we will discuss social

emotion prediction methods in each of these three cat-

egories, looking at the methods, the techniques used,

the language, and the type of prediction, whether a

single, predominant emotion label, an array of multi-

ple emotion labels representing the readers’ emotion

distribution, or as a ranking problem. A summary of

all surveyed works against these features is provided

in Table 1.

2.1 Word-based Methods

Social emotion prediction works are considered to

have been ﬁrst appeared at the 2007 SemEval work-

shop, in its Task 14 on Affective Text (Strapparava

and Mihalcea, 2007). All participating systems to

SemEval-2007 Task 14 adopted a word-based method

for classifying social emotion into the six basic emo-

tions identiﬁed in (Ekman, 1992), and they were all

based on English, as such was the corpus provided

for the task. SWAT (Katz et al., 2007), one of the

top-performing systems, used a bag-of-words (BOW)

model trained to label news headlines with emotions,

by expanding on synonyms and antonyms using Ro-

get’s Thesaurus to improve the prediction accuracy.

The system scored each word and considered the av-

erage score for the headline. The authors used the

SemEval-2007 Affective Text corpus for training and

evaluation, in addition to annotating 1000 headlines

for training. UPAR7 (Chaumartin, 2007), developed

a rule-based classiﬁer that depends on syntactic pars-

ing and utilises resources from WordNet, SentiWord-

Net, and WordNet-Affect. The system was eval-

uated on SemEval-2007 Affective Text corpus and

achieved comparable results to SWAT. The authors

suggested the exploration of statistical models for fu-

ture work to improve the classiﬁcation recall. UA-

ZBSA (Kozareva et al., 2007) adopted a slightly dif-

ferent approach based on the principle that adjec-

tives with similar polarity appear together (Hatzivas-

siloglou and McKeown, 1997; Turney, 2002). The

system utilises web search engines (MyWay, AllWeb,

and Yahoo!) to measure the Mutual Information score

between the emotion and the BOW of headlines.

Another notable early work collected 17,743 news

articles from Yahoo!China and the reactions ex-

pressed by its users for training a supervised sys-

tem (Lin et al., 2007; Lin et al., 2008), exploiting

a functionality in Yahoo!China that allows users to

express their feelings after reading news articles by

using one of more reactions among: happy, angry,

sad, surprised, heartwarming, awesome, bored, and

useful. The authors extracted features like unigrams,

bigrams, metadata and also used a lexicon to obtain

emotion categories of words. Then, a Support Vector

Machine (SVM) was trained with these features us-

ing 12,079 articles. The model was tested with 5,664

articles. Their results show an accuracy of 87.9%

in predicting the predominant emotion in each arti-

cle. In another work, the authors tackled the different

problem of ranking the possible readers’ emotions re-

garding a certain text (Lin and Chen, 2008). They

used pairwise loss minimization and regression using

Support Vector Regression (SVR) to produce a list of

ranked emotions that related to the text. The authors

stated a high decrease of accuracy, which shows the

difﬁculty of the ranking task for emotion classes.

Parallel to this, work in Japan used distant su-

pervision to obtain a dataset for emotion-provoking

events (Tokuhisa et al., 2008). The authors searched

the web for lexical patterns that represent expressions

about emotional events, such as “I was disappointed

that” and linked the emotion of disappointment to the

rest of the sentence, for example “it suddenly started

raining”. This resulted in a corpus of 1.3 million

events in Japanese with their related emotions, which

was used to build a two-step k-Nearest Neighbour

(kNN) classiﬁer for sentiment polarity and emotion.

Other work developed a multi-label emotion clas-

siﬁer using RAkEL, an ensemble model for multi-

label classiﬁcation (Bhowmick et al., 2010). The clas-

siﬁer was trained for emotion classiﬁcation on news

sentences collected from news archives. The emo-

tion categories covered are disgust, fear, happiness

and sadness. The system used unigrams as a fea-

ture, in addition to subject polarity, verb and object

of the sentences, and semantic frame from FrameNet

to explore semantically related words. The Emotion-

Term model (Bao et al., 2009) was proposed to map

DATA 2021 - 10th International Conference on Data Science, Technology and Applications

224

Table 1: A Summary of Social Emotion Prediction Methods.

METHOD NAME and/or SOURCE PREDICTION DESCRIPTION LANGUAGE

Word based

SWAT (Katz et al., 2007) Multi-label BOW model + word expansion English

UPAR7 (Chaumartin, 2007) Multi-label Rule-based English

UA-ZBSA (Kozareva et al., 2007) Multi-label Knowledge-based + Statistics (PMI) English

(Lin et al., 2007; Lin et al., 2008) Single label Support Vector Machine classiﬁer Chinese

(Lin and Chen, 2008) Ranked list Pairwise loss minimization and regression Chinese

(Tokuhisa et al., 2008) Single label k-Nearest Neighbors classiﬁer Japanese

(Bhowmick et al., 2009) Multi-label Ensemble classiﬁer Chinese

Emotion-Term model (Bao et al., 2009) Single label Naive Bayes classiﬁer Chinese

(Tang and Chen, 2011) Single label Support Vector Machine classiﬁer + non-linguistic features Chinese

(Wang et al., 2011) Ranked list Rank-LR Chinese

(Ye et al., 2012) Multi-label Ensemble classiﬁer Chinese

(Liu et al., 2013) Single label Maximum entropy classiﬁer Chinese

(Lei et al., 2014) Single label Document selection, Part-of-speech tagging, Emotion lexicon Chinese

(Yao et al., 2014) Multi-label RAkEL + Knowledge-level resources to enlarge the features Chinese

(Vu et al., 2014) Multi-label Pattern expansion + Clustering English

(Li et al., 2016a) Single label Conditional Random Field + Support Vector Machine English

Textual Relevance (Ramya et al., 2019; Ramya et al., 2020) Single label Word frequency and nearest neighbour analysis English

Hidden Topic - Emotion Transition model (Tang et al., 2019) Multi-label Emotions-topic transition w. Markov chain + linguistic features. Chinese

Topic based

Emotion-Topic model (Bao et al., 2009; Bao et al., 2012) Multi-label Additional emotion layer on LDA Chinese

(Xu et al., 2013a) Multi-label LDA + multi-label k-Nearest Neighbors classiﬁer Chinese

(Xu et al., 2013b) Multi-label PLDA + multi-label classiﬁers Chinese

MSTM (Rao et al., 2014a) Multi-label Extension of supervised topic model (Blei & McAuliffe, 2009) Chinese

SLTM (Rao et al., 2014a) Multi-label Generates topics directly from social emotions Chinese

ATM (Rao et al., 2014b) Multi-label Jointly link topic to emotions and words Chinese

(Quan et al., 2015) Single label Latent Discriminative Models Chinese

ML-sETM (Zhang et al., 2015) Multi-label Supervised topic model Chinese

WME and TME (Rao et al., 2016) Multi-label Topic-model + Maximum Entropy English

CSTM (Rao, 2015) Multi-label Adaptive social emotion detection Chinese

WMCM (Li et al., 2016b) Multi-label Emotion concentration w. Topic model to identify word senses Chinese

UAM (Liang et al., 2018) Multi-label Supervised Topic Model + word-emotion dictionary English + Chinese

Deep Learning

(Li et al., 2017a) Single label Semantic analysis using word- embeddings Chinese

(Li et al., 2017b) Multi-label Semantically rich hybrid neural networks English + Chinese

(Krebs. et al., 2018) Multi-label Ensemble model w. of neural network + lexicon emotion miner English

(Gambino and Calvo, 2018; Gambino and Calvo, 2019) Multi-label MEKA + BOW + Doc2Vec Spanish

(Wang et al., 2019) Multi-label Syntactic and topical features in deep learning model English + Chinese

(Guan et al., 2019) Multi-label Hierarchical LSTM based model w. attention mechanism Chinese

TESAN (Wang and Wang, 2020) Multi-label Uniﬁed deep learning model from semantic + topical features English + Chinese

terms with emotion label using a Naive Bayes clas-

siﬁer. Another interesting work studied the emotion

generation on the Plurk microblogging platform from

both reader and writer perspectives (Tang and Chen,

2011). Plurk provides both bloggers and repliers the

ability to tag their post or comments with an emo-

tion. The work used textual features in addition to

non-linguistic features for the model, i.e.: social re-

lation, user behaviour, and relevance degree. The

ﬁndings were that adding non-linguistic features im-

proved the model performance, and the best accuracy

was achieved for textual features with social and be-

havioural features.

List-LR (Wang et al., 2011) is a method for social

emotion ranking, which, unlike previous approaches,

such as learning pairwise preference (Pair-LR), used

listwise preference, which minimises the listwise loss

to rank emotion labels. The method was evaluated

on a dataset of 3000 news articles collected from

the Chinese news website Sohu with encouraging

results. Another RAkEL classiﬁer for multi-label

reader’s emotion prediction in (Ye et al., 2012) per-

formed an evaluation on a corpus from Sina news,

with the best performance found with Chi-square and

document frequency as features.

Plurk was also used in (Tang and Chen, 2012) to

investigate the linguistic factors that affect emotion

transition between poster and repliers, using the plat-

form log relative frequency ratio. Although the sys-

tem adopted sentiment polarity (positive, negative and

neutral) in the analysis, the dataset was built by cate-

gorising 35 emoticons based on their name and pop-

ular usage. The extracted sentiment words were used

to build an SVM classiﬁer for emotion transition.

Yahoo!Kimo news was used in (Liu et al., 2013)

as a dataset for modelling news reader’s emotion and

comment on writer’s emotion. The dataset contains

news articles labeled with eight different emotions:

happy, sad, angry, meaningless, boring, heartwarm-

ing, worried, and useful, though useful and meaning-

less are not considered as emotions, and were dis-

carded in a separate experiment. Only articles with

dominant emotions were selected, and that emotion

was considered as the reader’s emotion. A semi-

supervised model was used to exploit unlabelled data

and improve accuracy. Maximum entropy (logistic re-

gression) was adopted as a classiﬁer with unigrams of

each article and comment as features.

Another popular microblogging service in China,

Weibo, was used in (Yang et al., 2013) to predict the

social emotions based on user’s interest in text and

images, in addition to their social inﬂuence. The work

evaluated the method on data crawled from Weibo,

and adopted sentiment polarity (positive and negative)

rather than ﬁne-grained emotions. Findings were that

user interests and social inﬂuence affect user emo-

tions in different ways, and they both improved the

prediction performance signiﬁcantly.

A Survey of Social Emotion Prediction Methods

225

Authors in (Yao et al., 2014) tackled an impor-

tant issue in the classiﬁcation of news headline, that is

data sparseness: headlines are short pieces of text and

therefore insufﬁcient for training a model, or work-

ing on the lexicon. The model used HowNet (Dong

and Dong, 2003) an online common-sense knowledge

base to expand the features, was evaluated on Sina

news, with performance comparable to BOW-based

approaches using both headlines and contents, which

is promising for other short text analysis, e.g. tweets.

More recent work include complex architectures

and experimentation, e.g. a system in (Lei et al.,

2014) consisting of three modules: document se-

lection, POS tagging, and social lexicon generation,

which was evaluated for Chinese and compared re-

sults from SemEval-2007. Or, work in (Vu et al.,

2014), an extension of (Tokuhisa et al., 2008) meant

to overcome its lack of measurement of the quality

of events, and of the aggregation of similar events.

The authors built a dictionary of emotion-provoking

events using both manual and automatic methods:

they categorised and ranked a list of events col-

lected by asking 30 participants to describe emotion-

provoking events, and enlarged the list automatically

from the web using the pattern of (Tokuhisa et al.,

2008), with results showing an increase in preci-

sion and recall. In a similar experiment (Li et al.,

2016a), in order to establish whether speciﬁc emo-

tional words are more important for classiﬁcation of

news than other words, crowdsourcing was used to

annotate a news corpus, then a Conditional Random

Field method extracted emotional words to be used

as features in the emotion classiﬁer. The classiﬁer

was also evaluated on Semeval-2007 AffectiveText

dataset, with a comparable performance to BOW ap-

proaches, and improving on other lexicon features. A

further increase in performance was achieved by com-

bining this method with BOW.

Finally, it is worth mentioning word-based meth-

ods working at document level. An approach based

on Text Relevance (Ramya et al., 2019; Ramya et al.,

2020) categorised documents into emotion classes us-

ing word frequency. The evaluation was conducted

on a translated corpus of news articles from Chinese

to English, and improved on the similar approach in

(Li et al., 2017a). An approach of emotion detection

in sentence-level as well as document-level was used

in (Tang et al., 2019) working on an assumption that

all words in the same sentence share the same emo-

tion and topic, and modelling emotion and topic tran-

sition between sentences as a Markov chain. The ex-

periment performed on two datasets showed that the

method outperforms state-of-the-art methods on both

sentence-level classiﬁcation and document-level clas-

siﬁcation.

2.2 Topic-based Methods

The ﬁrst topic-based emotion prediction method (Bao

et al., 2009; Bao et al., 2012) started from the prin-

ciple that emotion in text is more correlated to the

topic than the terms. The model was built upon La-

tent Dirichlet Allocation (LDA) (Blei et al., 2003) a

popular topic modelling technique used in informa-

tion retrieval, and modiﬁes LDA to add a layer that

considers the emotion. The model improved the pre-

diction of social emotion signiﬁcantly compared to

the emotion-term model.

Expanding on this, work in (Xu et al., 2013a) used

topic modelling as a dimension reduction method

rather than BOW, in addition to multi-label kNN, for

a multi-label emotion classiﬁcation. To implement

topic modelling, a weighted LDA was used, which

expanded LDA to discover the semantic association

between topics and emotions. The approach was eval-

uated on a dataset from Sina containing news labelled

with readers emotions. In another work, (Xu et al.,

2013b) authors used the output of a Partitioned LDA

model as a feature for the multi-label classiﬁer to pre-

dict the reader’s emotion, again on a corpus from Sina

news for evaluation, and found that the system per-

forms better than BOW, LDA and Weighted LDA.

Multi-label Supervised Topic Model (MSTM) and

Sentiment Latent Topic Model (SLTM) (Rao et al.,

2014a) are two emotion-topic models for social emo-

tion prediction which also introduced an emotion

layer to LDA. They were evaluated on 4570 news

articles collected again from Sina news, resulting

more accurate and stabler than the common emotion-

term model. The Affective Topic Model (ATM) by

the same authors (Rao et al., 2014b) implemented a

multi-label model for detecting social emotion toward

certain topics.

Work in (Bao et al., 2009) also introduced an in-

termediate layer to the topic model, which enabled the

extraction of different meanings for the same word.

The model was able to distinguish between topics as-

sociated with one emotion, and merged topics linked

to several emotions. The evaluation was performed

on the same dataset from Sina news, and the model

signiﬁcantly outperformed the multi-label supervised

topic model, the emotion-topic model, SWAT, and the

emotion-term model, and achieved comparable accu-

racy to the sentiment latent topic model. The model

effectively discovered meaningful topics that were re-

lated to emotion, and their words can be used for

emotion-based information retrieval.

ML-sTEM (Zhang et al., 2015) is a multi-label su-

DATA 2021 - 10th International Conference on Data Science, Technology and Applications

226

pervised emotion-topic model which learns through

observing the associations of emotion labels linked to

the documents, and then uses a modiﬁed LDA with an

additional emotion layer to produce the emotion-topic

output. The evaluation was performed on a dataset

from Sina news collecting, for each of the eight emo-

tions that the platform allows readers to use for la-

belling, the most-viewed news. The authors reckon

this is the ﬁrst multi-label emotion tagging model for

news document from the reader’s perspective; how-

ever, the real novelty of this work is the way that it

used a supervised topic model based on LDA.

Work in (Quan et al., 2015) proposed a latent dis-

criminative model by introducing intermediate hidden

variables, on the assumption that social emotions are

dependent on each other: for example, it is likely for

people who are angry to also be sad. A similar prin-

ciple was used in (Li et al., 2016b) to tackle the prob-

lem of unstable performance of previous models, due

to noisy training documents, with a multi-label clas-

siﬁcation model (WMCM) based on ’emotion con-

centration’, which exploits the topic models to iden-

tify word emotional meanings in different documents.

The model was evaluated on two datasets, SemEval-

2007 news headlines and Sina news, and the results

showed the effectiveness of the model over the base-

lines of many systems we covered in the previous sec-

tions, such as SWAT, ET, ETM, MSTM and SLTM.

The Context Sentiment Topic Model (CSTM)

(Rao et al., 2016) differentiated between context-

dependent and context-independent topics. The

model provided an adaptive classiﬁcation by consid-

ering the context during the topic modelling stage. In

the same work, the authors proposed TME, a topic-

level maximum entropy model for social emotion pre-

diction, which combines output from unsupervised

topic model with a Maximum Entropy classiﬁer to

mitigate the problem of data spareness when learn-

ing from short-text. The model was evaluated on a

collection of short documents obtained from several

sources and manually labeled with reader’s emotion.

The experiments showed that the model successfully

addresses the problem of overﬁtting on sparse words.

Finally, the Universal Affective Model (UAM)

(Liang et al., 2018) classiﬁes short-text in social me-

dia into ﬁne-grained social emotions. The model

adopts a combination of a supervised topic model

with a word-emotion dictionary to solve the problem

of data sparsity when dealing with short-text. The

model consists of two sub-models: topic-level and

term-level. The evaluation was performed on real-

world datasets and validated the effectiveness and the

improved accuracy that the model achieved.

2.3 Deep Learning-based Methods

The ﬁrst work that used a deep learning model for

social emotion prediction was probably the one de-

scribed in (Li et al., 2017b), which integrated seman-

tic knowledge with a hybrid neural network (HNN)

to produce a semantically rich and effective model.

Unsupervised learning models, i.e., Bi-Term Topic

Model (BTM), Replicated Softmax Machine (RSM),

and Word2vec, were used to extract semantic features

in order to use them in the proposed HNN model.

Then, the model was trained on a labeled dataset to

predict the reader’s emotion. The evaluation was per-

formed on three datasets, SemEval-2007, Sina news,

and ISEAR (Scherer and Wallbott, 1994) (the Inter-

national Survey on Emotion Antecedents and Reac-

tions). It was found that the proposed model outper-

formed state-of-the-art models for social emotion pre-

diction, and it was suggested that semantically rich

models better predict different emotion contexts and

improve the performance of the predictive model.

The approach in (Li et al., 2017a) used a social

opinion model that measures similarity among news

documents. A social opinion network was built based

on the Wikipedia pre-trained word-embedding. The

network consisted of Word2vec vectors where nodes

represented the social opinions, and the edges rep-

resent their relationships. The semantic similarity

was calculated by the distance between nodes and

neighbour analysis was used for the prediction. As

a result, a strong correlation was found between the

news structure and the emotion associated. The model

showed a more stable and accurate performance than

most of the other models.

Twitter was used in (Gambino and Calvo, 2018;

Gambino and Calvo, 2019), a work which investi-

gated the social emotion for Twitter news articles

using annotated headlines and replies from popu-

lar Spanish news publishers. A multi-target clas-

siﬁcation model was developed for emotion distri-

bution and their intensity, which assumed that the

reader can express more than one emotion for the

same article. Different features were used with the

model, such as BOW and word embeddings using

Doc2Vec. Also, the implementation was performed

using MEKA, which is an extension for the WEKA

tool that provides methods for the multi-target classi-

ﬁcation. The classiﬁcation results were promising in

spite of the bias toward few emotions due to the highly

imbalanced training data of the emotion distribution.

Facebook was instead used in (Krebs. et al., 2018)

where a dataset was built of Facebook posts with

their reaction distributions, which was used to train

an ensemble model for the prediction of the then ﬁve

A Survey of Social Emotion Prediction Methods

227

Facebook reactions. The model combined a neural

network trained on GloVe vectors, with an emotion

miner that uses EmoLex for both posts and comments,

also analysing the effect of pre-processing on the per-

formance. It was found that combining baseline mod-

els of emotion analysis can improve the performance.

A hierarchical Long Short-Term Memory

(LSTM)-based model was proposed in (Guan et al.,

2019) with attention mechanism for social emotion

prediction, as word-level models suffer from the sen-

timent ambiguity and noisy words problems. Also,

the topic-level approaches are not able to encode

the order of words and sentences. The proposed

method attempts to capture the semantic of long-text

by employing the word-embedding on three different

levels: word, sentence, and document. Also, the

attention mechanism enables to build an emotion

lexicon that could be used in the future. Based

on the evaluation conducted on crawled datasets

from Sina news, the proposed model achieved better

performance than all baselines of word-level and

topic-level.

The model in (Wang et al., 2019) considered the

semantic and topical information and integrated syn-

tactic features in sentences with topical information

in each document to generate a document represen-

tation. In the ﬁrst step, a Deep Ensemble Learning

Model (DERN) is used to encode syntactic sentences

into vectors, then a Gated Recurrent Unit (GRU) con-

verts those vectors into a document vector. Secondly,

an multilayer perceptron (MLP) is used to convert the

output of LDA into a topic vector. Finally, a gate layer

generates the ﬁnal document representation from both

inputs. Experiments on two datasets from Sina news

and ISEAR showed that the proposed model outper-

forms the state-of-the-art models in terms of Micro F1

score and Average Precision.

A similar model was proposed in another work

by the same authors, TESAN, (Wang and Wang,

2020) which also utilises semantic and topical fea-

tures. TESAN jointly learns features in a uniﬁed

deep learning structure. The model consists of a neu-

ral topic model for topic modelling, a topic-enhanced

self-attention mechanism to generate the document

vector from the semantic and topical features, and a

fusion gate to integrate both inputs and generate the

ﬁnal document representation. TESAN was evalu-

ated again on Sina news and ISEAR, in addition to

SemEval-2014 dataset for news headlines. The exper-

iments showed that TESAN signiﬁcantly improved

the state-of-the-art results. Also, the model was ef-

ﬁcient and able to generate high-quality topics.

3 DISCUSSION

The survey of social emotion prediction methods

clearly show a chronological order of phases of the

research, with early works focussing on word-based

methods, then moving to topic-based methods, and,

recently, a majority of works using the deep learning-

based methods. This indicates both a shift in tech-

nology and a change in the availability and structure

of datasets, with an ever increasing range of possibil-

ity for readers to self identify the emotions that they

feel while reading directly at the time of reading. Not

surprisingly, the type of prediction has also changed,

with early systems focussing on searching the domi-

nant emotion associated with a piece of text, up until

the possibilities to provide multi-labels and ranked list

of labels. Therefore, whilst early word-based mod-

els had to rely heavily and almost exclusively on es-

tablished lexicon such as EmoLex and WordNet, and

use word frequency and linguistic features only, topic-

based systems can work on intermediate emotion lay-

ers such as LDA, and deep learning-based methods

beneﬁt from the semantically rich representation that

word-embeddings provide. Also, neural network ar-

chitectures such as CNN, LSTM, and GRU, in addi-

tion to the attention mechanism.

In terms of dataset availability and their impact on

the natural language used by these methods, clearly

the relatively recent practice by China based news

portals to allow users to self report a wider range of

emotions (e.g. curiosity, amusement, surprise) rather

than the limited set of reactions used in Western so-

cial media, has given a tremendous push to research

in China, and highlighted the need for more datasets

on other languages.

One aspect which is perhaps still missing is a

greater attention to the context in which the piece

of text and the reactions to it are placed, for in-

stance, which social media page features the news,

whether the news is shared by speciﬁc users or pages

or groups, or even a more comparative study of how

different platforms, as well as different set of users

social media connections, make a difference.

4 CONCLUSIONS

In this survey paper, we reviewed the available meth-

ods for social emotion prediction and presented them

in chronological order according to three categories:

word-based, topic-based, and deep learning-based.

Also, we discussed the techniques used in these meth-

ods and the different type of predictions. While social

emotion prediction is a challenging task, progress in

DATA 2021 - 10th International Conference on Data Science, Technology and Applications

228

prediction performance is promising, but there is a

pressing need for reliable labeled datasets for social

emotion in order to train supervised models.

REFERENCES

Alswaidan, N. and Menai, M. E. B. (2020). A survey of

state-of-the-art approaches for emotion recognition in

text. Knowledge and Information Systems, 62:2937–

2987.

Bao, S., Xu, S., Zhang, L., Yan, R., Su, Z., Han, D., and

Yu, Y. (2009). Joint emotion-topic modeling for so-

cial affective text mining. In Ninth IEEE International

Conference on Data Mining, pages 699–704. IEEE.

Bao, S., Xu, S., Zhang, L., Yan, R., Su, Z., Han, D., and

Yu, Y. (2012). Mining social emotions from affective

text. IEEE transactions on knowledge and data engi-

neering, 24(9):1658–1670.

Bhowmick, P. K., Basu, A., and Mitra, P. (2009). Reader

perspective emotion analysis in text through ensemble

based multi-label classiﬁcation framework. Computer

and Information Science, 2(4):64–74.

Bhowmick, P. K., Basu, A., and Mitra, P. (2010). Clas-

sifying emotion in news sentences: When machine

classiﬁcation meets human classiﬁcation. Interna-

tional Journal on Computer Science and Engineering,

2(1):98–108.

Blei, D. M. and McAuliffe, J. D. (2007). Supervised topic

models. In 20th International Conference on Neu-

ral Information Processing Systems, NIPS’07, pages

121–128.

Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent

dirichlet allocation. the Journal of machine Learning

research, 3:993–1022.

Buechel, S. and Hahn, U. (2017). Readers vs. writers vs.

texts: Coping with different perspectives of text un-

derstanding in emotion annotation. In 11th Linguistic

Annotation Workshop, pages 1–12.

Chaumartin, F.-R. (2007). Upar7: a knowledge-based sys-

tem for headline sentiment tagging. In 4th Interna-

tional Workshop on Semantic Evaluations, pages 422–

425.

Dong, Z. and Dong, Q. (2003). Hownet-a hybrid language

and knowledge resource. In International Conference

on Natural Language Processing and Knowledge En-

gineering, 2003. Proceedings. 2003, pages 820–824.

IEEE.

Ekman, P. (1992). An argument for basic emotions. Cogni-

tion & Emotion, 6(3-4):169–200.

Gambino, O. J. and Calvo, H. (2018). Modeling distribution

of emotional reactions in social media using a multi-

target strategy. Journal of Intelligent & Fuzzy Systems,

34(5):2837–2847.

Gambino, O. J. and Calvo, H. (2019). Predicting emotional

reactions to news articles in social networks. Com-

puter Speech & Language, 58:280–303.

Guan, X., Peng, Q., Li, X., and Zhu, Z. (2019). Social emo-

tion prediction with attention-based hierarchical neu-

ral network. In 2019 IEEE 4th Advanced Information

Technology, Electronic and Automation Control Con-

ference (IAEAC), volume 1, pages 1001–1005. IEEE.

Hatzivassiloglou, V. and McKeown, K. (1997). Predict-

ing the semantic orientation of adjectives. In 35th

Annual Meeting of the Association for Computational

Linguistics, pages 174–181.

Katz, P., Singleton, M., and Wicentowski, R. (2007). Swat-

mp: the semeval-2007 systems for task 5 and task 14.

In Fourth International Workshop on Semantic Evalu-

ations (SemEval-2007), pages 308–313.

Kozareva, Z., Navarro, B., V

azquez, S., and Montoyo, A.

(2007). Ua-zbsa: a headline emotion classiﬁcation

through web information. In Fourth International

Workshop on Semantic Evaluations (SemEval-2007),

pages 334–337.

Krebs., F., Lubascher., B., Moers., T., Schaap., P., and

Spanakis., G. (2018). Social emotion mining tech-

niques for facebook posts reaction prediction. In 10th

International Conference on Agents and Artiﬁcial In-

telligence, pages 211–220. INSTICC, SciTePress.

Lei, J., Rao, Y., Li, Q., Quan, X., and Wenyin, L. (2014).

Towards building a social emotion detection system

for online news. Future Generation Computer Sys-

tems, 37:438–448.

Li, M., Wang, D., Lu, Q., and Long, Y. (2016a). Event based

emotion classiﬁcation for news articles. In 30th Pa-

ciﬁc Asia Conference on Language, Information and

Computation: Oral Papers, pages 153–162.

Li, X., Peng, Q., Sun, Z., Chai, L., and Wang, Y. (2017a).

Predicting social emotions from readers’ perspec-

tive. IEEE Transactions on Affective Computing,

10(2):255–264.

Li, X., Rao, Y., Xie, H., Lau, R. Y. K., Yin, J., and Wang,

F. L. (2017b). Bootstrapping social emotion classiﬁ-

cation with semantically rich hybrid neural networks.

IEEE Transactions on Affective Computing, 8(4):428–

442.

Li, X., Xie, H., Rao, Y., Chen, Y., Liu, X., Huang, H., and

Wang, F. L. (2016b). Weighted multi-label classiﬁca-

tion model for sentiment analysis of online news. In

2016 International Conference on Big Data and Smart

Computing (BigComp), pages 215–222. IEEE.

Liang, W., Xie, H., Rao, Y., Lau, R. Y., and Wang, F. L.

(2018). Universal affective model for readers’ emo-

tion classiﬁcation over short texts. Expert Systems

with Applications, 114:322–333.

Lin, K. H.-Y. and Chen, H.-H. (2008). Ranking reader emo-

tions using pairwise loss minimization and emotional

distribution regression. In 2008 Conference on Empir-

ical Methods in Natural Language Processing, pages

136–144.

Lin, K. H.-Y., Yang, C., and Chen, H.-H. (2007). What

emotions do news articles trigger in their readers? In

30th annual international ACM SIGIR conference on

Research and development in information retrieval,

pages 733–734.

Lin, K. H.-Y., Yang, C., and Chen, H.-H. (2008). Emotion

classiﬁcation of online news articles from the reader’s

perspective. In 2008 IEEE/WIC/ACM International

A Survey of Social Emotion Prediction Methods

229

Conference on Web Intelligence and Intelligent Agent

Technology, volume 1, pages 220–226. IEEE.

Liu, H., Li, S., Zhou, G., Huang, C.-R., and Li, P. (2013).

Joint modeling of news reader’s and comment writer’s

emotions. In 51st Annual Meeting of the Association

for Computational Linguistics, pages 511–515.

Quan, X., Wang, Q., Zhang, Y., Si, L., and Wenyin,

L. (2015). Latent discriminative models for so-

cial emotion detection with emotional dependency.

ACM Transactions on Information Systems (TOIS),

34(1):1–19.

Ramya, R., Kishore, N., Sejal, D., Venugopal, K., Iyen-

gar, S., and Patnaik, L. (2019). Predicting social

emotions based on textual relevance for news docu-

ments. In 2019 IEEE International WIE Conference

on Electrical and Computer Engineering (WIECON-

ECE), pages 1–4. IEEE.

Ramya, R., Madhura, K., Sejal Venugopal, K., Iyengar, S.,

and Patnaik, L. (2020). EPNDR: Emotion prediction

for news documents based on readers’ perspectives.

International Journal of Scientiﬁc & Technology Re-

search, 9:531–539.

Rao, Y. (2015). Contextual sentiment topic model for adap-

tive social emotion classiﬁcation. IEEE Intelligent

Systems, 31(1):41–47.

Rao, Y., Li, Q., Mao, X., and Wenyin, L. (2014a). Sentiment

topic models for social emotion mining. Information

Sciences, 266:90–100.

Rao, Y., Li, Q., Wenyin, L., Wu, Q., and Quan, X. (2014b).

Affective topic model for social emotion detection.

Neural Networks, 58:29–37.

Rao, Y., Xie, H., Li, J., Jin, F., Wang, F. L., and Li, Q.

(2016). Social emotion classiﬁcation of short text via

topic-level maximum entropy model. Information &

Management, 53(8):978–986.

Scherer, K. R. and Wallbott, H. G. (1994). Evidence for

universality and cultural variation of differential emo-

tion response patterning. Journal of personality and

social psychology, 66(2):310.

Strapparava, C. and Mihalcea, R. (2007). Semeval-2007

task 14: Affective text. In Fourth International Work-

shop on Semantic Evaluations (SemEval-2007), pages

70–74.

Strapparava, C. and Mihalcea, R. (2008). Learning to iden-

tify emotions in text. In Proceedings of the 2008 ACM

symposium on Applied computing, pages 1556–1560.

Tang, D., Zhang, Z., He, Y., Lin, C., and Zhou, D. (2019).

Hidden topic–emotion transition model for multi-level

social emotion detection. Knowledge-Based Systems,

164:426–435.

Tang, Y.-j. and Chen, H.-H. (2011). Emotion modeling from

writer/reader perspectives using a microblog dataset.

In Workshop on Sentiment Analysis where AI meets

Psychology (SAAIP 2011), pages 11–19.

Tang, Y.-j. and Chen, H.-H. (2012). Mining senti-

ment words from microblogs for predicting writer-

reader emotion transition. In Eighth International

Conference on Language Resources and Evaluation

(LREC’12), pages 1226–1229.

Tokuhisa, R., Inui, K., and Matsumoto, Y. (2008). Emotion

classiﬁcation using massive examples extracted from

the web. In 22nd International Conference on Com-

putational Linguistics (Coling 2008), pages 881–888.

Turney, P. D. (2002). Thumbs up or thumbs down? seman-

tic orientation applied to unsupervised classiﬁcation

of reviews. In 40th Annual Meeting on Association

for Computational Linguistics, pages 417–424.

Vu, H. T., Neubig, G., Sakti, S., Toda, T., and Nakamura, S.

(2014). Acquiring a dictionary of emotion-provoking

events. In 14th Conference of the European Chapter of

the Association for Computational Linguistics, pages

128–132.

Wang, C. and Wang, B. (2020). An end-to-end topic-

enhanced self-attention network for social emotion

classiﬁcation. In Web Conference 2020, pages 2210–

2219.

Wang, C., Wang, B., Xiang, W., and Xu, M. (2019). Encod-

ing syntactic dependency and topical information for

social emotion classiﬁcation. In 42nd International

ACM SIGIR Conference on Research and Develop-

ment in Information Retrieval, pages 881–884.

Wang, Q., Wu, O., Hu, W., Yang, J., and Li, W. (2011).

Ranking social emotions by learning listwise prefer-

ence. In The First Asian Conference on Pattern Recog-

nition, pages 164–168. IEEE.

Xu, R., Ye, L., and Xu, J. (2013a). Reader’s emotion pre-

diction based on weighted latent dirichlet allocation

and multi-label k-nearest neighbor model. Journal of

Computational Information Systems, 9(6):2209–2216.

Xu, R., Zou, C., and Xu, J. (2013b). Reader’s emotion pre-

diction based on partitioned latent dirichlet allocation

model. In International Conference on Internet Com-

puting and Big Data, pages 133–137.

Yadollahi, A., Shahraki, A. G., and Zaiane, O. R. (2017).

Current state of text sentiment analysis from opinion

to emotion mining. ACM Computing Surveys (CSUR),

50(2):1–33.

Yang, Y., Cui, P., Zhu, W., and Yang, S. (2013). User inter-

est and social inﬂuence based emotion prediction for

individuals. In 21st ACM International Conference on

Multimedia, MM ’13, pages 785–788.

Yao, Y., Xu, R., Lu, Q., Liu, B., Xu, J., Zou, C., Yuan, L.,

Wang, S., Yao, L., and He, Z. (2014). Reader emotion

prediction using concept and concept sequence fea-

tures in news headlines. In International Conference

on Intelligent Text Processing and Computational Lin-

guistics, pages 73–84. Springer.

Ye, L., Xu, R.-F., and Xu, J. (2012). Emotion predic-

tion of news articles from reader’s perspective based

on multi-label classiﬁcation. In 2012 international

conference on machine learning and cybernetics, vol-

ume 5, pages 2019–2024. IEEE.

Zhang, Y., Su, L., Yang, Z., Zhao, X., and Yuan, X. (2015).

Multi-label emotion tagging for online news by super-

vised topic model. In Asia-Paciﬁc Web Conference,

pages 67–79. Springer.

DATA 2021 - 10th International Conference on Data Science, Technology and Applications

230