Sarcasm Detection Method to Improve Review Analysis

Shota Suzuki, Ryohei Orihara, Yuichi Sei, Yasuyuki Tahara and Akihiko Ohsuga

Graduate School of Information Systems, University of Electro-Communications, Tokyo, Japan

Keywords:

Natural Languages Processing, Opinion Mining, Sarcasm, Sentiment, Online Review.

Abstract:

Currently, classifying sarcastic sentences into positive and negative sentiments has been a difﬁcult problem

and an important task. The sarcastic sentences could indicate negative meaning by using positive expressions,

or positive meaning by using negative expressions.Sarcasm is a special kind of sentiment that comprise of

words which mean the opposite of what you really want to say, especially in order to insult or wit someone, to

show irritation, or to be funny.Therefore, determining sarcasm is an important task in order to correctly classify

the sentence. In this paper, we propose an approach to detect sarcasm.First, we apply dependency parsing to

amazon review data. After that, we classify phrases in the sentence into the proposed phrase based on the

sequence of part-of-speech as proposed by Bharti et al. After being classiﬁed into either one of the phrase

types, it is determined whether each phrase is positive or negative.If the emotions of the situation phrases and

the sentiment phrases are different, the sentence is determined to be a “sarcasm”. Using the above method ,

the experimental result shows the effectiveness of our method as compared with the the existing research.

1 INTRODUCTION

With the spread of web services, everyone can post a

review of a product on the Web and it is readable from

anyone. The user review published on the web is a

valuable information resource to which another user

refers when she/he is making decision on a purchase.

However, selecting the useful information manually

to the user from the vast amount of review requires

much effort. Therefore, the study of current reputa-

tion analysis to automate the analysis of the text have

been conducted. In general, reputation analysis uses

polarity detection techniques that classify statements

into positive and negative ones. However, the current

polarity detection techniques only consider the emo-

tion of each word of the sentences. Thus it is difﬁ-

cult to correctly judge the polarity of expressions such

as sarcastic sentences that does not directly express

their intention. Sarcasm is a special type of sentiment

which plays a role as an interfering factor that can ﬂip

the polarity of the given text. Given an example of

tweet: “Nothing I love more than a crowded library

with no seats #sarcasm”. Although this example uses

a word “love” to express the positive sentiment, the

tweet as a whole expresses negative sentiment toward

the library. Figure 1 shows a typical procedure of cur-

rent polarity detection techniques. In the left balloon

of Figure 1, although the text should be classiﬁed as

negative, it is mistakenly determined as positive by

With the spread of web services, everyone can post a

review of a product on the Web and it is readable from

anyone. The user review published on the web is a

valuable information resource to which another user

refers when she/he is making decision on a purchase.

However, selecting the useful information manually

to the user from the vast amount of review requires

much effort. Therefore, the study of current reputa-

tion analysis to automate the analysis of the text have

been conducted. In general, reputation analysis uses

arity detection techniques that classify statements

into positive and negative ones. However, the current

arity detection techniques only consider the emo-

tion of each word of the sentences. Thus it is difﬁ-

cult to correctly judge the polarity of expressions such

as sarcastic sentences that does not directly express

ƌĞǀŝĞǁƐĞƚ

ϭƌĞǀŝĞǁ

ŶĞƵƚƌĂů

ƉŽƐŝƚŝǀĞ

ŶĞŐĂƚŝǀĞ

㼟㼑㼚㼠㼕㼙㼑㼚㼠㻌㼟㼡㼙㼙㼍㼞㼕㼦㼍㼠㼕㼛㼚

tŽƌĚůĞǀĞůŽĨƚŚĞ

ƉŽůĂƌŝƚǇũƵĚŐĞŵĞŶƚĚŝĐƚŝŽŶĂƌǇ

ŽƵůĚŝŶĐůƵĚĞƐĂƌĐĂƐƚŝĐ

ƐƚĂƚĞŵĞŶƚƐƚŚĂƚƐŚŽƵůĚďĞ

ĐůĂƐƐŝĨŝĞĚĂƐŶĞŐĂƚŝǀĞ͘

Ğǆ䠗/ĂƉƉƌĞĐŝĂƚĞƚŚŝƐŐĂŵĞ

ďǇŵĂŬŝŶŐŵĞƐǁĞĂƌŶŽƚƚŽ

ďƵǇƚŚŝƐŐĂŵĞĂŐĂŝŶ͘

ŽƵůĚŝŶĐůƵĚĞƐĂƌĐĂƐƚŝĐ

ƐƚĂƚĞŵĞŶƚƐƚŚĂƚƐŚŽƵůĚďĞ

ĐůĂƐƐŝĨŝĞĚĂƐƉŽƐŝƚŝǀĞ͘

Ğǆ䠗dŚŝƐŐĂŵĞŝƐƚŽŽŵƵĐŚ

ĂƚƚƌĂĐƚŝǀĞ͕ƐŽŝƚƚƌŽƵďůĞƐŵĞ͘

/ĂŵŐŽŝŶŐƚŽďĞĂũƵŶŬŝĞ͘

Figure 1: Typical procedure of current polarity detection.

an existing polarity detector. Similarly, the detector

would determine as negative by mistake the text that

needs to be classiﬁed positively, as in the right bal-

loon of the Figure 1. As mentioned above, sarcastic

texts affect the classiﬁcation accuracy of the polarity

detector.

In this paper, we propose a sarcasm-emotion de-

tection method for improvement of accuracy of emo-

tion determination intended for reviews. Our method

is based on Bharti’s method and determines whether

Suzuki S., Orihara R., Sei Y., Tahara Y. and Ohsuga A.

Sarcasm Detection Method to Improve Review Analysis.

DOI: 10.5220/0006192805190526

In Proceedings of the 9th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2017), pages 519-526

ISBN: 978-989-758-220-2

519

a review in Japanese is a sarcasm. Since in Japanese

ambiguous expressions appear more frequently than

in English, we propose a method that can cope with

ambiguous expressions. Bharti et al. dealt with sar-

casm detection using Twitter. Bharti et al. proposed

phrases called situation phrases and sentiment phrases

to detect sarcasm. In this paper, situation phrases

and sentiment phrases are called proposed phrases

in total. Using the phrase above, Bharti et al. pro-

posed a PBLGA(a parsing-based lexical generation

algorithm) method for determining the sarcasm. For

more information about Bharti’s method refer to 2.1.

Bharti’s method ﬁrst parses the tweets. Then it ana-

lyzes the clauses to detect proposed phrases in them.

If a phrase is detected as a situation phrase or a sen-

timent phrase, it phrases are stored the instance of

positive/negative sentiment/situation phrases. If pro-

posed phrases were determined a negative situation

phrase and a positive sentiment phrase, or a positive

situation phrase and a negative sentiment phrase, the

tweet is determined as “sarcasm”. In Bharti’s method,

the instances of positive/negative sentiment/situation

phrases are stored in order to ﬁnd sarcastic tweets

in a new corpus by means of simple string match

only. On the basis of the Bharti’s method, we pro-

pose a sarcasm-emotion detection method based on

polarities of words. We determine the emotion of

the proposed phrases based on the number of words

with the emotion included in the phrases just as the

Bharti’s method. From the results of counting, our

approach determines the emotions of the proposed

phrases. Further, if the numbers of P and N are the

same, we will determine the emotion of the proposed

phrases using the tf-idf scores. After judging the emo-

tion of the phrase, we determine the sarcastic state-

ment. Speciﬁcally, if the emotions of the proposed

phrase are different, we determine “sarcasm”. The

main difference between this research and Santoth et

al. research are shown in as follows.

• We propose sarcasm-emotional detection method

based on emotion of phrase.

• In order to improve the determinetion accuracy of

phrase emotion, feelings of phrases are selected

manually.

We conducted the evaluation experiment in order to

demonstrate the usefulness the proposed method. The

proposed method is implemented to analyze review

texts of computer games. When applied to 140 re-

views of a game “MSG EXTREME VS-FORCE”, the

proposed method could determine sarcastic sentences

with the precision of 0.79 and the recall of 0.56.

The structure of this paper is as follows. Section

2 surveys the existing research of sarcasm detection.

In Section 3, as an assumption of our research, we

discuss the concept of sarcastic sentence. Section 4

describes our proposed method. In Section 5, we eval-

uate how our proposed method can detect sarcasms in

comparison with the existing sarcasm detection meth-

ods. Section 6 discusses the results of the proposed

method and Section 7 concludes this paper.

2 RELATED WORK

In this section, we survey the existing research of sar-

casm detection.

2.1 Study in English Text in SNS Sites

Bharti et al. determined sarcasm in Twitter based

on two of their proposed method, namely, PBLGA

and IWS algorithm. In order to determine sarcasm in

Twitter data, PBLGA, or parsing based lexical gen-

eration algorithm, either one of the following combi-

nations must present in a tweet: (a) contradiction of

negative sentiment and positive situation (b) contra-

diction of positive sentiment and negative situation.

Bharti’s method ﬁrst parses the tweets. Then it ana-

lyzes the clauses to detect proposed phrases in them.

If a phrase is detected as a situation phrase or a senti-

ment phrase, its sentiment score is determined based

on the number of positive and negative words in it. If

the sentiment score is positive, the proposed phrase

becomes positive, and if the sentiment score is nega-

tive, the proposed phrase becomes negative. If a tweet

contains a negative situation phrase and a positive

sentiment phrase, or a positive situation phrase and a

negative sentiment phrase, the tweet is determined as

sarcastic. In Bharti’s method, the instances of pos-

itive/negative sentiment/situation phrases are stored

in order to ﬁnd sarcastic tweets in a new corpus by

means of simple string match only. IWS algorithm

is an algorithm to determine the sarcasm in view of

the interjection representation. If interjection appears

in the beginning of the tweet and intensiﬁer appears

other than in the beginning, than a tweet has high

probability to classify as sarcastic.

As a preliminary of the experiment, dealing 50000

tweets with hash tags as training data in order to select

a phrase that becomes the emotion of the proposed

phrase. They constructed proposed phrases using the

training data. The test experiment was applying the

two algorithms perform two kinds experiment with-

out hash tag and hash tags into two types of data sets.

The result of applying the IWS approach with respect

to sarcasm tweets with the hash tag called #sarcasm

representing the sarcasm, obtained 96%of accuracy,

superiority of Bharti et al. Also, when they compared

ICAART 2017 - 9th International Conference on Agents and Artiﬁcial Intelligence

520

the two algorithms, IWS algorithm is a superiority

compared to PBLGA algorithm. Examples of each

emotion phrase data Bharti et al. have been used is as

shown in Table 1.

Table 1: Phrases used by Bharti et al.

positive sentiment phrase: Great, lucky, cute, good,

glad, delicious, best, Awesome, Perfect, joy, strong,

hilarious, better, goreous, honest, innocent, talented,

nice, happy, Pretty, proud, excellent

negative sentiment phrase: terrible, little, bit,

little, expensive, half, cold, food, tight jeans,

rude, wrong, dramatic, abusive, unhealthy, crap,

ugly, excuse, dirty, troubled, least, hard, bad,

bad, a few days

positive situation phrase: no regret, love,

love discovering, just love, effectively making,

absolutely Love, feel so loved, wanna be loved,

falling, honesty tell, will ﬂy,

love seeing, winning, now enjoying managing

negative situation phrase: Clarifying, are pumped,

released, are arriving, babysitting, only run, kicking,

crashing, destroyed, attacking, criticizing, is lying,

confused, dividing, exhausted, keep arguing,

will be, gets stuck, is losing, biting, shouting

2.2 Sarcasm Extraction Method based

on Patterns of Evaluation

Expressions

Hiai et al.(Hiai et al.(2016)) propose a extraction

method of sarcastic sentences in prodect review. First,

they analyze sarcastic sentences in prodect reviews

and classify the sentences into 8 classes by focus-

ing on evaluation expressions. Next, they generate

classiﬁcation rules for each class and use them to ex-

tract sarcastic sentences. Their method consists of

three stage; judgement processes based on reles for

8 classes, boosting rules and rejection rules. In the

experiment, they compare their method with a base-

line based on a simple rule. The experimental result

shows the effectiveness of their method. However,

they did not compare with other sarcasm determina-

tion method.

2.3 Automatic Detection of Sarcasm in

BBS Posts

Isono et al. (Isono et al.2013) propose two detec-

tion systems that determine sarcasm and slander in

posts on bullentim board system(BBS). They made a

corpus of sarcasm in BBS, and classiﬁed sarcasm in-

stances into eight classes: interrogative, guess, give-

up, unbalance, exaggeration, shock, metaphor, and

constract. For each sarcasm class, they constracted

syntactic patterns for detection of sarcasm that in-

clude sentence structures and polarity conditions of

the target sentence, the previous sentence and the next

sentence. Their ﬁrst system detects sarcasm using a

database of the syntactic patterns. They made a cor-

pus of slander using Support Vector Machine(SVM),

where as features, they use frequencies of words in

the list, and positive expressions and negative expres-

sions in the target sentence, the previous sentence and

the next sentence. In the experiment, the proposed

systems can achieve superior F-measures compared

with baseline systems. But, the accuracy of the sys-

tem for determining the sarcasm was low.

3 TASK SETTINGS

In this section, we describe the deﬁnition and exam-

ples of “sarcastic statement”. Around the deﬁnition of

sarcasm, actively debate is carried out in psychology

and a wide range of ﬁelds. However, it cannot obtain

a clear answer what kind of linguistic phenomenon

sarcasm is (Utsumi reference).

In this study, we focus on sarcastic sentences that

cause errors in polarity determination. We deﬁne the

sarcastic statement as follows.

Hiai et al.(Hiai et al.(2016)) propose a extraction

method of sarcastic sentences in prodect review. First,

they analyze sarcastic sentences in prodect reviews

and classify the sentences into 8 classes by focus-

ing on evaluation expressions. Next, they generate

Deﬁnition of sarcasm

✓ ✏

“Negative expression that convey a positive

meaning”or“Positive expression that convey a

negative meaning”

✒ ✑

An example of sarcastic sentence corresponding

to “Negative expression that convey a positive mean-

ing” is as follows.

tract sarcastic sentences. Their method consists of

three stage; judgement processes based on reles for

8 classes, boosting rules and rejection rules. In the

experiment, they compare their method with a base-

line based on a simple rule. The experimental result

shows the effectiveness of their method. However,

they did not compare with other sarcasm determina-

Automatic Detection of Sarcasm in

Example 1

✓ ✏

I bought a very tasty

cake on the way home

after I bought the game. I disposed

of the

game wrapped in wrapping paper of the cake!

Thank you!

✒ ✑

Example 1 includes more positive words (with sin-

gle underlines) than negative words (with double un-

derlines), while its overall meaning is negative. This

Example 1 tends to be classiﬁed as positive though

actually it expresses a negative feeling. Further, an

example of sarcastic sentence corresponding to “Pos-

itive expression that convey a negative meaning” is as

follows.

Sarcasm Detection Method to Improve Review Analysis

521

follows.

Example 2

✓ ✏

This game is too much attractive

, so it

troubles me

. I am going to be a junkie.

✒ ✑

Example 2 includes more negative words(with sin-

gle underlines) than positive words(with double un-

derlines), while its overall sentiment is positive. This

Example 2 tends to be classiﬁed as negative though

actually it expresses a positive feeling. We identify

the sarcastic statement that intention and expression

are different. Based on the above deﬁnition, we deter-

mine the sarcasm in the next section.

4 PROPOSED METHOD

In this section, we describe in detail the proposed

method. On the basis of the Bharti’s method, we pro-

pose a sarcasm-emotion detection method based on

polarities on words. It describes the method determin-

ing the sarcasm or emotion in the rest of the section.

4.1 Flow of Sarcasm Extraction

Technique

We describe an overview of the proposed method.

Figure 2 shows an overview of the proposed method.

The following describes the detail of Figure 2.

As the ﬁrst step, we carry out crawling from the

review to collect evaluation data. As the second

step, we parse the review and break down the whole

sentence into phrases. As the third step, we judge

whether the proposed phrase in the parsed ones are

included. In this study, we propose situation phrase

and sentiment phrase as our proposed phrase. In ad-

dition, we deﬁne the proposed phrase as follows.

Deﬁnition of proposed phases

✓ ✏

We call sentiment phrases and situation phrases

as proposed phrases in total.

✒

✑

For the parsed ones corresponding to the proposed

phrase, we manually determine the emotion (positive

or negative) in the parsed ones. When judging emo-

tion of parsed ones, we create the dictionary manu-

ally. As a ﬁnal step, we judge the sarcasm-emotion

detection by using the proposed phrases. By count-

ing the parsed ones to the positive or negative, we

determine the feeling of the proposed phrases. We

detect whether the review is sarcastic by using the

feeling of the proposed phrases. Sarcasm-emotion de-

tection is described in detail in 4.3. We applied the

proposed method to the review of the online review

site. In the rest, we describe in detail with respect to

the proposed method. Preprocessing before sarcasm-

emotion detection is described in 4.2. Section 4.3 de-

scribes sarcasm-emotion detection method to detect

sarcasm.

ƌĞǀŝĞǁ

ĚĂƚĂƐĞƚ

ĐƌĂǁůŝŶŐ

ĚĞƉĞŶĚĞŶĐǇƉĂƌƐŝŶŐ

ƉŚƌĂƐĞ

ƐŝƚƵĂƚŝŽŶƉŚƌĂƐĞ ƐĞŶƚŝŵĞŶƚƉŚƌĂƐĞ

ũƵĚŐĞǁŚĞƚŚĞƌ

ƉƌŽƉŽƐĞĚƉŚƌĂƐĞ

ƉŽƐŝƚŝǀĞŽƌŶĞŐĂƚŝǀĞ

ƐĂƌĐĂƐŵͲĞŵŽƚŝŽŶĚĞƚĞĐƚŝŽŶ

Figure 2: The proposed method.

4.2 Phrase Analysis

Here, we describe preprocessing for sarcasm-

emotional detection. Speciﬁcally, it is judged and

counted whether emotions are included in the parsed

ones. After crawling the review, we apply the depen-

dency parsing for a review. The dependency pars-

ing is a technique to analyze the syntactic relation-

ship, such as relationships between words to isolate

the whole sentence to morpheme. In this research, we

used Cabocha. CaboCha is a Japanese dependency

analyzer based on support vector machines. Figure 3

shows an example which carry out the syntax analysis

using Cabocha.

The following sentence is the input of the dependency

parsing shown in Figure 3. “This game is very inter-

esting. I threw it from the top ﬂoor of the momentum

left over by Abenoharukasu. Thanks to alchemist who

has drilling this kind of terrible game!!!. Thank you

Bannam.” From a result of the dependency parsing,

we made a review the set of the parsed ones. Next,

we disucuss the dependency relation. In Figure 3,



corresponds to “threw”, and

 corresponds to “from

ICAART 2017 - 9th International Conference on Agents and Artiﬁcial Intelligence

522

䛸䛶䜒

㠃ⓑ䛔䛷䛩

ໃ䛔

వ䛳䛶

䛒䜉䛾䝝䝹䜹䝇䛾

᭱ୖ㝵䛛䜙䐠

ᢞ䛢ᤞ䛶䜎䛧䛯䐟

䛣䜣䛺

䜽䝋䝀䞊䜢

䝞䞁䝘䝮䛥䜣䛒䜚䛜䛸䛖͊͊

㘐ᡂ䛧䛶䛟䜜䛯

㘐㔠⾡䛯䛱䛻

ឤㅰ͊͊͊

Figure 3: Dependency parsing.

the top ﬂoor”. Considering the analysis result of Fig-

ure 3, it turns out that

 qualiﬁes

.

Using the syntax analysis, we determine whether

the part of speech of the parsed ones can be included

as the part of speech of the proposed phrases. As a

proposed phrases, we propose a situation phrase and

a sentiment phrase as the type of phrase. Situation

phrase means to the “action” in the sentence, and sen-

timent phrase means to the “emotion” in the sentence.

Proposed phrases is represented by a combination of

parts of speech. Table 2 shows a part of speech corre-

sponding to the proposed phrases.

Table 2: Correspondence table of the phrases.

We treat a parsed ones as a situation phrase or sen-

timent phrase according to which combination of the

parts of speech given in Table 2 matches the target

phrase. Table 3 shows the result of sorting those ap-

plicable to proposed phrase from the parsed ones in

Figure 3. It should be noted that in this paper is the

selection of the phrase to depend on parsing the re-

sults of Cabocha. Also, if it contains proper nouns in

the phrase, the phrase is treated as noise representa-

tion.

Finally, we determine the phrase that can be deter-

mined include feelings to phrases correspond to the

proposed phrase (including positive or negative). In

this study, we determined whether positive or neg-

Table 3: Results of phrase classiﬁcation of the example.

sentiment phrase: interesting,

the top ﬂoor, told me to drilling,

alchemy, thanks

situation phrase: left over, threw,

terrible

ative manually. Table 4 shows the result of emo-

tional sorting those applicable to proposed phrases

from the phrase in Figure 3. Incidentally, we deal with

Table 4: Emotional phrase correspondence table.

Positive sentiment phrase : thanks, interesting

Negative sentiment phrase : None

Positive situation phrase : None

Negative situation phrase : threw

phrases that include proper names as noise representa-

tion. From the above procedure, we calculate whether

contains a number of emotions subject to review. By

using a number of emotions subject to review, we de-

termine the feelings of phrase. In the next section,

we describe the method of calculating the phrase of

emotion and the sarcasm-emotion detection.

4.3 Sarcasm-Emotion Detection

Here, we describe a method sarcasm-emotion detec-

tion by using the emotion of the proposed phrases.

In the previous section, we examined whether or not

there is a phrase that can be judged as positive or

negative for the phrase that corresponds to the pro-

posed phrases. Here, we describe about the judges

approach the feelings of the “proposed phrase”. After

determiningthe feeling of proposed phrases, we judge

sarcasm-emotion detection method as a ﬁnal evalua-

tion.

4.3.1 Deciding Sentiment of a Proposed Phrase

As a prestage of sarcasm-emotion detection, we eval-

uated the feelings of the phrase. In order to assess

the feeling of the “proposed phrases”, we deﬁne the

following formula.

PR =

PWP

TWP

NR =

NWP

TWP

SentimentScore = PR − NR

For the proposed phrases, PR is positive ratio, NR is

negative ratio, TWP indicates the number of all of the

Sarcasm Detection Method to Improve Review Analysis

523

proposed phrase with respect to each reviews. In ad-

dition, PWP and NWP are the numbers of positive

and negativeproposed phrases respectively. Using the

above equations, we describe technique for determin-

ing the emotion of the phrases below. Procedure for

determining the emotion of the proposed phrases is as

shown in Figure 4. Let us explain Figure 4.

^ĞŶƚŝŵĞŶƚ^ĐŽƌĞŽĨĂƉƌŽƉŽƐĞĚƉŚƌĂƐĞ് Ͳǫ

dŚĞĞŵŽƚŝŽŶŽĨƚŚĞƉŚƌĂƐĞ

/ƐĚĞƚĞƌŵŝŶĞĚďǇƵƐŝŶŐƚŚĞƚĨͲŝĚĨ;ϮͿ

/Ĩ^ĞŶƚŝŵĞŶƚ^ĐŽƌĞфϬїEĞŐĂƚŝǀĞ

/Ĩ^ĞŶƚŝŵĞŶƚ^ĐŽƌĞхϬїWŽƐŝƚŝǀĞ;ϭͿ

EŽ

zĞƐ

^ĂƌĐĂƐŵͲĞŵŽƚŝŽŶĂůĚĞĐŝƐŝŽŶ

ĞĐŝĚŝŶŐƐĞŶƚŝŵĞŶƚŽĨĂƉƌŽƉŽƐĂůƉŚƌĂƐĞ

Figure 4: Emotion detection of proposed phrases.

If the SentimentScore of the proposed phrases is

unequal to 0, as in (1) of the Figure 4, then the emo-

tion of the phrase is determined based on the Senti-

mentScore, namely, positive when SentimentScore >

0 and negativeotherwise. If the SentimentScore of the

proposed phrases is equal to 0, as in (2) of the Figure

4, then the emotion of the phrase is determined us-

ing tf-idf. The tf-idf is a method of calculating the

weight of a particular word in the document that by

combining tf method and idf method. This technique

has been utilized in the ﬁeld of information retrieval

and text mining and machine learning. It is shown

below with respect to the meaning of tf method and

idf method. tf (term frequency) represents term fre-

quency, it represents the number of times a particu-

lar word is found at the text in. idf (inverse docu-

ment frequency) represents the natural logarithm of

the inverse of the number of documents that contain

the words in the training data. A high frequency in

the text it is seen that as much the word is important.

Calculation formula for weighting is as formula.

t,d

= t f

t,d

log

d f

Here, N is the data for the entire review data. Further,

t f

t,d

in the number oft appears in a document d, d f

the number of documents in which the word t appears

in the N sentences. By using the above determination

process, it was evaluated in the emotions of the sit-

uation phrase and sentiment phrase. Taking Table 4

as an example, the emotion of the sentiment phrase

is determined to be positive, emotion of the situation

phrase is determined to be negative. After determin-

ing the feelings of the proposed phrase, we will make

a ﬁnal sarcasm-emotion detection.

4.3.2 Deciding Sarcasm-Emotion Detection of

Reviews

Next, we determine the sarcasm or emotion by using

the emotion of the proposed phrase. Figure 5 shows

sarcasm-emotion detection method.

^ŝƚƵĂƚŝŽŶWŚƌĂƐĞ

;W͕E͕EĞŝƚŚĞƌͿ

^ĞŶƚŝŵĞŶƚ

WŚƌĂƐĞ

;W͕E͕EĞŝƚŚĞƌͿ

ƉŽƐŝƚŝǀĞ EĞŐĂƚŝǀĞ EĞŝƚŚĞƌ

WŽƐŝƚŝǀĞ WŽƐŝƚŝǀĞ ^ĂƌĐĂƐŵ WŽƐŝƚŝǀĞ

EĞŐĂƚŝǀĞ ^ĂƌĐĂƐŵ EĞŐĂƚŝǀĞ EĞŐĂƚŝǀĞ

EĞŝƚŚĞƌ WŽƐŝƚŝǀĞ EĞŐĂƚŝǀĞ EĞŝƚŚĞƌ

Figure 5: Sarcasm-emotion detection.

Based on the deﬁnition, it determines that the sar-

castic statement in this study if the emotions of the

proposed phrases are different. Taking Table 4 as an

example, the emotion of sentiment phrase is judged

to be a positive, the emotion of the situation phrase is

determineded to be a negative, therefore this review is

determined to be the sarcastic statement. Also, if both

of the emotion of the situation phrase and sentiment

phrase are the same, it is determined that the target

sentence is non-sarcastic statement, the same emotion

of the entire sentence. By using the sarcasm-emotion

detection method described above, we determine the

sarcasm.

5 SARCASM JUDGEMENT

EVALUATION

In this section, we conﬁrm the accuracy of the pro-

posed method. At that time, we compared to the ex-

isting sarcasm detection method.

5.1 Experimental Set-up

We conduct an experiment using reviews of a product

that are “ﬂaming”, that is, including many negative

reviews. We evaluate as a target star5 reviews of Mo-

bile Suit Gundam EXTREME VS-FORCE of amazon

products. The subject of the review has been “ﬂam-

ing” for game content is unpopular. Because there

are many negative expressions in spite of high scores,

we treate as experimental data in this experiment. We

applied the proposed method to 140 reviews of star5

reviews. Before applying the approach, we labeled

“sarcastic statement or non-sarcastic statement” and

“positive or negative or neither”. Determination of

ICAART 2017 - 9th International Conference on Agents and Artiﬁcial Intelligence

524

sarcasm is based on the deﬁnition given in section

3. Also, if you cannot determine whether positive or

negative in the non-sarcastic statement, we treate as

“neither”. Examples of non-sarcasm statement “nei-

ther” are included those such as the following.

Example3

✓ ✏

I want to say about this game only the following

words: May the Force be with you.

✒ ✑

As mentioned above, we dealt things not estab-

lished as sentences as “neither”. Table 5 and Table 6

show the result of labeling.

Table 5: Labeling result 1.

sarcastic statement 89

non-sarcastic statement 51

Table 6: Labeling result 2.

positive 33

negative 11

neither 7

Table 5 is a result of the labeling process to ei-

ther the entire review “sarcastic statement” or “non-

sarcastic statement”. Table 6 is a result of the labeling

process in three of emotion “ is non-sarcastic state-

ment”. Result of labeling, 89 reviews from 140 re-

views is determined to be sarcastic statement. Fur-

ther, we can assume that “ﬂaming reviews” are more

likely to contain sarcastic in the star5 reviews. Using

the above results, we perform each experiment.

5.2 Evaluation Method

Here, we describe the experimental results of the pro-

posed method. To compare the results with exist-

ing research, three parameters are considered, namely,

precision, recall and f-score. We describe below for

each evaluation method. Table 7 shows evaluation

method in this research.

Table 7: Evaluation method.

fact

sarcasm non-sarcasm

prediction sarcasm T

@non-sarcasm F

We write the following precision , recall , F-score by

using the evaluation method in Table 7.

Precision: Among the predicted data, actually shows

the proportion of what is positive. Formula becomes

(1)

Precision =

+ F

(1)

Recall: From what is positive, indicating the percent-

age is expected to be positive. Formula becomes (2)

Recall =

+ F

(2)

F-score: Show the harmonic mean of precision and

recall.Formula becomes(3)

F − score =

2· Precision· Recall

Precision+ Recall

(3)

Using the above evaluation method, we evaluate the

accuracy of the proposed method.

5.3 Evaluation Results

We compare with the existing sarcastic statement de-

tection method and the proposed method in this data

set. We apply to this data set the sarcasm detection

method of Isono et al. (2013) and Hiai et al. (2016).

Table 8 shows the existing sarcastic statement detec-

tion method and the proposed method comparison re-

sult.

The result of applying the proposed method to the

data set, proposed method achieved 0.79, 0.56, and

0.63 precision, recall and f-score respectivelyin Ama-

zon dataset. As the result of applying the existing sar-

casm determination method in this data set, it found

that the proposed method outperformed the existing

method in terms of the accuracy to determine sarcas-

tic statements. The reason of this result would be be-

cause the expressions speciﬁc to games did not match

the syntactic patterns of the existing research. There-

fore we make clear that the advantage of our proposed

method is due to no use of syntactic patterns. From

the above, determining the sarcastic statement by us-

ing the our proposed method is useful method in the

online revies site.

Table 8: Experiment result 1.

precision recall F score

proposed method 0.79 0.56 0.63

Hiai et al. 0.60 0.03 0.06

Isono et al. 0.63 0.07 0.13

6 DISCUSSIONS

From the experimental result, it shows the results

of high accuracy of proposed method for sarcasm

Sarcasm Detection Method to Improve Review Analysis

525

of our deﬁnition. However, they also show the re-

sults in which our method sometimes determine Sen-

timentScores that the reviews do not actually mean

if the numbers of positive and negative phrases are

much different. It will be discussed because our pro-

posed method depends on the numbers of the pro-

posed phrases. Therefore, we consider that it can not

determine the sentences as “sarcasm” such as follow-

ing.

Example 4

✓ ✏

It is like a dream that I can use such a cool minia-

ture aircraft so early by purchasing such a terri-

ble game!

✒ ✑

A result of the proposed method apply to the ex-

ample 4, the proposed phrase determined “positive is”

by containing many positive phrase. Thus our method

determined that this review praised the product posi-

tively. This result shows that there are sarcastic sen-

tences that cannot be correctly classiﬁed by counting

the phrases in the reviews. In order to judge this re-

view as sarcastic, we will need to consider combina-

tions of phrases or interjections.

7 CONCLUSION

In this paper, we proposed a sarcastic statement deter-

mination for the classiﬁcation accuracy improvement

of emotional judgment in the online review sites. The

result of the comparison of the existing research, we

could conﬁrm the usefulness of the proposed method.

However, there are sarcastic reviews our method can-

not judge as sarcastic may be because our method de-

tect sarcasm on the basis of the numbers of positive

and negativephrases. In order to determine the review

such as described above, we will need to improve our

method. We are going to improve our method to in-

crease the accuracy of sarcasm and emotion detection

by incorporating interjection words and considering

combinations of phrases.

ACKNOWLEDGEMENT

This work was supported by JSPS KAKENHI Grant

Numbers 26330081, 26870201, 16K12411. In per-

forming this study, we would like to thank everyone

that has helped us.

REFERENCES

Santoth Kumar Bharti , Korra Sathya Babu , Sanjay Kumar

Jena. Parsing-based Sarcasm Sentiment Recognition

in Twitter Data, The 2015 IEEE/ACM International

Conference on Advances in Social Networks Analysis

and Mining

Ashwin Rajadesingan , Reza Zafarani , Huan Liu. Sarcasm

Detection on Twitter: A Behavioral Modeling Ap-

proach, Proceedingsof the Eighth ACM International

Conference on Web Search and Data Mining, 2015

Dmitry Davidov , Ari Rappoport . Semi-supervised recog-

nition of sarcastic sentences in Twitter and Amazon,

Proceedings of the Fourteenth Conference on Compu-

tational Natural Language Learning, 2010

F. Barbieri, H. Saggion, and F. Ronzano . Modeling sarcasm

in twitter , a novel approach, in Association for Com-

putational Linguistics, 2014

Fumiya Isono , Suguru Matsuyoshi , Fumiyo Fukumoto

. Automatic Detection of Sarcasm in BBS Posts

Based on Sarcasm Classiﬁcation, Information Pro-

cessing Society of Japan SIG Technical Report, 2013

in Japanese

Satoshi Hiai , Kazutaka Shimada. Analysis and Detection

of Sarcastic Sentences Focused on Evaluative Expres-

sions, Information Processing Society of Japan SIG

Technical Report, 2016 in Japanese

Akira Utsumi , A Cognitive and Computational Model of

Irony Interpretation, Transactions of Information Pro-

cessing Society of Japan, Vol41 , No9, pp.2498-2509,

2000 in Japanese.

CaboCha , http://chasen.org/˜taku/software/cabocha/

SentimentAnalyZer, http://www.ohsuga.is.uec.ac.jp/

sentiment

ICAART 2017 - 9th International Conference on Agents and Artiﬁcial Intelligence

526