Profiling Media Outlets and Audiences on Facebook: COVID-19

Coverage, Emotions and Controversy

Luciana Oliveira

and Joana Azevedo

CEOS.PP ISCAP Polytechnic of Porto, Rua Jaime Lopes Amorim s/n, Matosinhos, Portugal

ISCAP Polytechnic of Porto, Rua Jaime Lopes Amorim s/n, Matosinhos, Portugal

Keywords: Social Media, Facebook, Covid-19, Media Coverage, Audience, Emotion Detection, Controversy.

Abstract: The outburst of the COVID-19 pandemic was accompanied by a steeply rise of worldwide media coverage of

the phenomena, in which social media were deemed as critical platforms and became a popular place to

receive and share information, as well as express personal views. In this paper, we present the preliminary

results of an ongoing work devoted to analysing the media coverage of the COVID-19 outburst in Portugal

(March-May 2020), the subsequent emotional engagement of audiences and the entropy-based emotional

controversy generated. Using a cross-sectional descriptive methodology, we analyse the activity of the three

major news outlets in the country for the category of general news. Our results reveal three distinct profiles

of media coverage, negativity as the core engine for interacting with news and spreading information, negative

and volatile Facebook reactions (“Angry”, “Haha” and “Wow”) as main inputs for controversy, prevailing on

COVID-19 news, and a general tendency of audiences to share controversial news.

1 INTRODUCTION

The SARS-CoV-2 virus wielded from Wuhan in

December 2019. The World Health Organization

(WHO) later confirmed 41 cases and one death on

January, 2020, and by 11

March a global

pandemic was declared. Since then, the world has

been transformed into a highly infected environment

with the community-sustaining transmission. Daily

activities were halted or limited across the globe, and

people were confined to their homes in an

unprecedented circumstance, totally unprepared and

unsure of how the crisis would unfold. The stay-at-

home movement drove news outbreaks into social

media, where viewers had quick access to material

that would have been otherwise unavailable via

conventional means.

Social media platforms have been transforming

the journalism business dramatically in recent years

(Ferrucci, 2020; Poell, 2020). While the news

industry conventional value generation process has

been company-centric and self-contained, with little

contact with consumers, the consumer value creation

in the social era is part of a larger transformation of

https://orcid.org/0000-0003-2419-4332

https://orcid.org/0000-0001-5163-6103

the media and society (Serrano, Greenhill, &

Graham, 2015). Network journalism is a structural

concept that spans the global journalistic sphere,

affecting journalists, organisations and audiences, as

the journalistic narrative began to rely on audience

participation in a public and immediate manner

(Dalmaso, 2017). As stated by (Castells, 2004), we

currently live in an informational and networked

society as a result of the digital and global

communication era.

Moreover, pandemics pose collective health

dangers but also daily challenges for mental and

public health. Strong (1990) states that every

epidemic causes three social epidemics: fear (being a

carrier of the illness), morality (moral reactions to the

epidemic itself, which may be good or bad), and

action (rational or irrational changes in daily habits in

response to the disease). He also emphasises that

these are produced by language and gradually

nourished by it via different social interactions.

People's expression of views, emotional state and

how they react to a subject may be used to assess the

effect of events and news on their life. Collective

emotions arise when a large number of people share

186

Oliveira, L. and Azevedo, J.

Proﬁling Media Outlets and Audiences on Facebook: COVID-19 Coverage, Emotions and Controversy.

DOI: 10.5220/0010717400003058

In Proceedings of the 17th International Conference on Web Information Systems and Technologies (WEBIST 2021), pages 186-196

ISBN: 978-989-758-536-4; ISSN: 2184-3252

one or more emotional states, which tends to happen

in online communities (Kappas, 2017), and they can

spread like a virus (Ferrara & Yang, 2015). Moreover,

collective feelings tend to last longer than individual

emotional responses (Garcia, Kappas, Küster, &

Schweitzer, 2016), amplifying the extent of a crisis.

Thus, understanding the behaviour of the general

population may help identify abnormal affective

dynamics, which have even been associated with

mental illnesses like depression (Koval, Pe, Meers, &

Kuppens, 2013; Su et al., 2021).

In this context, social media were deemed critical

platforms and became a popular place to receive and

share news updates and express personal views on the

pandemic. With the enormous influx of health,

normative, political, and social information, social

media rapidly became the venue to which

communication and engagement converged, fostering

the sharing of thoughts and emotions. For this reason,

it has also become a thriving field for understanding

how people cope with the crisis and react to

uncertainty as a window into the current social

landscape.

In this paper, we present the preliminary results of

an ongoing work devoted to profiling news outlets in

Portugal, based on the media coverage of the COVID-

19 outburst (March-May 2020), the emotional

engagement of audiences and the entropy-based

emotional commotion generated, which is translated

in the controversy produced around the phenomena.

2 BACKGROUND

In this section we briefly refer to the media coverage

of COVID-19, the use of click-based reactions as

proxies to analyse public emotions and emotion-

based controversial news on social media.

2.1 COVID-19 Media Coverage

The media is an essential link between science and

society since citizens use mass media to inform their

attitudes, views, and behaviour. Peoples’ views about

the epidemic's origins, attitudes about suitable

governmental solutions, and general politicisation of

the situation have been shown to be significantly

influenced by media coverage of the pandemic

(Bolsen, Palm, & Kingsland, 2020; Pearman et al.,

2021).

Pearman et al. (2021) reported on a steeply rise in

worldwide media coverage of COVID-19 events in

102 high-circulation newspaper sources across 50

countries around the world, as other pressing matters,

such as climate change, dropped drastically. The

authors also state that, despite the fact that the

COVID-19 epidemic continues to spread quickly

across the world, its media coverage has diminished

since the first flurry of attention it got at the start of

the crisis in early 2020. Oliveira, Sequeira, Oliveira,

Silva, and Mesquita (2021) also confirm the same

pattern in Portugal, stating that media coverage

remained relatively low after the first wave of the

pandemic, even as the country passed through the

second, and even most severe, the second wave of

infections. As argued by the authors, we believe that

this is a reflection of the normal and expected

variation of the attention given to the public issue, as

it is explained by the issues-attention cycle model

proposed by Downs (1972).

The issue-attention cycle refers to the fluctuations

of public or media attention given to a particular topic

(Downs, 1972), and includes five stages. The first is

the pre-problem phase when an issue does not get

much public notice. Only a few individuals, like

specialists or interest groups, are aware of it. In the

second phase, public awareness grows, and a time of

alarming discovery may ensue. But this is frequently

coupled with the idea that taking action would fix the

issue. The third stage occurs when individuals realise

that addressing the issue is bigger and more resource-

intensive than they thought. The fourth phase is

characterised by a gradual loss of public attention and

a sense of detachment, even though the issue persists.

In the last phase, issues are replaced by new ones,

causing “spasmodic recurrences of interest” (Downs,

1972, p. 39).

As devised by Downs, the issues-attention cycles

applies both to the media coverage of news and to the

interest and engagement of audiences with those same

issues, as they can evolve at different paces.

2.2 Social Media Emotions

Social media emotions have been increasingly used to

gain better insights into the audiences’ behaviour.

Emotion detection involves categorising text into

several emotion categories. Some studies in this

domain have identified sentiment analysis and

emotion identification under sentiment analysis, but

they are different (Balahur, 2013). Emotions are more

expressive than sentiments since they do not need a

feeling to exist (Liu, 2012; Wang & Pal, 2015).

Emotion models may be dimensional or

categorical (Wang & Pal, 2015). Valence, arousal,

and dominance are three temporal dimensions of the

dimensional models (Ekkekakis, 2013). A

contemporary example is Pellert, Schweighofer, and

Proﬁling Media Outlets and Audiences on Facebook: COVID-19 Coverage, Emotions and Controversy

187

Garcia's model of emotional dynamics on social

media (2020). The most well-known categorical

emotion model includes the emotions anger, disgust,

fear, happiness, sadness, and surprise (Ekman, 1992).

The author sees emotions as distinct, instinctive

reactions to global, cultural, and personal events

(Ekman & Cordaro, 2011). Several studies have

utilised Ekman's work to assess public mood by

automatically classifying social media content. For

example, Ofoghi, Mann, and Verspoor (2016) studied

Twitter emotions linked to Ebola, and Li et al. (2020)

studied cultural emotional disparities between

America and China to portray public affection

dynamics during COVID-19.

Giuntini et al. (2019) and Oliveira et al. (2021)

believe that the attribution of emotions and polarity

suggests that there may be a connection between the

emotions felt and the reactions exhibited in the virtual

world.

On Facebook, users often utilise emoticons in

posts, chats, and comments to convey more meaning

without having to write. Emoticons are tiny pictures

or combinations of diacritical symbols designed to

replace nonverbal components of communication

(Giuntini et al., 2019). Emoticons have become the

most popular way to communicate feelings on social

media (Oleszkiewicz et al., 2017), and several studies

have built upon emotions and emoticon reactions on

social media (such as Cazzolato et al. (2019);

(Giuntini et al., 2019; Tian, Galery, Dulcinati,

Molimpakis, & Sun, 2017).

Giuntini found significant links between the set of

fundamental emotions and the Facebook click-based

reactions set. For instance, "Angry" means angry,

"Wow" means surprised, "Sorrowful" means sad, and

"Love" means pleasure. “Like” is ambiguous in terms

of polarity and sentiment. Fear is the only

fundamental emotion that has no corresponding

visible reaction (Giuntini et al., 2019). However,

click-based responses remain an underused resource

in social media research, despite quick-draw, ready-

made expressive features are becoming more

common across various platforms, attracting research

interest in recent years (Freeman, Alhoori, &

Shahzad, 2020).

2.3 Controversy on Social Media

It is known that the media attention has been

disproportionately directed toward COVID-19 news,

with little consideration for how the pandemic-related

media coverage might influence people’s mental

health (Su et al., 2021). Some of the most recent risks

and potential dangers of social media communication

have been aggravated by the tremendous spread of

COVID-19 news and information. In fact, along with

a pandemic caused by a lethal virus, the globe has

been experiencing an "infodemic", as defined by the

World Health Organization (Organization, 2020,

2021). This refers to the epidemic of misleading or

incorrect information spreading rapidly via social

media's fertile ground, fuelled by the fear, worry, and

uncertainty generated by this new danger.

The tremendous number of efforts devoted to

combating fake news, the creation of international

alliances (like Poynter) and the growing cooperation

between journalists and social networks have been

ensuring that, at least, the legacy media outlets do not

spread disinformation nor misinformation.

The spread of fake news, however, is not the only

threat fostered by the COVID-19 infodemic. User-

generated content (UCG) remains as one of the main

challenges in controlling the spread of fake news

(Ferrari, 2020), a challenge that escalates in the

spread of hate speech, which requires a lot less

creativity and effort from users. Dori-Hacohen, Sung,

Chou, and Lustig-Gonzalez (2021) state that healthy

online discourse is becoming less and less accessible

beneath the growing noise of controversy, mis- and

dis-information, and toxic speech.

Controversial heated discussions are a prolific

field for hate speech on social media, and according

to Dori-Hacohen et al. (2021), controversy is also

saliently connected with disinformation. One of the

main current challenges of hate speech recognition is

the automatic detection of irony (MacAvaney et al.,

2019) because people verbalise an idea while

implying the opposite meaning; thus, textual features

alone fail in recognising the implicit meanings of the

discourse.

Irony serves the additional social and emotional

functions of projecting emotions like humour or

anger, and ironic comments may provoke stronger

emotional responses than literal comments

(Thompson, Mackenzie, Leuthold, & Filik, 2016). In

their research about irony, the authors introduce

paralinguistic features (emoticons) to improve the

detection of praise and criticism in written messages.

Such methods had already been employed by other

studies such as Carvalho, Sarmento, Silva, and De

Oliveira (2009) and Derks, Bos, and Von Grumbkow

(2008).

More recently, with the expansion of the

Facebook like button into a full set of click-based

emotional reactions to content, other studies emerged

taking advantage of the convenience of the

systematised and bulk emotional response that is

WEBIST 2021 - 17th International Conference on Web Information Systems and Technologies

188

promptly captured to study the emotional irony

conveyed by the audiences.

This research stream is predicated on the premise

that controversial posts divide a community's

preferences, garnering both substantial positive and

negative responses or polarised towards extremes

(e.g. “Love”-“Angry”). As such, these works build on

the study of social media click-based emotions such

as the one conducted by Freeman et al. (2020), who

measured the Pearson correlation coefficients for all

reaction pairs in their dataset of scholarly articles

published on Facebook; or the work of Tian et al.

(2017), who used a K-means to cluster reactions and

investigate which reactions were most likely to be

seen together on a post in UK, US, France and

Germany.

Related research using Facebook reactions as

proxies to identify controversy can be found in

Sriteja, Pandey, and Pudi (2017), who have used this

method for detecting controversial topics during the

US Presidential elections 2016. Basile, Caselli, and

Nissim (2017), also followed the same procedure to

identify controversy among four major Italian

newspapers and one media agency, using an entropy-

based model to compute the ‘disorderliness’ of

emotional reactions to posts. Finally, Gray (2020)

studied gender bias in the Facebook pages of the

United States 2020 Senate candidates, using the exact

same method as Basile et al. (2017).

Agile methods for early detection of controversy

may be helpful in assisting media outlets, journalists,

social networks and fact-checkers in preventing hate

speech and disinformation.

3 METHODS AND PROCEDURES

This work follows the general approach of

quantitative content analysis (Bryman, 2016), and it

consists of a cross-sectional descriptive study. We

used the Facebook Graph API to retrieve the news

posted by the three major daily news providers in

Portugal, Sic Notícias (1,717,794 followers), TVI 24

(1,088,453 fans) and CMTV (580,703 followers)

between the 1st March and 31st May, 2020. The

choice of news outlets was based on two principles:

a) high visibility, expressed through a high number of

fans, and b) have a generalist editorial line, with a

broad spectrum, not segmented for specific

audiences. The timeframe for the analysis was

delimited according to the first mandatory

confinement imposed by the government, including

the moment of detection of the first case of infection

(first week of March 2020) and the announcement of

the first measures of deconfinement (May 2020).

Thus, the full period of analysis consists of three

months.

The dataset is composed of 30,607 news posted

on the network, for which we collected the created

date and time, link (news external URL), message

(text included in the post), link text (the title of the

news), description (news lead), Likes, Comments,

Shares, Love, Wow, Haha, Sad, Angry and Care. We

refer to “Like” (somewhat a default type of

interaction with content), “Comment” and “Share” as

forms of interaction with content; and to “Love”,

“Wow”, “Haha”, “Sad”, “Angry” and “Care” as

reactions, in the sense that these convey emotional

responses. The dataset of news was manually

categorised into two subsets: COVID-19 news and

Other news and their subdomains (e.g. politics,

education, prevention, etc.). For this stage of the

research, we refer only to the top-level binary

categorisation of COVID-19 and Other news, as our

first set of goals is to a) characterise and compare the

media coverage given to COVID-19 in news outlets,

b) explore the public response to these news, namely

their emotional state and c) identify the most

controversial news and their content.

For the analysis of media attention and audiences’

emotional involvement, we follow the general

principles of the issues-attention cycles proposed by

Downs (1972) and the detection of emotions through

Facebook’s click-based reactions, as used by Giuntini

et al. (2019). For the analysis of controversial news,

we follow Basile et al. (2017) model and compute the

entropy (quantitative measure of disorder) of the

Facebook’s reaction set per post as a function to

determine controversy.

Table 1 provides an overview of the data

collected, depicting the post type for each news outlet

and topic of news - COVID-19 news (“COV”) and

Other news (“Oth”).

Table 1: Total posts per outlet and category.

SICNotícias TVI24 CMTV

Type COV

Oth COV Oth COV Oth

Link 11866

5760 4013 3966 1935 1941

Video 3

99 30 134 281 217

Photo 0

21 10 325 0 1

Status 0

3 0 0 0 2

NSub 11869

5883 4053 4425 2216 2161

NTot 17752

8478 4377

N% 66.86

33.14 47.81 52.19 50.63 49.37

Proﬁling Media Outlets and Audiences on Facebook: COVID-19 Coverage, Emotions and Controversy

189

For all news outlets, “Link” is the most frequent

post type, which is consistent with the current practice

of sharing news links directly from their news portals.

Photos and videos are rarely posted and are more

frequent for CMTV and TVI24. The news outlet with

the highest number of posts, i.e., the highest

communication investment, is Sic Notícias, four

times higher than CMTV and two times higher than

TVI24. Additionally, this is the entity with the highest

rate of COVID-19 news posted in the trimester

(66.86%), followed by CMTV (50.63%) and TVI 24

(47.81%).

4 RESULTS

In this section, we present the results concerning the

evolution of the media attention given to COVID-19

news, the emotional response from the audiences and

the detection of emotion-based controversy. Findings

are discussed in this section for a matter of simplicity.

4.1 Media Coverage

The analysis of the media attention given to COVID-

19 news permits the contextualisation of the evolution

of the public emotional response.

Figure 1 illustrates the evolution of the media

attention given to COVID-19 and Other news during

the fourteen weeks of the trimester for the three news

outlets under analysis. For the sake of data

visualisation quality, we present the corresponding

audiences’ emotional engagement with the news side

by side, depicting it in Figure 2, for the same trio of

outlets. Five key moments are marked to provide a

clearer insight on the national context regarding the

(1) first case of infection in the country (2

March),

(2) first confinement measures (12

March), (3)

declaration of the State of Emergency and total

lockdown (19

March), (4) declaration of the State of

Calamity and the first stage of deconfinement

measures (3

May), and (5) second stage of

deconfinement measures (17

May).

It is possible to observe an overall tendency of

confirmation of the issues-attention cycles proposed

by Downs (1972), which is usually represented by a

bell-shaped curve with a stretched right side to

indicate that the subject takes more time to fade away

than the one it took to reach its peak of interest. This

stretched right side might then suffer from spasmodic

occurrences of interest or events that lead to slight

rises (small bumps), which never ascend to the first

stance of alarming discovery. This is particularly

visible in Figure 1c, concerning CMTV, where two

spasmodic occurrences happen in weeks 7 and 12,

and in Figure 1b, concerning TVI24, in weeks 8 and

9. For the trio of news outlets, the stage of alarmed

discovery happens in week 4, which includes all

communication and news regarding the declaration of

the State of Emergency and total lockdown. The

reason why the percentages of news in week 14 for

all outlets is very low is that this week only refers to

one day, 31st May.

Despite this overall tendency, it is also possible to

observe three distinct behaviours in terms of the

intensity and duration of the attention given to

COVID-19 news. CMTV (Figure 1c), despite having

published the lowest absolute among of COVID-19

news (Table 1) was the one with the highest

percentage of media coverage of the phenomena,

reaching a peak of nearly 14%, in a shorter time span

(seven weeks straight), followed by a drastic

reduction of coverage. We believe that this is

consistent with the reputation for sensationalism that

precedes this outlet, which is also enhanced by the

fact the media coverage began later and with a more

drastic increase.

Both TVI24 (Figure 1b) and Sic Notícias (Figure

1a) present a more gradual decline of media coverage

keeping, with variation between 6% and 10% for nine

weeks straight, after which the Other news surpasses

the volume of COVID-19 news. It is also worth

noticing that TVI24 (Figure 1b) was the only outlet

with a lower discrepancy between the coverage of the

COVID-19 phenomena and Other news.

4.2 Interaction and Emotional

Engagement

Figure 2 illustrates the evolution of audiences’

interaction with the news (“Comments” and

“Shares”) and their emotional engagement, computed

according to the click-based reaction set of Facebook

(“Love”, “Wow”, “Haha”, “Sad” and “Angry”). We

intentionally left out “Likes”, as previously

explained, and the reaction “Care”, because it was

introduced mid-period in the first week of April, thus

not permitting consistent comparisons.

Before contextualising the audiences’ emotional

engagement in the trimester, we analyse the

emotional profile of the audiences per news outlet.

Table 2 provides an overview of the average

interaction and emotions per outlet and news topic,

highlighting the statistically significant differences

detected by a one-way ANOVA test. An overall

prevalence of sadness and anger is also visible in

Figure 2.

WEBIST 2021 - 17th International Conference on Web Information Systems and Technologies

190

Figure 1: Evolution of the percentage of COVID-19 news

and Other news per outlet – a) SICNotícias, b) TVI24, c)

CMTV.

Table 2: Average interaction per outlet and news topic.

SICNoticias TVI24 CMTV

Type COV Oth COV Oth COV Oth

Love 10 6 11 45 9 8

Wow 9 4 14 11 11 11

Haha 6 8 6 10 6 7

Sad 35 14 57 55 50 57

Angry 12 9 16 27 20 35

Com. 46 32 52 86 42 58

Shares 89 28 119 149 119 98

TVI24 is, undoubtedly, the news outlet generating

the highest emotional commotion among audiences,

for almost all types of reactions, [“Love” (p<.001),

“Wow” (p<.001), “Haha” (p<.005), “Sad” (p<.001)],

except for “Angry”, which predominates in CMTV’s

audiences (p<.001). This commotion of emotions is

quite visible in Figure 2b, for the entire period. The

news outlet is also the one registering the highest

average of “Comments” (p<.001) and “Shares”

(p<.001) per post.

Figure 2: Evolution of the percentage of Facebook

emotions, comments and shares – a) SICNotícias, b)

TVI24, c) CMTV.

We deepened this analysis to find out if the

variance of emotional expression is related to the

presence of COVID-19 news or not. A series of t-tests

were applied, and results are summarised in Table 3,

where significant differences have been highlighted.

Table 3: Average emotions and interaction per outlet and

news topic.

SICNoticias TVI24 CMTV

Type COV Oth COV Oth COV Oth

Love 10.03 6.05 11.01 44.51 9.35

7.63

Wow 9.33 4.49 13.61 11.35 10.57 10.86

Haha 6.40 8.32 6.21 10.22 5.50 7.40

Sad 34.60 13.92 57.33

54.61 50.31 56.93

Angry 12.12 9.01

15.71 27.22 20.46 35.39

Com. 46.20 32.23 51.86 86.28 42.11 57.74

Shares 88.75 27.61

118.80 148.56

118.81

98.28

n.s.

Proﬁling Media Outlets and Audiences on Facebook: COVID-19 Coverage, Emotions and Controversy

191

In the case of Sic Notícias, the emotional reactions

“Love”, “Wow” and “Sad” are significantly

associated with COVID-19 news (p<.001), as well as

the interactions “Comments” and “Shares” (p<.001).

“Haha” is more present in Other news (p<.001). This

is quite noticeable in Figures 1a and 2a, as the

emotional expression of audiences mirrors the

decrease in COVID-19 media coverage. For all news

outlets, the sharing behaviour is more frequent in the

first 6 weeks of the trimester and commenting is more

frequent in the last 6, particularly for CMTV. This is

consistent with spreading new information about the

COVID-19 outburst, followed by the public sharing

of views after a period of information abundance.

In the case of TVI24, most of the emotional

reactions are directed at Other news - “Love”,

“Haha”, and “Angry” (p<.001). The same occurs with

”Comments” (p<.001). Surprise, conveyed by

“Wow”, is the most common reaction to COVID-19

news (p<.05).

In the case of CMTV the only significant

differences found reside in audiences sharing mainly

COVID-19 posts (p<.05), and commenting (p<.001)

and expressing anger (p<.001) on Other news. For the

remaining emotions, there are no statistically

significant differences, as they are expressed towards

both types of news.

It was only for Sic Notícias that COVID-19 news

have expressively modelled the emotional attitude of

audiences. The opposite applies to TVI24, in which

Other news are more reactive. Emotional behaviour

is more disperse in CMTV, with a tendency of

increased verbalisation (“Comments”) and emotional

expression, particularly anger towards Other news.

This is, we believe, totally in line with the

journalistic discourse adopted by each of the news

outlets. For instance, in Figure 1c we observe that

CMTV has an isolated shorter sequence of events on

the news. On the audience side, we notice a

predominance of sharing (spreading) of COVID-19

news/information, which is typical in the stages of

alarmed discovery. For TVI24, the coverage of

COVID-19 news was not so distinct as in the other

outlets, with Other news never being overly

neglected. On the audience side, most of the reactions

tend towards Other news, as well as the “Comments”,

while only the surprise (“Wow”) is mainly expressed

towards COVID-19 news. In Sic Notícias, the most

predominant, persistent, and extended coverage of

COVID-19 news has resulted in a significant

emotional expression of audiences’ love, surprise and

sadness towards this type of news (except for

laughter).

This leads us to ascertain that the media coverage

and journalistic discourse greatly impact the

audiences’ emotions and are provided with the ability

to prolong sadness or joy, hope or frustration,

depression or wellbeing, in any ordinary context, but

especially in periods of crisis when people are more

sensitive.

Given the three distinct emotional profiles of

audiences, we further explored the correlations

among emotions and interactions per news outlet, to

determine how they mutually reinforce each other and

assess their polarity. The following significant

Pearson correlations were found (p<.01).

Sic Notícias

 Moderate: Wow-Sad (r=.572)

 Moderate: Haha-Comments (r=.580)

 Moderate: Angry-Comments (r=.481)

 Weak: Sad-Angry (r=.324)

TVI24

 Strong: Love-Comments (r=.724)

 Strong: Love-Shares (r=.703)

 Moderate: Share-Comments (r=.531)

 Moderate: Sad-Shares (

r=.436)

 Weak: Wow-Sad (r=.375)

 Weak: Angry-Comments (r=.345)

 Weak: Sad-Comments (r=.312)

 Weak: Sad-Angry (r=.275)

 Weak: Haha-Comments (r=.263)

CMTV

 Moderate strong: Angry-Comments (r=.643)

 Moderate: Wow-Shares (r=.588)

 Moderate: Comments-Shares (r=.558)

 Moderate: Sad-Shares (r=.533)

 Moderate: Angry-Shares (r=.530)

 Moderate: Sad-Angry (r=.471)

 Moderate: Sad-Comments (r=.445)

 Weak: Wow-Sad (r=.366)

 Weak: Haha-Comments (r=.342)

 Weak: Wow-Comments (r=.328)

Among the three news outlets, there is a

prevalence of association between negative emotions

(the pairs Sad-Angry and Wow-Sad) and between

these and the highest interaction rates with the news

(comments and shares). Negativity appears as the

overall main engine for interacting with news and

spreading information. There is one exception in

TVI24, in which interaction is also strongly

correlated with positivity regarding Other news (the

pairs Love-Comments and Love-Shares). The pairs

Angry-Comments and Haha-Comments are also

evident among the trio of outlets.

Laughter and surprise, conveyed by the click-

based reactions “Haha” and “Wow” consist of

WEBIST 2021 - 17th International Conference on Web Information Systems and Technologies

192

volatile emotions, as they can acquire distinct polarity

according to other prevalent emotions they are paired

with. For instance, the pair Haha-Love can translate

into passion, affection, friendship, happiness,

amusement, joy and fun.

On the other hand, the pair Haha-Angry can

translate into rage, fury, frenzy, indignation, scorn,

disdain, cynicism, and irony. What we observe in the

above analysis of correlations is the incidence of

these latter types of associations, in which emotional

volatility tends towards negative polarity. We also

believe that this indicates the presence and/or

prevalence of sarcasm, generally defined as content

that attracts both positive and negative feedback

(Hessel & Lee, 2019), or in our case, falling into two

or more classes of emotion, which may or may not be

opposed in terms of polarity.

Since our research at this stage is mainly focused

on revealing cues for profiling the news outlets, we

further explore their controversy profiles, considering

the COVID-19 and Other news.

4.3 Controversy

Following Hessel and Lee (2019) methodology to

determine controversy, we computed the entropy of

the set of Facebook reactions per post, according to

entropy formula shown below, where 𝑥

𝑖

is the number

of each reaction for a post, and 𝑝(𝑥

𝑖

) is the ratio of that

reaction to the total reactions on a post.

𝐻

(

𝑋

)

= − 𝑃

(

𝑥



)

𝑙𝑜𝑔𝑃(𝑥



)





(1)

We consider that if the users’ reactions fall under two

or more emotion classes with high frequencies, the

controversy of a news item is higher; thus, the higher

the entropy, the higher the controversy. Examples are

provided in Table 4, for better clarification.

Table 4: Examples of variation of entropy per post.

Love Wow Haha Sad Angry H

a) 32 0 0 0 0 0

b) 12 12 13 9 9 2.30

c) 26 80 26 62 222 1.85

Users' differing responses indicate that a text is likely

to be controversial, as shown by the high values of

entropy (H), as demonstrated in examples b) and c).

The overall profile of controversy per news outlet,

based on the entropy means is presented in Table 5.

Considering the overall average of entropy for the

full news dataset, and according to our previous

reasoning, Sic Notícias is the entity producing the

news with the least controversial potential (below

average). The news outlet TVI24 has the highest

overall average entropy (.993), followed by CMTV

(.929) and SICNotícias (.795) (F(30604)=303.870;

p<0,001). Both TVI24 and CMTV present above-

average entropy values, and TVI24 leads in the

amount of controversy produced.

Table 5: Overall profile of controversy per news outlet,

based on entropy means.

N Mean SD Max

SICNoticias 17752 .795 .648 2.321

TVI24 8478 .993 .611 2.321

CMTV 4377 .929 .615 2.251

Total 30607 .869 2.321

Our overall entropy average is slightly lower than

the one reported (H=.939) by Basile et al. (2017), who

analysed four Italian newspapers and one news

agency. The Italian newspaper with the highest

average of entropy is Il Gionale (H=1.127), an openly

biased right-wing newspaper. Although this was not

a feature in the detection of sarcasm in the Italian

case, it is curious to notice that the two Portuguese

media outlets with higher entropy averages are also

(not openly) right-wing news outlets, according to the

European Journalism Observatory (Cardoso,

Couraceiro, & Ana, 2019).

This reality, however, might be altered by the

COVID-19 phenomena, as our dataset dates back to

the outburst of the pandemic in Portugal. For this

reason, we believe that it is relevant to analyse the

amount of controversy specifically generated around

COVID-19 news, which we depict in Table 6.

We found statistically significant differences

between the average entropy among the types of news

and news outlets. On average, COVID-19 news have

higher entropy (.895) than Other news (.831) (t

(26112)=8.529;p<0,001).), as depicted in Table 6.

However, since we try to profile the news outlets, we

analysed these differences within their subsets of

news, also included in Table 6.

Table 6: Average entropy per news type and outlet.

Type Outlet N Mean Max MeanTot

COVID-

19 news

SICNotícias

11869

.855 2.321

.895

TVI24

4053 .997 2.311

CMTV

2216 .924 2.252

Other

news

SICNotícias

5883 .674 2.252

.831

TVI24

4425 .989 2.322

CMTV

2161

.934 2.246

A set of independent samples t-Test only confirms

significant differences of entropy between news

categories for Sic Notícias (higher in COVID-19) and

CMTV (higher in Other news), although with no

significant differences for CMTV.

Proﬁling Media Outlets and Audiences on Facebook: COVID-19 Coverage, Emotions and Controversy

193

Still, the overall averages of entropy are relevant

for both categories of news and overall more

prevalent on COVID-19 news, namely when

considering other entropy values reported in the

literature (Basile et al., 2017; Gray, 2020). Thus, we

analysed which Facebook reactions mostly

contributed to the formation of controversy. To do so,

we annotated the dataset considering as

“Controversial” all news with entropy values one

standard deviation above the mean entropy value for

each given news outlet (c.f. Table 5). The results

show significant differences in the average

distribution of Facebook reactions and interactions

per controversial and noncontroversial news (t-Test),

which we depict by news category in Table 7.

Both for COVID-19 and Other controversial

news, the most prevalent reactions, in decreasing

order of average (p<.005):

 “Angry”

 “Haha”

 “Wow”

The remaining emotions, “Sad” (47.36) and “Love”

(11.11) are significantly associated with

noncontroversial news (p<.005).

Table 7: Average of reactions and interactions per

(un)controversial news.

COVID-19 news Other news

Contr. Uncont. Contr. Uncont.

Love 6.35 11.11 8.70 22.43

Wow 13.27 9.74 12.30 7.10

Haha 15.02 4.08 18.79 6.66

Sad 18.22 47.36 17.02 39.91

Angry 22.68 11.79 24.47 19.08

Comments 90.08 36.33 102.81 45.59

Shares 124.78 92.81 81.99 82.95

Considering the interactions with the news,

“Comments” are always substantially higher in

controversial news (p<.001), but the average of

“Shares” is significantly higher for COVID-19

controversial news.

This means that controversy is mainly built upon

negative (“Angry”) and volatile emotions (“Haha”,

“Wow”), which reinforces the notion of irony.

Considering Hessel and Lee (2019) thoughts on

controversy not being necessarily a bad thing, namely

in bringing up a point that warrants a spirited debate

that can improve community health, we believe this

not to be the case. In fact, irony rarely permits the

development of a civilised and constructive debate.

However, this requires, for instance, content analysis

over the comments posted in controversial news for

further elaboration.

We also observe that the COVID-19 controversial

news are the ones harvesting higher “Shares”, i.e.,

they consist of the news with the highest reach and

potential of spread of controversy on social media.

This contradicts Freeman et al. (2020), who state that

content that is more likely to inspire a negative

reaction from users is less likely to be shared. Bellow,

we present the top five COVID-19 news with the

highest number of “Shares”, which curiously were all

posted by CMTV:

 “The heat takes the Portuguese people to the

beaches the day the coronavirus pandemic was

decreed.”

 “School deans urge early Easter holidays to

combat coronavirus.”

 “Disrespect quarantine is punishable by five

years in prison.”

 “761 inmates released since Saturday during

the State of Emergency.”

 “Australian minister claims coronavirus was

created in a lab.”

At this stage of research, not having conducted

effective content analysis, we speculate that we will

find hate speech towards people not complying with

confinement measures (non-compliance), towards the

educational system and/or educational professionals,

towards minorities (criminal offenders) and towards

politics or public figures.

5 CONCLUSIONS

We presented preliminary findings from an ongoing

study of profiling Portuguese media outlets, based on

the media coverage of the COVID-19 outburst

(March-May 2020), the audience emotional

engagement, and the entropy-based emotional

commotion generated, manifested in emotional

controversy.

Our results show three profiles of COVID-19

news coverage: (1) one more consistent (Sic

Notícias), least controversial, with less drastic

fluctuations of attention, which resulted in the

significant emotional expression of audiences’ love,

surprise and sadness; (2) another more diffuse with

approximate levels of attention to COVID-19 news

and Other news (TVI24), which generated higher

emotional commotion among audiences towards

COVID-19 unrelated news; (3) and a more spasmodic

and reactive profile of COVID-19 related and Other

news, which translates into the predominance of

anger among audiences (CMTV).

WEBIST 2021 - 17th International Conference on Web Information Systems and Technologies

194

We have also uncovered a prevalent association

between negative emotions (the pairs Sad-Angry and

Wow-Sad) and between these and the highest

interaction rates with the news, mainly through

comments and shares. We believe that this is a clear

indication that negativity is the main engine for

interacting with news and spreading information.

Finally, we have detected high levels of

controversy among news outlets and among

categories of news. Controversy is more prevalent in

COVID-19 related news and is mostly fostered by

negative and volatile Facebook reactions (“Angry”,

“Haha” and “Wow”)

Controversial COVID-19 news were also the

most shared news on Facebook during the outburst of

the pandemic in Portugal.

These results have implications for media outlets,

social media managers and society at large. The

expedite methods of analysis used in this work

encourage the persistent monitoring of social media

to prevent the large spread of hate speech and

unhealthy mindsets in such a way that is immediately

recognisable by media outlets and people navigating

news content on social media.

This work is not without its limitations. We focus

on presenting preliminary tri-folded findings for

profiling behaviour; thus, we have favoured diversity

over depth in some stances. Future research stages are

set to include the content analysis of the users'

comments to the news providing effective insights on

the nature of the speech surrounding the identified

controversial news.

REFERENCES

Balahur, A. (2013). Sentiment analysis in social media

texts. Paper presented at the Proceedings of the 4th

workshop on computational approaches to subjectivity,

sentiment and social media analysis.

Basile, A., Caselli, T., & Nissim, M. (2017). Predicting

Controversial News Using Facebook Reactions. Paper

presented at the CLiC-it.

Bolsen, T., Palm, R., & Kingsland, J. T. (2020). <?

covid19?> Framing the Origins of COVID-19. Science

communication, 42(5), 562-585.

Bryman, A. (2016). Social research methods: Oxford

university press.

Cardoso, G., Couraceiro, P., & Ana, M. (2019). A esquerda

no parlamento e a direita na televisão? Retrieved from

https://pt.ejo.ch/top-stories/a-esquerda-no-parlamento-

e-a-direita-na-televisao

Carvalho, P., Sarmento, L., Silva, M. J., & De Oliveira, E.

(2009). Clues for detecting irony in user-generated

contents: oh...!! it's" so easy". Paper presented at the

Proceedings of the 1st international CIKM workshop on

Topic-sentiment analysis for mass opinion.

Castells, M. (2004). The network society A cross-cultural

perspective: Edward Elgar.

Cazzolato, M. T., Giuntini, F. T., Ruiz, L. P., Kirchner, L.

d. F., Passarelli, D. A., dos Reis, M. d. J. D., Traina, A.

J. (2019). Beyond Tears and Smiles with ReactSet:

Records of Users’ Emotions in Facebook Posts.

Dalmaso, S. C. (2017). Jornalismo e relevância: o discurso

dos leitores dos jornais de referência no Facebook.

Derks, D., Bos, A. E., & Von Grumbkow, J. (2008).

Emoticons and online message interpretation. Social

Science Computer Review, 26(3), 379-388.

Dori-Hacohen, S., Sung, K., Chou, J., & Lustig-Gonzalez,

J. (2021). Restoring Healthy Online Discourse by

Detecting and Reducing Controversy, Misinformation,

and Toxicity Online. Paper presented at the

Proceedings of the 44th International ACM SIGIR

Conference on Research and Development in

Information Retrieval, Virtual Event, Canada.

https://doi.org/10.1145/3404835.3464926

Downs, A. (1972). Up and down with ecology: The issue-

attention cycle. The public, 462-473.

Ekkekakis, P. (2013). The measurement of affect, mood,

and emotion: A guide for health-behavioral research:

Cambridge University Press.

Ekman, P. (1992). An argument for basic emotions.

Cognition & emotion, 6(3-4), 169-200.

Ekman, P., & Cordaro, D. (2011). What is meant by calling

emotions basic. Emotion review, 3(4), 364-370.

Ferrara, E., & Yang, Z. (2015). Measuring emotional

contagion in social media. PloS one, 10(11), e0142390.

Ferrari, E. (2020). Sincerely Fake: Exploring User-

Generated Political Fakes and Networked Publics.

Social Media+ Society, 6(4), 2056305120963824.

Ferrucci, P. (2020). It is in the numbers: How market

orientation impacts journalists’ use of news metrics.

Journalism, 21(2), 244-261.

Freeman, C., Alhoori, H., & Shahzad, M. (2020).

Measuring the diversity of facebook reactions to

research. Proceedings of the ACM on Human-

Computer Interaction, 4(GROUP), 1-17.

Garcia, D., Kappas, A., Küster, D., & Schweitzer, F.

(2016). The dynamics of emotions in online interaction.

Royal Society open science, 3(8), 160059.

Giuntini, F. T., Ruiz, L. P., Kirchner, L. D. F., Passarelli, D.

A., Dos Reis, M. D. J. D., Campbell, A. T., & Ueyama,

J. (2019). How do i feel? identifying emotional

expressions on facebook reactions using clustering

mechanism. IEEE Access, 7, 53909-53921.

Gray, L. (2020). Gender Bias Detection Using Facebook

Reactions.

Hessel, J., & Lee, L. (2019). Something's Brewing! Early

Prediction of Controversy-causing Posts from

Discussion Features. arXiv preprint arXiv:1904.07372.

Kappas, A. (2017). The psychology of (cyber) emotions

Cyberemotions (pp. 37-52): Springer.

Koval, P., Pe, M. L., Meers, K., & Kuppens, P. (2013).

Affect dynamics in relation to depressive symptoms:

variable, unstable or inert? Emotion, 13(6), 1132.

Proﬁling Media Outlets and Audiences on Facebook: COVID-19 Coverage, Emotions and Controversy

195

Li, X., Zhou, M., Wu, J., Yuan, A., Wu, F., & Li, J. (2020).

Analysing Covid-19 on online social media: Trends,

sentiments and emotions. arXiv preprint

arXiv:2005.14464.

Liu, B. (2012). Sentiment analysis and opinion mining.

Synthesis lectures on human language technologies,

5(1), 1-167.

MacAvaney, S., Yao, H.-R., Yang, E., Russell, K.,

Goharian, N., & Frieder, O. (2019). Hate speech

detection: Challenges and solutions. PloS one, 14(8),

e0221152.

Ofoghi, B., Mann, M., & Verspoor, K. (2016). Towards

early discovery of salient health threats: A social media

emotion classification technique. Paper presented at the

biocomputing 2016: proceedings of the Pacific

symposium.

Oleszkiewicz, A., Karwowski, M., Pisanski, K.,

Sorokowski, P., Sobrado, B., & Sorokowska, A. (2017).

Who uses emoticons? Data from 86 702 Facebook

users. Personality and Individual Differences, 119, 289-

295.

Oliveira, L., Sequeira, A., Oliveira, A., Silva, P., &

Mesquita, A. (2021). Exploring the public reaction to

COVID-19 news on social media in Portugal. arXiv

preprint arXiv:2102.07689.

Organisation, W. H. (2020, 23/09/2020). Managing the

COVID-19 infodemic: Promoting healthy behaviours

and mitigating the harm from misinformation and

disinformation. Retrieved from https://www.who.int/

news/item/23-09-2020-managing-the-covid-19-infode

mic-promoting-healthy-behaviours-and-mitigating-the

-harm-from-misinformation-and-disinformation

Organisation, W. H. (2021). Infodemic. Retrieved from

https://www.who.int/health-

topics/infodemic#tab=tab_1

Pearman, O., Boykoff, M., Osborne-Gowey, J., Aoyagi, M.,

Ballantyne, A. G., Chandler, P., Jiménez-Gómez, I.

(2021). COVID-19 media coverage decreasing despite

deepening crisis. The Lancet Planetary Health, 5(1), e6-

e7.

Poell, T. (2020). Three challenges for media studies in the

age of platforms. Television & New Media, 21(6), 650-

657.

Serrano, M. J. H., Greenhill, A., & Graham, G. (2015).

Transforming the news value chain in the social era: a

community perspective. Supply Chain Management:

An International Journal.

Sriteja, A., Pandey, P., & Pudi, V. (2017, 18-21 Nov. 2017).

Controversy Detection Using Reactions on Social

Media. Paper presented at the 2017 IEEE International

Conference on Data Mining Workshops (ICDMW).

Strong, P. (1990). Epidemic psychology: a model.

Sociology of Health & Illness, 12(3), 249-259.

Su, Z., McDonnell, D., Wen, J., Kozak, M., Abbas, J.,

Šegalo, S., Cai, Y. (2021). Mental health consequences

of COVID-19 media coverage: the need for effective

crisis communication practices. Globalisation and

health, 17(1), 1-8.

Thompson, D., Mackenzie, I. G., Leuthold, H., & Filik, R.

(2016). Emotional responses to irony and emoticons in

written language: evidence from EDA and facial EMG.

Psychophysiology, 53(7), 1054-1062.

Tian, Y., Galery, T., Dulcinati, G., Molimpakis, E., & Sun,

C. (2017). Facebook sentiment: Reactions and emojis.

Paper presented at the Proceedings of the Fifth

International Workshop on Natural Language

Processing for Social Media.

Wang, Y., & Pal, A. (2015). Detecting emotions in social

media: A constrained optimisation approach. Paper

presented at the Twenty-fourth international joint

conference on artificial intelligence.

WEBIST 2021 - 17th International Conference on Web Information Systems and Technologies

196