A Multimodal Workﬂow for Modeling Personality and Emotions to

Enable User Proﬁling and Personalisation

Ryan Donovan, Aoife Johnson, Aine de Roiste and Ruairi O’Reilly

Cork Institute of Technology, Cork, Ireland

Keywords:

Automatic Personality Recognition, Affective Computing, Personality, Five Factor Model, Emotion, Basic

Emotions, User Personalisation, User Proﬁling.

Abstract:

The Personality Emotion Model (PEM) is a workﬂow for generating quantiﬁable and bi-directional mappings

between 15 personality traits and the basic emotions. PEM utilises Affective computing methodology to

map this relationship across the modalities of self-report, facial expressions, semantic analysis, and affective

prosody. The workﬂow is an end-to-end solution integrating data collection, feature extraction, data analysis,

and result generation. PEM results in a real-time model that provides a high-resolution correlated mapping

between personality traits and the basic emotions. The robustness of PEM’s model is supported by the work-

ﬂow’s ability to conduct meta-analytical and multimodal analysis; each state-to-trait mapping is dynamically

updated in terms of its magnitude, direction, and statistical signiﬁcance as data is processed. PEM provides a

methodology that can contribute to long-standing research questions in the ﬁelds of Psychology and Affective

computing. These research questions include (i) quantifying the emotive nature of personality, (ii) minimising

the effects of context variance in basic emotions research, and (iii) investigating the role of emotion sequencing

effects in relation to individual differences. PEM’s methodology enables direct applications in any domain that

requires the provision of individualised and personalised services (e.g. advertising, clinical care, research).

1 INTRODUCTION

The interdisciplinary ﬁeld of Personality science is

concerned with identifying, taxonomizing, and un-

derstanding the key aspects of personality. Person-

ality traits are the foundation for Personality science

(McAdams and Pals, 2006) (John et al., 2008). Traits

are the cognitive, behavioural, emotional, and motiva-

tional characteristics of a person that are stable across

situations (McAdams, 2015). Each person varies

along a continuum on any given trait (e.g. a person

can range from highly introverted, to a cross between

introverted and extraverted, to highly extraverted).

And while a person’s traits can and do change during

their lifetime, the person’s relative position on each

trait’s continuum remains stable (e.g. a highly ex-

traverted child will also be a highly extraverted adult,

even if their absolute levels of extraversion has de-

creased (Little, 2014)). Therefore, personality traits

provide information about the enduring and unique

structure of a person’s psyche relative to other peo-

ple

Personality is hereafter used as a shorthand for the col-

lection of traits a person has. For information about the

There is an increasing demand for individualised

and personalised technology and services that better

realise and meet the idiosyncratic needs of each in-

dividual (Vinciarelli and Mohammadi, 2014b) (Sub-

ramanian et al., 2016). Personality science is a valu-

able tool for meeting this demand, as personality traits

indicate the characteristic way a person thinks, acts,

feels, and desires relative to other people. Personality

enables researchers and professionals to apply client-

centred approaches given the personal characteristics

of their client-base, thereby gaining an advantage over

those that use blanketed approaches. (El Bachari

et al., 2010).

Personality is displayed via a variety of detectable

cues (Mehl et al., 2006). These cues span across mul-

tiple modalities, ranging from central-nervous sys-

tem activity, speech and language patterns, behaviour

across a variety of settings (e.g. consumer behaviour,

workplace behaviour), and self/peer report. To har-

ness the potential of personality in user adaptation,

there is a need to identify cues that (i) reliably indicate

personality across situations and (ii) can be detected

other aspects of personality, see (McCrae and Costa Jr,

1999).

Donovan, R., Johnson, A., de Roiste, A. and O’Reilly, R.

A Multimodal Workﬂow for Modeling Personality and Emotions to Enable User Proﬁling and Personalisation.

DOI: 10.5220/0010203301450152

In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP, pages

145-152

ISBN: 978-989-758-488-6

145

automatically and accurately.

This paper argues that emotions meet both of these

conditions. The relationship between personality and

emotions has been (i) likened to that between the

climate and the weather: “what one expects is per-

sonality [climate], what one observes at any par-

ticular moment is emotion [weather]” (Revelle and

Scherer, 2004). A signiﬁcant body of both psycho-

logical and neuropsychological research supports this

analogy (Wilson et al., 2017) (Davis and Panksepp,

2018). Emotions can be (ii) detected automatically

across multiple modalities (e.g. writing, expressions,

neurophysiological activity, behaviour, etc). There-

fore emotions provide an opportunity for automatic

personality recognition and categorisation research.

The ﬁeld of Affective computing (AC) is well-

positioned to avail of this opportunity. AC is con-

cerned with the development of technological systems

that can automatically detect, process, categorise, and

display human emotion (Picard, 2000). Signiﬁcant

advances in automatic emotion detection have led to

successful AC applications in multiple domains, such

as in healthcare and education (Poria et al., 2017)

(D’Mello et al., 2018). The relationship between

personality and emotions provides an avenue for AC

research to extend its reach. AC methodology en-

ables automatic extraction of emotive-states, which

can be statistically correlated with personality traits,

to gain information about the enduring characteristics

of users.

The goal of the Personality Emotion Modelling

(PEM) workﬂow is to provide an end-to-end solu-

tion that enables researchers to map personality and

emotions via AC methodology at scale. This work-

ﬂow identiﬁes the key components for both person-

ality and emotion that have been validated and are

readily accessible via predeﬁned interfaces. Addition-

ally, this workﬂow is automated in that the appropri-

ate AC methodology for extracting and categorising

emotive-states would be built into the workﬂow, en-

abling researchers to upload their data for analysis.

The analysed data would then be correlated with the

personality-traits of their participant sample and the

resulting state-trait mappings are visualised in terms

of their magnitude, signiﬁcance, and direction.

The remainder of the article is organised as fol-

lows. Section 2 deﬁnes the key psychological phe-

nomena of the workﬂow. Section 3 presents the work-

ﬂow of the model. Section 4 discusses the value of

PEM in contributing to open research questions and

the sustainability of the workﬂow. Section 5 discusses

the potential applications of PEM and provides a di-

rect use case scenario. Section 6 summarises the work

and presents conclusions.

2 PERSONALITY TRAITS AND

THE BASIC EMOTIONS

Personality traits are typical expressions of emo-

tion, cognition, behaviour, and motivation across time

(Costa and McCrae, 1995). Personality traits reli-

ably and accurately predict important life outcomes

concerning health, relationships, and career success

(Soto, 2019). Personality traits represent our unique

general temperament or disposition towards the world

(DeYoung, 2015).

The Five Factor Model (FFM) is considered an

overarching paradigm from which to view Person-

ality science (John et al., 2008). The FFM pro-

vides a taxonomy of personality based on ﬁve broad

traits: Openness to Experience, Conscientiousness,

Extraversion, Agreeableness, and Neuroticism (Mc-

Crae and John, 1992). These ﬁve traits are consid-

ered to be at the highest level of the trait hierarchy

with each factor being an aggregate of multiple lower-

level facets (i.e. a speciﬁc aspect of personality) that

correlate highly with one another (e.g. Agreeable-

ness is made up of facets such as empathetic, co-

cooperativeness, and modesty (Graziano and Tobin,

2009)). This aggregation provides a greater band-

width that strengthens the accuracy of predictions

across situations, but it weakens the accuracy of pre-

dictions in speciﬁc situations (Soto and John, 2017).

A potential solution to this limitation is the provi-

sioning of an intermediate level between the FFM and

their facets. This intermediate level would (i) retain

adequate generalisability to assess cross-situational

psychological consistency (ii) but would also be “nar-

row” enough to enable speciﬁc behavioural predic-

tion. Candidates for such an intermediate level have

been identiﬁed; psychometric and genetic research

have demonstrated that each FFM trait can be broken

down into two further aspects, deﬁned here as “sub-

traits” (DeYoung et al., 2007).

The beneﬁt of using this intermediate level has

been demonstrated in experimental studies (Allen

et al., 2018) (Leon et al., 2017). For example, re-

searchers that previously investigated the relation-

ship between the FFM factor Agreeableness and po-

litical orientation found inconsistent results. How-

ever, research that incorporated the sub-trait hierarchy

demonstrated that the two sub-traits of Agreeableness

- Compassion and Politeness - correlated with left-

wing and right-wing political orientation respectively

(Hirsh et al., 2010). The importance of sub-traits in

identifying new links between personality and life-

outcomes has also been showcased in clinical and ed-

ucational domains (Allen et al., 2018) (Leon et al.,

2017). These are promising indications that sub-traits

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

146

Personality

Traits

O C

E A N

Openness

Intellect

Industriousness

Orderliness

Enthusiasm

Assertiveness

Compassion

Politeness

Withdrawal

Volatility

Figure 1: The Five Factor Model’s ﬁve broad traits (Openness to Experience, Conscientiousness, Extraversion, Agreeableness,

and Neuroticism; OCEAN) and the associated sub-traits as speciﬁed by (DeYoung et al., 2007). The ﬁve broad traits represent

the highest level of the personality trait hierarchy. The foundational level of the trait hierarchy are facets, which aggregate

to form each of the ﬁve broad factors. The sub-traits operate in an intermediate level between these two levels and enable a

higher resolution analysis of an individuals personality.

are an integral component for assessing the inﬂuence

of personality in applied research settings.

Therefore, the PEM workﬂow incorporates the

sub-trait level of personality to (a) enable investiga-

tion of the particular inﬂuence of sub-traits in user-

tailoring and targeting and (b) to assess the claim

that the sub-traits offers a higher level of speciﬁcity

than the FFM without sacriﬁcing generalisability. If

(a) and (b) can be empirically demonstrated, then

this supports the presupposition that this intermedi-

ate level is the next necessary component for future

FFM theories. PEM’s primary focus is to assess both

(a) and (b) in relation to the basic emotions.

2.1 The Basic Emotions

Basic Emotion Theory (BET (Hutto et al., 2018))

states that there exists a set of emotions that are

considered distinct from one another (Ekman, 1999).

These distinct emotions are labelled basic because

they are (i) deeply rooted in subcortical areas of the

brain and have been the most inﬂuenced by evolution

(Ekman and Cordaro, 2011) and (ii) are the founda-

tion for more cognitive and culturally mediated emo-

tions, such as shame or guilt. BET asserts that for

an emotion to be considered basic it should be cou-

pled with highly consistent and distinct psychologi-

cal, behavioural, and physiological patterns (Ekman

and Rosenberg, 1997).

Therefore, emotion is deﬁned here as the subjec-

tive experience of a particular emotion coupled with

physiological and behavioural activity that occurs si-

multaneously (Ekman and Rosenberg, 1997). This

deﬁnition encapsulates both the “raw” subjective feel-

ing of each emotion and the associated activity that

researchers can objectively measure (Ekman, 1999).

PEM considers the following emotions to be the basic

emotions: Anger, Anxiety, Disgust, Fear, Joy, Sad-

ness, Surprise. Donovan et al. (2020) discuss the ra-

tionale behind this selection.

In the basic emotion literature, there is an ongoing

debate about the universality of emotions and whether

emotions have culturally independent and universal

signals (Lewis et al., 2010) (Barrett et al., 2016) (Bar-

rett, 2017) (McDuff et al., 2017). A strong position on

the universality of these emotion signals is not taken

by the authors, as the validity of PEM is not depen-

dent on the apriori assumptions of BET. PEM focuses

on the basic emotions for pragmatic purposes, as the

basic emotions are well-deﬁned and have been thor-

oughly researched. However, PEM is seen as a tool

for providing insights into this debate. The meta-

analytical and multimodal approach provides a means

for robust analysis that can add clarity to this debate.

3 THE PEM WORKFLOW

The primary goal of the PEM workﬂow is to pro-

vide an end-to-end solution for personality-emotion

modelling that results in a live instance. Live in-

stances will provide (a) empirical insight into multiple

open questions surrounding Personality science; (b)

providing a high-resolution understanding of the spe-

ciﬁc emotive nature of personality traits; (c) providing

an analytical dashboard to enable a meta-analysis of

the magnitude and direction of results across research

groups. The varied nature of these insights reﬂects the

multi-purpose uses of the PEM workﬂow.

Two primary groups are envisaged as users of

PEM, namely theoretical researchers and industry

professionals. Researchers will consist mostly of psy-

chology and Affective computing practitioners. Simi-

larly, industry users will consist mostly of psycholog-

ical professionals in various domains (e.g. education)

and those interested in user-proﬁling (e.g. advertising

A Multimodal Workﬂow for Modeling Personality and Emotions to Enable User Proﬁling and Personalisation

147

Figure 2: The PEM Workﬂow.

15 Personality Traits Fear Anger Joy Anxiety Sadness Surprise Disgust

Openness to Experience X X X X X X X

Openness X X X X X X X

Intellect X X X X X X X

Conscientiousness X X X X X X X

Industriousness X X X X X X X

Orderliness X X X X X X X

Extraversion X X X X X X X

Assertiveness X X X X X X X

Enthusiasm X X X X X X X

Agreeableness X X X X X X X

Compassion X X X X X X X

Politeness X X X X X X X

Neuroticism X X X X X X X

Withdrawal X X X X X X X

Volatility X X X X X X X

Figure 3: Analytical dashboard for visualising statistical correlation of state-to-trait mappings for a PEM live instance. Note:

the magnitude and direction indicated by this ﬁgure were generated randomly. For an example of the analytical dashboard

based on empirical research, see (Donovan et al., 2020b) and (Donovan et al., 2020c).

or political campaigning). In both cases, PEM enables

a holistic comprehension of the needs and character-

istics of target populations.

The PEM workﬂow consists of several stages as

depicted in Figure 2:

(i) Participant Pool: A suitable participant pool,

which is dependent on the aims of the speciﬁc study,

is recruited by the user groups for data collection.

(ii) Data Collection:

The PEM workﬂow necessitates two types of data

input:

1. Personality Data Types. The hierarchical trait

model used in PEM requires the use of data from

the Big Five Aspects Scale (BFAS). The BFAS

breaks down the personality of the participant

based the Five Factors and their sub-traits as de-

scribed in Section 2.

2. Emotive Data Types. PEM necessitates data col-

lection from at least one of the emotion modali-

ties. Data collection can either be conducted di-

rectly or taken from already existing data-sets.

The modalities are video capturing, written or

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

148

transcribed text, audio, and self/peer report.

(iii) Feature Extraction: Features are extracted

for emotional content based on the modality of the

data. For example, from video capturing PEM ex-

tracts facial landmarks points that indicate emotional

expressions (e.g. muscle movements around the face,

eyebrows, nose, and mouth). This process is currently

automated by a support vector machine classiﬁcation

trained on two different datasets (Healy et al., 2018).

(iv) Data Analysis: Descriptive statistics are de-

rived from the underlying features to compute the

mean for each basic emotion per person. The data

is also assessed to ensure that it matches parametric

assumptions of the inferential statistical tests.

The processed emotion data is then statistically

correlated via a Pearson R correlational test with the

personality traits of the participants contributing a live

instance. Currently, the PEM method for conducting

correlations is Pearson R. Pearson R measures the lin-

ear relationship between two variables via the R effect

size, ranging from a score of -1 (indicating a complete

negative linear relationship between two variables) to

+1 (a complete positive linear relationship), with 0 in-

dicating no linear relationship (Field et al., 2012).

(v) PEM Model Instance: Figure 3 depicts an

example of the analytical dashboard of a live instance

with correlations between each of the 15 personality

traits and the 7 basic emotions.

The nature of the relationship (positive, negative,

or no relationship) and the strength of that relation-

ship is represented by the colours green, red, and

white. Green indicates a positive relationship, red in-

dicates a negative relationship, and white indicates no

relationship. Darker shades indicate a stronger rela-

tionship. This shading is based on the magnitude of

the Pearson R effect size (Cohen, 2013).

Each live instance dynamically updates the analyt-

ical dashboard as new data is inputted. This enables

live analysis of state-trait mappings in terms of statis-

tical signiﬁcance, magnitude, and direction.

4 DISCUSSION

An experimental study with 38 participants was car-

ried out to validate the PEM workﬂow (Donovan

et al., 2020b). The results of the study demonstrated

that 4 of the 7 basic emotions (Disgust, Joy, Sadness,

and Surprise) considered in PEM correlated signiﬁ-

cantly and substantially with one or several person-

ality traits. Two further basic emotions approached

statistical signiﬁcance (Fear and Anger) in their cor-

relations with personality traits.

Several of the signiﬁcant relationships found be-

tween personality traits and basic emotions either

only (i) emerged at the sub-trait level, or (ii) the mag-

nitude of the relationship was greater at the sub-trait

level than the Five Factor level. This result sup-

ports the need for high-resolution analysis of person-

ality incorporated by PEM, as otherwise these ﬁnd-

ings would have been undetected

4.1 PEM’s Contribution to

Longstanding Research Questions

PEM can provide contributions to several research

questions. These research questions include:

1. What is the emotive nature of personality?

2. How to minimise the effects of context variance

in basic emotion studies?

3. What are the main emotion sequencing effects and

how do they differ with respect to individual dif-

ferences?

4.1.1 What is the Emotive Nature of

Personality?

Outside of a few research studies, e.g. (Donovan

et al., 2020a) and (Gavrilescu, 2015), the relationship

between personality and emotion outside of Extraver-

sion and Neuroticism has been overlooked. This is

unfortunate, because understanding the relationship

between these two phenomena is important, as re-

search that can detect this relationship has the poten-

tial to enable greater user adaptability in both theoret-

ical and practical domains.

PEM addresses this question by explicitly mod-

elling the interaction between personality and emo-

tions across modalities. The results of PEM studies

- once pooled together into an analytical dashboard -

will enable one to draw empirically driven observa-

tions of the emotive nature of personality traits.

4.1.2 How to Minimise the Effects of Context

Variance in Basic Emotion Studies?

The debate regarding the inﬂuence of cultural differ-

ences on the BET warrants further empirical study

and analysis. PEM contributes to this debate by (i)

enabling comparisons between real-time PEM dash-

board instances across modalities. This enables a

comparison between populations from different cul-

tures for differences in emotional expressions. PEM

See also (Donovan et al., 2020c). This preprint com-

pares personality to emotion mappings across the modali-

ties of self-report and facial expressions.

A Multimodal Workﬂow for Modeling Personality and Emotions to Enable User Proﬁling and Personalisation

149

also contributes to this debate by (ii) enabling a mul-

timodal perspective, enabling researchers to assess

emotions based on a broad spectrum of input, includ-

ing voice, facial expressions, language, and self/peer

reports. This contribution (ii) avoids the criticism tar-

geted at BET research, that the BET ﬁeld is over-

reliant on using subjective self-report methodology

for collecting data (Barrett, 2017). Both contributions

represent means of either minimising and/or under-

standing the effects of culturally based variances con-

cerning basic emotions.

4.1.3 What Are the Main Emotion Sequencing

Effects and How Do They Differ with

Respect to Individual Differences?

A common experimental protocol in BET research is

to separate emotional stimuli with a neutral task to

bring participants back to a “baseline” state. This is

done to avoid sequencing effects, where the experi-

ence of one emotion either increases or decreases the

likelihood of experiencing subsequent emotions. For

example, the emotions Anger and Disgust have been

shown to have sequencing effects with one another,

where the experience of one makes the experience of

the other emotion more likely to occur (Salerno and

Peter-Hagene, 2013). However, it is not clear whether

there exist sequencing effects across other basic emo-

tions, and whether personality plays any mediating

role for such effects. PEM provides a platform for

researchers to design future studies to test such hy-

potheses and to assess sequencing effects with respect

to individual differences.

4.2 The Sustainability of the PEM

Workﬂow

The sustainability and reusability of the workﬂow are

differentiated between the two primary user groups -

theoretical researchers (both in AC and Psychology)

and industry professionals.

PEM provides researchers with a workﬂow for

generating, testing, and developing applications to

improve human-computer interaction. For instance,

in semantic analysis research the challenge of distin-

guishing between emotional states in the presence of

sarcasm or jokes is a difﬁcult one. However, per-

sonality has been successfully applied to determine

the meaning of ambiguous statements (Poria et al.,

2016). AC researchers could utilise personality data

to minimise ambiguities in content from client or user

groups (Mehta et al., 2019) resulting in higher accu-

racy.

PEM provides professionals with a workﬂow for

a better understanding of the emotional states or per-

sonality traits of a target audience. For example, to

gauge the personality of a target audience, text from

the audience could be collected. Semantic analysis

can then be conducted to assess the basic emotional

states of the audience, which can then be used to infer

their associated personality traits.

5 PEM USE CASE: A NON

QUESTIONNAIRE METHOD OF

PERSONALITY ASSESSMENT

There are several potential applications for the PEM

workﬂow. One potential application is in the do-

main of personality assessment. The current method-

ology for assessing personality is the use of Likert-

scale questionnaires (Anglim and O’Connor, 2018).

These questionnaires ask participants to rate how well

statements regarding typical patterns of thought, be-

haviour, emotion, and motivation apply to themselves

or another person. The accuracy of questionnaires as

a personality assessment tool indicates that they re-

port quite well on both accounts, but they are not ex-

haustive (Vinciarelli and Mohammadi, 2014a).

There are several deﬁciencies with questionnaires

that inhibit an exhaustive detection of personality:

• Highly accurate personality questionnaires are

time-consuming and monotonous to complete

(e.g. the NEO-FI-R contains over 200 questions

and takes over an hour to complete on average).

Shorter questionnaires can take less than 10 min-

utes to complete, but they are signiﬁcantly less ac-

curate (Anglim and O’Connor, 2018).

• Questionnaires assume a level of cognitive com-

petence that effectively excludes several groups of

people (e.g. young children, people with literary

issues, and cognitive disorders).

• Results also show that repeatedly taking person-

ality questionnaires decreases its accuracy for the

results of that person, a phenomenon known as a

“retest artifact” (Durham et al., 2002).

• Personality questionnaires are a secondary source.

Questionnaires require participants to sit and fo-

cus on a set number of questions for a length

of time, rather than assessing personality “in the

wild” from real-world interactions (Baumeister

et al., 2007).

PEM provides a foundation for non-

questionnaire-based inferences of personality

via state-to-trait mappings. Interested parties can

acquire and analyse data via stages (ii) - (iii) to

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

150

extract information about emotion states. Users can

then use this information along with a state-to-trait

live instance (v-vi) to infer the likely personality

traits of each member of the participant pool, given

their observed emotive states.

There are several advantages to such an approach,

as it would (a) be inclusive to all members of the

population, and (b) it enables continuous personal-

ity assessment based on data acquired from multiple

modalities. The value of assessing personality via the

modalities listed in stage (i) is that such signals can

indicate personality in natural settings (e.g. semantic

analysis of text messages). This would constitute a

valuable personality assessment tool.

6 CONCLUSIONS

The Personality Emotion Model (PEM) work-

ﬂow enables automated and quantiﬁed bi-directional

personality-emotion mappings. PEM speciﬁes a stan-

dardised approach for data collection, specifying the

required data input for personality and emotion to en-

able mappings. PEM identiﬁes methods of feature ex-

traction to identify the emotive content per modality

that is needed for inferential statistical analysis. The

appropriate statistical tests to correlate these phenom-

ena and to generate a model instance is also identiﬁed

in the workﬂow. Overall, PEM provides automation

for many of these aspects, providing a tool for extract-

ing emotive states and correlating them with person-

ality traits that are built-into the workﬂow.

PEM provides several advantages over modern ap-

proaches in personality/emotion research: (i) PEM

utilises a meta-analytical and data-driven approach to

quantify and reduce the impact of sampling/cultural

variance; (ii) PEM provides a methodology that facil-

itates contributions to research questions in the ﬁelds

of Personality science and Affective computing; (iii)

PEM assesses personality at a high-resolution level of

the trait hierarchy increasing the likelihood of captur-

ing statistically signiﬁcant relationships between per-

sonality traits and the basic emotions; (iv) PEM fa-

cilitates multi-purpose applications of results across

research and industrial domains; (v) PEM is ﬂexible,

enabling the model to be used in domains relating to

personality and/or emotions; (vi) PEM is adaptive, in

that the model will be reﬁned as more data is col-

lected; (vii) The results of PEM will be open-sourced

and available online for interested groups to use.

PEM enables practical applications. One po-

tential application is the establishment of a non-

questionnaire method of personality assessment.

Such a method of personality assessment would en-

able researchers to study personality in more natural-

istic settings, using a combination of modalities to ex-

tract information and identify personality in natural

settings.

REFERENCES

Allen, T. A., Carey, B. E., McBride, C., Bagby, R. M., DeY-

oung, C. G., and Quilty, L. C. (2018). Big Five aspects

of personality interact to predict depression. Journal

of Personality, 86(4):714–725. Publisher: John Wiley

& Sons, Ltd.

Anglim, J. and O’Connor, P. (2018). Measurement and

research using the Big Five, HEXACO, and narrow

traits: A primer for researchers and practitioners. Aus-

tralian Journal of Psychology.

Barrett, L. F. (2017). How emotions are made: The secret

life of the brain. Houghton Mifﬂin Harcourt.

Barrett, L. F., Lewis, M., and Haviland-Jones, J. M. (2016).

Handbook of emotions. Guilford Publications.

Baumeister, R. F., Vohs, K. D., and Funder, D. C. (2007).

Psychology as the science of self-reports and ﬁnger

movements: Whatever happened to actual behavior?

Perspectives on Psychological Science, 2(4):396–403.

Publisher: Sage Publications Sage CA: Los Angeles,

CA.

Cohen, J. (2013). Statistical power analysis for the behav-

ioral sciences. Routledge.

Costa, P. T. and McCrae, R. R. (1995). Solid ground in the

wetlands of personality: A reply to Block.

Davis, K. L. and Panksepp, J. (2018). The Emotional Foun-

dations of Personality: A Neurobiological and Evolu-

tionary Approach. WW Norton & Company.

DeYoung, C. G. (2015). Cybernetic big ﬁve theory. Journal

of Research in Personality, 56:33–58.

DeYoung, C. G., Quilty, L. C., and Peterson, J. B. (2007).

Between facets and domains: 10 aspects of the Big

Five. Journal of personality and social psychology,

93(5):880.

D’Mello, S., Kappas, A., and Gratch, J. (2018). The affec-

tive computing approach to affect measurement. Emo-

tion Review, 10(2):174–183. Publisher: SAGE Publi-

cations Sage UK: London, England.

Donovan, R., Cotter, J., deRoiste, A., and O’Reilly, R.

(2020a). Improving Academic Performance Amongst

First Years Computer Science Students Through Goal-

Setting. Donegal. IEEE.

Donovan, R., Johnson, A., deRoiste, A., and O’Reilly, R.

(2020b). Quantifying the Links between Personal-

ity Sub-Traits and the Basic Emotions. In Affective

Computing and Emotion Recognition, Lecture Notes

in Computer Science, Cagliari. Springer.

Donovan, R., Johnson, A., and O’Reilly, R. (2020c). Dif-

ferentiation in Personality-Emotion Mappings Self-

Reported and Automatically Extracted Emotions (pre-

print).

Durham, C. J., McGrath, L. D., Burlingame, G. M.,

Schaalje, G. B., Lambert, M. J., and Davies, D. R.

A Multimodal Workﬂow for Modeling Personality and Emotions to Enable User Proﬁling and Personalisation

151

(2002). The effects of repeated administrations on

self-report and parent-report scales. Journal of Psy-

choeducational Assessment, 20(3):240–257.

Ekman, P. (1999). Basic emotions. Handbook of cognition

and emotion, pages 45–60.

Ekman, P. and Cordaro, D. (2011). What is meant by calling

emotions basic. Emotion review, 3(4):364–370.

Ekman, P. and Rosenberg, E. L. (1997). What the face

reveals: Basic and applied studies of spontaneous

expression using the Facial Action Coding System

(FACS). Oxford University Press, USA.

El Bachari, E., Abdelwahed, E., and El Adnani, M. (2010).

Design of an adaptive e-learning model based on

learner’s personality. Ubiquitous Computing and

Communication Journal, 5(3):1–8.

Field, A., Miles, J., and Field, Z. (2012). Discovering statis-

tics using R. Sage publications.

Gavrilescu, M. (2015). Study on determining the Big-Five

personality traits of an individual based on facial ex-

pressions. pages 1–6. IEEE.

Graziano, W. G. and Tobin, R. M. (2009). Agreeableness.

Healy, M., Donovan, R., Walsh, P., and Zheng, H. (2018). A

machine learning emotion detection platform to sup-

port affective well being. pages 2694–2700. IEEE.

Hirsh, J. B., DeYoung, C. G., Xu, X., and Peterson, J. B.

(2010). Compassionate liberals and polite conserva-

tives: Associations of agreeableness with political ide-

ology and moral values. Personality and Social Psy-

chology Bulletin, 36(5):655–664.

Hutto, D. D., Robertson, I., and Kirchhoff, M. D. (2018). A

new, better BET: rescuing and revising basic emotion

theory. Frontiers in psychology, 9:1217. Publisher:

Frontiers.

John, O. P., Naumann, L. P., and Soto, C. J. (2008).

Paradigm shift to the integrative big ﬁve trait taxon-

omy. Handbook of personality: Theory and research,

3(2):114–158.

Leon, F., Morales, O., and V

ertiz, H. (2017). Person-

ality traits that differentiate Attendants of Higher-

Education Online Courses. Journal of E-Learning and

Knowledge Society, 13:141–148.

Lewis, M., Haviland-Jones, J. M., and Barrett, L. F. (2010).

Handbook of emotions. Guilford Press.

Little, B. (2014). Me, myself, and us: The science of per-

sonality and the art of well-being. Public Affairs.

McAdams, D. P. (2015). The art and science of personality

development. Guilford Publications.

McAdams, D. P. and Pals, J. L. (2006). A new big ﬁve:

Fundamental principles for an integrative science of

personality. American psychologist, 61(3):204.

McCrae, R. R. and Costa Jr, P. T. (1999). A ﬁve-factor the-

ory of personality. Handbook of personality: Theory

and research, 2(1999):139–153.

McCrae, R. R. and John, O. P. (1992). An introduction to

the ﬁve-factor model and its applications. Journal of

personality, 60(2):175–215.

McDuff, D., Girard, J. M., and El Kaliouby, R. (2017).

Large-scale observational evidence of cross-cultural

differences in facial behavior. Journal of Nonverbal

Behavior, 41(1):1–19. Publisher: Springer.

Mehl, M. R., Gosling, S. D., and Pennebaker, J. W. (2006).

Personality in its natural habitat: manifestations and

implicit folk theories of personality in daily life. Jour-

nal of personality and social psychology, 90(5):862.

Mehta, Y., Majumder, N., Gelbukh, A., and Cambria, E.

(2019). Recent trends in deep learning based person-

ality detection. Artiﬁcial Intelligence Review, pages

1–27. Publisher: Springer.

Picard, R. W. (2000). Affective computing. MIT press.

Poria, S., Cambria, E., Bajpai, R., and Hussain, A. (2017).

A review of affective computing: From unimodal

analysis to multimodal fusion. Information Fusion,

37:98–125. Publisher: Elsevier.

Poria, S., Cambria, E., Hazarika, D., and Vij, P.

(2016). A deeper look into sarcastic tweets using

deep convolutional neural networks. arXiv preprint

arXiv:1610.08815.

Revelle, W. and Scherer, K. R. (2004). Personality and emo-

tion.

Salerno, J. M. and Peter-Hagene, L. C. (2013). The in-

teractive effect of anger and disgust on moral outrage

and judgments. Psychological Science, 24(10):2069–

2078.

Soto, C. J. (2019). How replicable are links between person-

ality traits and consequential life outcomes? The Life

Outcomes of Personality Replication Project. Psycho-

logical Science, 30(5):711–727.

Soto, C. J. and John, O. P. (2017). The next big ﬁve inven-

tory (bﬁ-2): Developing and assessing a hierarchical

model with 15 facets to enhance bandwidth, ﬁdelity,

and predictive power. Journal of personality and so-

cial psychology, 113(1):117.

Subramanian, R., Wache, J., Abadi, M. K., Vieriu, R. L.,

Winkler, S., and Sebe, N. (2016). ASCERTAIN:

Emotion and personality recognition using commer-

cial sensors. IEEE Transactions on Affective Comput-

ing, 9(2):147–160. Publisher: IEEE.

Vinciarelli, A. and Mohammadi, G. (2014a). More person-

ality in personality computing. IEEE Transactions on

Affective Computing, 5(3):297–300.

Vinciarelli, A. and Mohammadi, G. (2014b). A survey of

personality computing. IEEE Transactions on Affec-

tive Computing, 5(3):273–291.

Wilson, R. E., Thompson, R. J., and Vazire, S. (2017). Are

ﬂuctuations in personality states more than ﬂuctua-

tions in affect? Journal of Research in Personality,

69:110–123.

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

152