A Connexionist Model for Emotions in Digital Agents
Jean-Claude Heudin
Devinci Media Lab, Pôle Universitaire Léonard de Vinci, Paris – La Défense, France
Keywords: Emotion, Affect, Mood, Neural Network, Artificial Creature, Digital Agent.
Abstract: This paper introduces a bio-inspired model of affects for digital agents. This model provides three distinct
layers: emotions as short-term affect, moods as medium-term affect, and personality as a long-term affect. It
describes an implementation based on a connexionist architecture using a dedicated neural network
designed for the “Living Mona Lisa” research project.
1 INTRODUCTION
In this paper we introduce a bio-inspired model of
affects for digital agents. Modeling emotions is a
recurrent problem in the design of artificial
creatures, including virtual characters and robots.
When we ask to anyone what is the difference
between a machine and a human, emotion is always
the answer before any other aspect.
One of our long term projects is to design a
software architecture for believable conversational
agents. We have conducted experimentations in the
past showing that a “multi-personality” approach –
called “schizophrenic” – leads to believable and
complex characters (Heudin, 2011). However, we
have also concluded that an emotion engine is
required to balance between these different
“personalities” resulting in coherent and pertinent
behaviors.
There have been many artificial emotion models
proposed in the past. One of the most complete,
mixing short, medium and long-term aspects of
emotional behaviors, was designed by Gebhard with
ALMA (Gebhard, 2005). We have also proposed a
similar approach with the first version of EVA
(Heudin, 2004). Both approaches were based of the
PAD model (Pleasure, Arousal and Dominance)
proposed by Mehrabian (Mehrabian, 1996).
In this paper, we first describe a new model for
implementing a bio-inspired model of affects in
digital agents. This model is a layered neural
network architecture implementing three levels of
affects: short-term emotions, mi-term moods and
long-term personality.
In the second part of the paper, we describe the
implementation of this model in the “Living Mona
Lisa” research project. The aim of this project is to
design an interactive installation displaying an
animated, high-resolution, full-scale reproduction of
the famous painting of Leonardo da Vinci. This
project is conducted in the spirit of the “Living Art”
approach by a multidisciplinary team including
researchers, artists and students from the Institute of
Internet and Multimedia and Strate School of Design
in Paris. Living Art is a burgeoning field that uses
Artificial Intelligence to create interactive works of
art, bridging digital technologies and more
traditional art forms (Aziosmanoff, 2015).
The paper concludes by showing qualitative
results and discussing the future steps in our
research.
2 THE EMOTION MODEL
2.1 A Layered Model of Affects
We propose here a new layered model of affects
based on three main interacting forms of affects:
Emotion reflects a short-term affect, usually
bound to a specific event, action or object, which is
the cause of this emotion. After its elicitation
emotions usually decay and disappear from the
individual’s focus (Becker, 2001).
Mood reflects a medium-term affect, which is
generally not related with a concrete event, action or
object. Moods are longer lasting stable affective
states, which have a great influence on human’s
cognitive functions (Morris, 1989).
272
Heudin, J-C.
A Connexionist Model for Emotions in Digital Agents.
DOI: 10.5220/0005669202720279
In Proceedings of the 8th International Conference on Agents and Artificial Intelligence (ICAART 2016) - Volume 2, pages 272-279
ISBN: 978-989-758-172-4
Copyright
c
2016 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
Personality reflects long-term affect. It shows
individual differences in mental characteristics
(McCrae, 1992).
2.1.1 Personality
This layer is based on the “Big Five” model of
personality (McCrae, 1992). It contains five main
variables with value varying from 0.0 (minimum
intensity) to 1.0 (maximum intensity). These values
specify the general affective behavior by the traits of
openness, conscientiousness, extraversion,
agreeableness and neuroticism.
Openness is a general appreciation for art,
emotion, adventure, unusual ideas, imagination,
curiosity, and variety of experience. The trait
distinguishes imaginative people from down-to-
earth, conventional people. People who are open to
experience are intellectually curious, appreciative of
art, and sensitive to beauty. They tend to be,
compared to closed people, more creative and more
aware of their feelings. They are more likely to hold
unconventional beliefs. People with low scores on
openness tend to have more conventional, traditional
interests. They prefer the plain, straightforward, and
obvious over the complex, ambiguous, and subtle.
They may regard the arts and sciences with
suspicion, regarding these endeavors as abstruse or
of no practical use. Closed people prefer familiarity
over novelty. They are conservative and resistant to
change.
Conscientiousness is a tendency to show self-
discipline, act dutifully, and aim for achievement.
The trait shows a preference for planned rather than
spontaneous behavior. It influences the way in
which we control, regulate, and direct our impulses.
The benefits of high conscientiousness are obvious.
Conscientious individuals avoid trouble and achieve
high levels of success through purposeful planning
and persistence. They are also positively regarded by
others as intelligent and reliable. On the negative
side, they can be compulsive perfectionists and
workaholics.
Extraversion is characterized by positive
emotions and the tendency to seek out stimulation
and the company of others. The trait is marked by
pronounced engagement with the external world.
Extraverts enjoy being with people, and are often
perceived as full of energy. They tend to be
enthusiastic, action-oriented. In groups they like to
talk, assert themselves, and draw attention to
themselves. Introverts lack the exuberance, energy,
and activity levels of extraverts. They tend to be
quiet, low-key, deliberate, and less involved in the
social world. Their lack of social involvement
should not be interpreted as shyness or depression.
Introverts simply need less stimulation than
extraverts and more time alone.
Agreeableness is a tendency to be
compassionate and cooperative rather than
suspicious and antagonistic towards others. The trait
reflects individual differences in concern with for
social harmony. They are generally considerate,
friendly, generous, helpful, and willing to
compromise their interests with others. Agreeable
people also have an optimistic view of human
nature. They believe people are basically honest,
decent, and trustworthy. Disagreeable individuals
place self-interest above getting along with others.
They are generally unconcerned with others’ well-
being, and are less likely to extend themselves for
other people. Sometimes their skepticism about
others motives causes them to be suspicious,
unfriendly, and uncooperative.
Neuroticism is the tendency to experience
negative emotions, such as anger, anxiety, or
depression. Those who score high in neuroticism are
emotionally reactive and vulnerable to stress. They
are more likely to interpret ordinary situations as
threatening, and minor frustrations as hopelessly
difficult. Their negative emotional reactions tend to
persist for unusually long periods of time, which
means they are often in a bad mood. At the other end
of the scale, individuals are less easily upset and are
less emotionally reactive. They tend to be calm,
emotionally stable, and free from persistent negative
feelings. Freedom from negative feelings does not
mean that low scorers experience a lot of positive
feelings. Frequency of positive emotions is a
component of the Extraversion domain.
2.1.2 Moods
We choose a bio-inspired approach which tries to
mimic the effects of three important monoamine
neurotransmitters involved in the Limbic system.
They are endogenous chemicals that transmit signals
across synapses from neurons to other neurons. The
three virtual neurotransmitters are:
Dopamine (D) is related to experiences of
pleasure and the reward-learning process. It is a
special neurotransmitter because it is considered to
be both excitatory and inhibitory.
Norepinephrine (N) helps moderate the mood
by controlling stress and anxiety. It is an excitatory
neurotransmitter that is responsible for stimulatory
processes.
A Connexionist Model for Emotions in Digital Agents
273
Serotonin (S) is associated with memory and
learning. An imbalance in serotonin levels results in
an increase in anger, anxiety, depression and panic.
It is an inhibitory neurotransmitter.
Let say that the values for these three virtual
neurotransmitters are between 0.0 (minimum value)
to 1.0 (maximum value). They form a three
dimensional mood space called the “Lövheim Cube”
of emotion (Lövheim, 2012). In this model, the three
monoamine neurotransmitters form the axes of a 3D
coordinate system, and the eight basic emotions,
labeled according to the Affect Theory of Silvan
Tomkins (Tomkins, 1991) are placed in the eight
corners (cf. figure 1).
Figure 1: The Lövheim Cube.
This model attempts to organize affects into
discrete categories and connect each one with its
typical response. For example, the affect “joy” is
observed through the display of smiling. There are
eight basic affects (2 positives and 6 negatives)
listed with a low/high intensity label for each affect
and accompanied by its biological expression:
Enjoyment/Joy: smiling lips wide and out.
Interest/Excitement: eyebrows down, eyes
tracking, eyes looking, closer listening.
Surprise/Startle: eyebrows up, eyes
blinking.
Anger/Rage: frowning, a clenched jaw, a red
face.
Contempt/Disgust: the lower lip raised and
protruded head forward and down.
Distress/Anguish: crying, rhythmic sobbing,
arched eyebrows, mouth lowered.
Fear/Terror: a frozen stare, a pale face,
coldness, sweat, erect hair.
Shame/Humiliation: eyes lowered, head
down and averted, blushing.
2.1.3 Emotions
Emotions are very short term affects with relatively
high intensities. They are triggered by inducing
events, which suddenly increase one or more
neurotransmitters:
D is both excitatory and inhibitory and
mainly involved in pleasure/rewards.
N is excitatory and increase active vs. passive
feelings.
S is inhibitory and increase positive vs.
negative feelings.
After a short time, neurotransmitter values decrease
due to a natural decay function. Most of the time, the
system tends to return toward an attractor, which is a
point in the system's phase space. This attractor is
the transposition in the Lövheim Cube of the
personality traits. This is not the neutral mood,
which is by definition in the center of the 3D space:
D = N = S = 0.5
2.1.4 Primordial Emotions
Craig and Denton include pain in a class of feelings
they name, respectively, “homeostatic” (Craig,
2003) or “primordial” emotions (Derek, 2006).
These are feelings such as hunger, thirst and fatigue,
evoked by internal body states, communicated to the
central nervous system by interoceptors, which
motivate behavior aimed at maintaining the internal
milieu at its ideal state. They distinguish these
feelings from the “classical emotions” such as joy,
fear and anger, which are elicited by environmental
stimuli. In our model, we choose to implement two
basic primary emotions:
Energy is the machine transposition of internal
feelings such as hunger, thirst and fatigue.
Pain measures the amplitude of unpleasant
feelings often caused by intense, noxious or
damaging stimuli.
2.2 Implementation
In this section, we describe an approach for
implementing the previous layered model of affects
using a connexionist neural-based architecture. The
resulting implementation is called an Emotion
Engine.
2.2.1 Anna
We choose to implement our own connexionist
javascript-based framework called ANNA:
ICAART 2016 - 8th International Conference on Agents and Artificial Intelligence
274
Algorithmic Neural Network Architecture. This
architecture can be described as a deep highly non-
linear neural network.
More precisely an application can include an
arbitrary number of interconnected networks, each of
them having its own interconnection pattern between
an arbitrary number of layers. Each layer is
composed of a set of neurons.
Each neuron has an arbitrary number of weighted
inputs, a single output, and an operator function that
computes the output given the inputs. This function
can be a classical and homogenous activation
function or any heterogeneous non-linear
programmed function. In other words, each neuron
can be programmed as a dedicated cell. Each
input’s weight can be learned using a machine
learning algorithm, or statically programmed, or
dynamically tuned by another network.
Figure 2: A typical ANNA deep feed-forward network
with a full connection pattern between layers.
This general-purpose architecture enables to
design any types of feed-forward, recurrent or
heterogeneous complex sets of networks.
2.2.2 Emotion Engine
The next figure gives a simplified diagram of the
Emotion Engine. Each square represents a small
network composed of one or more layers. There are
seven networks:
Figure 3: The Emotion Engine architecture.
Input: connect and convert external signals to
the internal representation.
Integration: computes the three DNS virtual
neurotransmitters spike values.
Personality: implements the personality layer
based on the big five traits.
Primary Metabolism: implements the Energy
and Pain system.
Mood Metabolism: computes the current DNS
mood values with a decay function.
Lövheim Cube: convert the DNS mood values
into the eight main affects using a distance
calculation.
Output: select the emerging mood and computes
its level.
Most of the networks use a feed-forward layers
or a simple layer. All neurons have dedicated
programmed operator functions with the exception
of the Integration network, which uses a classical
nonlinear weighted sum and a local supervised back-
propagation learning scheme.
As an example, the Lövheim Cube implements a
Euclidian distance function between the current
DNS mood values and each main affect, that is:
The Emotion Engine is updated by propagating
the inputs using a cyclic trigger called “lifepulse”.
The frequency of this signal ranges from a few
milliseconds to a few seconds depending on the
application.
3 THE LIVING MONA LISA
The first research prototype implementing the model
of affects is the “Living Mona Lisa” installation. The
aim of this project is to create an interactive and
animated reproduction of the famous painting from
Leonard da Vinci in the framework of the Living Art
approach (Aziosmanoff, 2015).
3.1 Architecture Overview
The Living Mona Lisa architecture is based on three
major and straightforward building blocks: the
Sensory Module, the Artificial Intelligence Module
and the Display Module. These three building blocks
are connected together and form with the user(s) an
interacting closed loop (cf. figure 4).
A Connexionist Model for Emotions in Digital Agents
275
Figure 4: The Living Mona Lisa architecture. The sensory
module captures the behaviors of spectators, the AI
module computes Mona Lisa’s emotional state, and the
display module updates Mona Lisa’s emotional
expression.
3.2 Sensors
The sensory module is responsible for sensing the
environment and sending pertinent information to
the AI module. Typical information includes the
presence of one or more persons, their position, their
moves, facial expressions, recognition of some
keywords, noise, etc.
The sensory module is implemented using a
Microsoft Kinect 2 sensor system allowing the
detection of up to six people with advanced facial
tracking (Microsoft, 2014).
3.3 Emotion Engine
The AI module is the central part of the architecture,
implementing the Emotion Engine described in this
paper (cf. figure 5). The Primary Metabolism, that is
Energy and Pain, was not implemented in the first
version of the prototype.
The inputs of the Emotion Engine are connected
to a set of 13 variables coming from the Sensory
Module. The “lifepulse” update rate is set to 20
milliseconds. The “spike” value for each DNS signal
is 0.01.
Mona Lisa’s personality traits are Openness =
0.7; Conscientiousness = 0.8; Extraversion = 0.4;
Agreeableness = 0.6; Neuroticism = 0.1. The
Metabolism returns to this point in the DNS space at
the decay rate of 0.01 per cycle.
The outputs are the following:
The selected emotional expression as a string:
“Neutral”, “Shame”, “Distress”, “Surprise”,
“Disgust”, “Fear”, “Anger”, “Interest”,
“Enjoyment”.
The intensity of this emotion: a float value in
the range 0.0 to 1.0.
The behavior of the eyes regarding the
position of the main user: “Follow”, “Avoid”,
“Fixed”, “Indefinite”. The face can move
with an angle in the range of -50° to +50°
left-right and up-down according to the
normal in the center of the face.
A “blink” trigger for the eyes.
Figure 5: Diagram of the Emotion Engine’s layers.
3.4 Display
The display module embodies the autonomous Mona
Lisa character.
Figure 6: The Living Mona Lisa installation.
The display module is a reproduction of the painting
at its original scale (77 cm x 53 cm not including the
frame). The hardware uses an Ultra High Definition
55’ LCD screen in portrait mode. The image is
ICAART 2016 - 8th International Conference on Agents and Artificial Intelligence
276
composed of a 3D model made of over 500.000
polygons with advanced texturing techniques (cf.
figure 7). We use the Unity Pro 3D rendering engine
for real-time animation of this 3D model (Unity,
2015).
Figure 7: The Living Mona Lisa 3D model.
4 RESULTS
The next figures give some example views of the
final rendering showing different facial expressions.
Figure 8: A close-up to the Living Mona Lisa with a new
emotional expression compared to the genuine painting.
Figure 9/10: Living Mona Lisa showing its original
expression (top) and an “interest” expression combined
with face move and user follow behavior (down).
A Connexionist Model for Emotions in Digital Agents
277
Figure 11/12: Living Mona Lisa showing an “anger”
expression (top) and a “joy” expression combined with
face move and “indefinite” eyes behavior (down).
The Living Mona Lisa prototype was showed to
the public during Futur en Seine exhibition in Paris
organized by Cap Digital during June 2015. First
results were encouraging and show that the system
was successfully functioning in the respect of the
genuine masterpiece and known history of the real
Mona Lisa character. All people interacting with the
system were impressed by the “presence” of Mona
Lisa.
However, there are some limitations that we
must solve in the next version of the prototype. In
the current version, the emotional behaviors are
limited to the face of the character. Thus, we want to
extend the expression to the entire body. For
example the hands could have slightly moves
according to emotions.
In the current version, the Emotion Engine
selects one of the main emotions, one of the edges of
the Lövheim Cube, according to the distance with
the current DNS coordinates. A more realistic
approach could be to express more subtle
combinations of moods: for example one can be at
the same time sad and surprised.
5 CONCLUSIONS
In this paper we introduced a layered model of
affects for digital agents and an implementation as
an Emotion Engine using a connexionist bio-
inspired neuron-based architecture. The model
provides three distinct affects: emotions as short-
term affect, moods as medium-term affect, and
personality as a long-term affect.
The model was implemented in an interactive
installation called “Living Mona Lisa”.
In parallel with the design of a new version of
this “non-verbal” prototype, we also work for
implementing the model in a “multi-personality”
conversational agent in order to obtain a better
balance between the different behaviors according to
the context of the verbal interaction with a user.
ACKNOWLEDGEMENTS
The Living Mona Lisa project is partly funded by
the French Région Île-de-France. We want to thank
Le Louvre Museum and La Réunion des Musées
Nationaux (RMN) for their contribution.
For their participation at every phase of the
Living Mona Lisa project, I would like to thank
Florent Aziosmanoff (author) and Dominique
ICAART 2016 - 8th International Conference on Agents and Artificial Intelligence
278
Sciamma. Also, I would like to thank the Living
Mona Lisa team at the Institute of Internet and
Multimedia (IIM): Marc Bellan, Fabrice Houlné,
Emanuel Perotti, Frédéric Rolland-Porché and all the
students who have contributed to this project.
REFERENCES
Aziosmanoff, F., 2015. Living Art Fondations, CNRS
Editions, Paris.
Becker, P., 2001. Structural and Relational Analyses of
Emotion and Personality Traits. Zeitschrift für
Differentielle und Diagnostische, Psychologie, 22, 3,
155-172.
Craig, A.D., 2003. Interoception: The Sense of the
Physiological Condition of the Body. Current Opinion
in Neurobiology, 13(4), 500-505.
Derek, A.D., 2006. The Primordial Emotions: The
Dawning of Consciousness. Oxford University Press.
Gebhard, P., 2005. ALMA – A Layered Model of Affect.
Proceedings of the Fourth Int. Joint Conference on
Autonomous Agents and Multiagent Systems, 29-36.
Heudin, J.-C., 2004. Evolutionary Virtual Agent.
Proceedings of the Intelligent Agent Technology Int.
Conference (IAT 2004), IEEE/WIC/ACM, 93-98.
Heudin, J.-C., 2011. A Schizophrenic Approach for
Intelligent Conversational Agent, 3rd Int. ICAART
Conference on Agents and Artificial Intelligence.
Lövheim, H., 2012. A New Three-dimensional Model for
Emotions and Monoamine Neurotransmitters, Med
Hypotheses, 78, 341-348.
McCrae, R.R. and John, O.P., 1992. An Introduction to the
Five Factor Model and its Implications. Journal of
Personality, vol. 60, 171-215.
Mehrabian, A., 1996. Pleasure-Arousal-Dominance: A
General Framework for Describing and Measuring
Individual Differences in Temperament, Current
Psychology, vol. 14, 261-292.
Morris, W. N., 1989. Mood: The Frame of Mind. New
York: Springer-Verlag.
Microsoft, 2015. Kinect 2 Manual.
Tomkins, S.S., 1962-1991. Affect Imagery Consciousness,
vol. I-IV, London: Tavistock and New York: Springer.
Unity, 2015. www.unity3d.com.
A Connexionist Model for Emotions in Digital Agents
279