EMOTION AS AN ENABLER OF CO-OPERATION
Martyn Lloyd-Kelly, Katie Atkinson and Trevor Bench-Capon
Department of Computer Science, The University of Liverpool, Ashton Street, Liverpool, U.K.
Keywords:
Emotion, Multi-agent systems, Agent models and architectures, Simulation.
Abstract:
We investigate the emergence of co-operation through emotions. We use agents playing an iterated Prisoner’s
Dilemma game, and show how the emotions of gratitude and anger enable co-operation to emerge as a re-
sponse to the emotional state of the agents without reference to payoffs or history. We investigate the effect of
different thresholds for these emotions to change behaviour on individual performance and system scores.
1 INTRODUCTION
The interplay and differences between rationality and
emotion has long been the subject of philosophi-
cal and psychological debates. Emotions appear
to be particularly significant as determinants of be-
haviour in co-operative relationships (Fessler and Ha-
ley, 2003). We take the view that emotions play a
functional role, following (Frijda, 1987).
Our contribution therefore is to model emotions so
that they are able to play a functional and beneficial
role in allowing agents to determine how to respond to
information received from the environment. We use
agents playing an iterated Prisoners’ Dilemma game
(Axelrod, 1984), and show how the emotions of grat-
itude and anger enable co-operation to emerge as a
response to the emotional state of the agents without
reference to payoffs or history. Please see (Lloyd-
Kelly et al., 2011) for an in-depth discussion of the
work that follows.
The concept of emotion is hotly contested and no
real consensual definition has been found. The idea
that an emotion is characterised by a collection of
physiological responses is noted as early as (James,
1884) but other psychologists such as (Baumeister
et al., 2009) take the view that emotions are functional
in that they provide motivations for future behaviour
and can act as evaluative tools to explain behaviour
that is exhibited in certain situations. We adopt the
functional view, and although we recognise that phys-
iological factors are important, we do not consider
this aspect here.
A logic of individual emotions for agents has been
developed by Steunebrink et al. (e.g (Steunebrink
et al., 2007)). They take small subsets of the 22 emo-
tions defined in the Ortony, Clore and Collins model
(the OCC model (Ortony et al., 1988)) and adopt a
functional view of emotions by prescribing actions
that follow after an emotion has been elicited. We
adopt the idea of implementing small subsets of emo-
tions from this much larger set as it allows a concen-
tration of effort with respect to the emotions chosen,
faithful modelling of these emotions and recognition
that different emotions may have different functional
roles. The OCC is a framework of emotions that
serves as one of the standard psychological frame-
works that is well adapted for use in computer sci-
ence. The applicability of the OCC from a computer-
science standpoint and its extensive use by others is
the primary reason that we chose to use the model.
Our work furthers the functional use of emotions
as behavioural mediators and takes inspiration from
Axelrod’s Prisoner’s Dilemma tournament. One of
the general rules Axelrod suggested for constructing
a successful strategy i.e. one that maximises the indi-
vidual’s score and the total score of the system, states
that strategies should not be overly complex; in some
cases in his tournament, strategies were so elaborate
that they might as well have been acting randomly. It
is this rule that allows us to distinguish between our
emotional agent and a rational agent. Axelrod’s tour-
nament contains rational agents that determine their
behaviour on the basis of past, present and future pay-
offs; in contrast, our emotional agent has no concept
of payoffs, it is a simple reactive agent inspired by
the notions outlined in (Brooks, 1991). Essentially,
the emotional agent’s behaviour is a product of its
emotional character and its current emotional state
with the layer of rationality associated with consider-
ation of past and future payoffs stripped away. This
concept of a continuously updated current emotional
state is what motivates the design of a novel agent
164
Lloyd-Kelly M., Atkinson K. and Bench-Capon T..
EMOTION AS AN ENABLER OF CO-OPERATION.
DOI: 10.5220/0003730101640169
In Proceedings of the 4th International Conference on Agents and Artificial Intelligence (ICAART-2012), pages 164-169
ISBN: 978-989-8425-96-6
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
architecture since other architectures suggest actions
to perform in the present by consulting emotional re-
sponses to the outcomes of actions performed in the
past, when faced with the same situation again.
While rational, self-interested agents can enable
co-operation, (Frank, 1988) argues that such be-
haviour can be self-defeating and emotional individ-
uals are much more likely to establish and main-
tain co-operation. The importance of gratitude in co-
operation is explained by (Berg et al., 1995) who il-
lustrates that altruistic financial loss to an individual
can be tolerated if the other party offers gratitude for
this action. Also, (Fehr and G
¨
achter, 2002) shows that
altruistic punishment resulting from anger is essen-
tial in order for co-operation to flourish. It is impor-
tant to note here that the rate at which anger/gratitude
is elicited may vary and so we introduce the idea of
emotional characters (described in detail in section
2) in order to take into account these various rates.
Consequently, we have chosen to focus our efforts on
implementing gratitude and anger and we investigate
how these emotions can influence the total score of
the system.
2 EXPERIMENTS
The experiments we ran were a set of iterated Pris-
oners’ Dilemma games played as a tournament as in
(Axelrod, 1984). This was realised in the Tileworld
environment (Pollack and Ringuette, 1990). Tile-
world was used because further experiments will ex-
ploit more features of the environment as we move to
consider populations of agents. Emotional agents are
initially defectors (liars) or co-operators (truth-tellers)
and were given two emotions: gratitude and anger. If
the other player co-operates, gratitude is experienced,
and if it defects, anger is experienced. Agents have
different characters corresponding to how intense an
emotion must be before it changes behaviour. Agents
can thus vary in tolerance (the number of defections
required to make anger sufficient to move from co-
operation to defection) and responsiveness (the num-
ber of co-operations required to make gratitude suf-
ficient to give rise to co-operation). Thus, there are
nine possible characters, as shown in table 1.
Payoffs were as standard for the Prisoner’s
Dilemma: dual co-operation rewards both players
with 3 points, dual defection earns 1 point for each
player and asymmetric co-operation/defection earns
the defector 5 points and the co-operator 0 points.
These payoffs are assigned to agents by a mediation
agent. Every round comprised 200 games, although
this was not known to the agents to avoid exploitation
Table 1: Emotional agent character (ch.#) descriptions.
If defecting,
#co-ops required to
co-op.
1 2 3
If co-op,
#defects
required
to defect.
1 Ch.1 Ch.2 Ch.3
2 Ch.4 Ch.5 Ch.6
3 Ch.7 Ch.8 Ch.9
of this information.
The agents used were: an emotional agent; a tit-
for-tat agent (which co-operates on the first round
and then repeats the opponent’s previous move, the
most successful strategy in Axelrod’s tournament); a
mendacious agent, which always defects; a veracious
agent, which always co-operates; a random agent,
which co-operates or defects with equal probability;
a joss agent, which plays tit for tat, but with a 10%
probability of random defection; and a tester agent,
which defects on the first round n, if the opponent co-
operated on round n then the agent co-operates up to
round n + 2 and defects on round n + 3. If, the oppo-
nent defects on round n, the agent plays tit-for-tat for
the rest of the game. The emotional agent began play
first as a co-operator then as a defector for each of the
nine characters in turn.
We wished to explore which emotional characters
would replicate the behaviour of the tit-for-tat strat-
egy (to prove if emotional response can replicate ra-
tionality) and which emotional characters would be
most conducive to success. Success would be con-
sidered both from the perspective of individual agents
and from the perspective of the system as a whole.
3 RESULTS AND DISCUSSION
With respect to the question of whether the tit-for-
tat strategy can be replicated by emotional response,
the results obtained from the Tileworld Dilemma are
clear-cut. An initially co-operative emotional agent
with character 1 replicates the behaviour of the ratio-
nal tit-for-tat agent exactly. To demonstrate this, we
present tables 2 and 3, which contain the average in-
dividual scores of the initially co-operative emotional
agent with character 1 and the tit-for-tat agent ver-
sus the random and joss agents respectively. We have
chosen to only present the results from playing these
two agents as the behaviour of the random and joss
agents is non-deterministic whilst the behaviour of ev-
ery other agent in the simulation is completely deter-
ministic. Therefore, playing against these two agent
EMOTION AS AN ENABLER OF CO-OPERATION
165
types gives the greatest potential for disparateness to
exist between the scores of the emotional agent and
tit-for-tat agent. Consequently, by presenting these
two graphs as evidence we can assert that the be-
haviour of the tit-for-tat agent is exactly replicated by
the emotional agent with the set-up described.
Table 2: Individual scores of intially co-operative emotional
agent with character 1/tit-for-tat agent vs. random agent.
Game Number
Agent 1 2 3 4 5
Emotional 462 466 448 445 424
Tit-for-tat 462 466 448 445 424
Table 3: Individual scores of intially co-operative emotional
agent with character 1/tit-for-tat agent vs. joss agent.
Game Number
Agent 1 2 3 4 5
Emotional 219 213 213 255 242
Tit-for-tat 219 213 213 255 242
As can be seen in tables 2 and 3, the scores of
the emotional agent and tit-for-tat agent exactly over-
lap showing that their behaviour is undisputedly the
same. Explanation of these results is elementary:
whereas the tit-for-tat agent responds to its payoffs,
the emotional agent responds to information sent to
it (as detailed in section 1). Therefore, both agents
react in exactly the same way to inputs that are of dif-
ferent types but which will arise from the same sit-
uations. To clarify, if the tit-for-tat agent observes
that it has scored 0 in a round when it is currently co-
operating or 1 if is it defecting, then it can safely infer
that the opponent is defecting, therefore its behaviour
will switch to defection. Similarly, if the mediator
informs an emotional agent with character 1 that the
opponent has defected, then the emotional agent will
defect immediately in the next round.
We now address the question of whether any other
emotional character set-up is more successful with
respect to maximising the total system score when
playing against periodically defecting strategies than
the set-up previously discussed. To determine this,
we measure success in terms of total system pay-
off or, more specifically, the aggregated average to-
tal system score (the sum of all average total system
scores achieved by an emotional agent). As demon-
strated in table 4, we find that an initially co-operative
agent with character 7 - the most tolerant and most re-
sponsive - offers the greatest aggregated total average
system score, so a more successful strategy does in-
deed exist. To explain this outcome we have iden-
tified three criteria, which must be considered and
Table 4: Intially co-operative emotional agent aggregated
average total system scores.
Character
Aggregated
Average Total
System Score
1
5230.80
2
5069.80
3
4979.80
4
5774.80
5
5241.80
6
5140.80
7
5895.60
8
5328.80
9
5235.80
discussed in turn: fairness, responsiveness and toler-
ance.
We define fairness as the extent to which all mem-
bers of a system are equal; in the context of the Tile-
world Dilemma, the fairest system possible is one
where each agent has an equal score at the end of each
game. Systems that are maximally fair are achieved
by agents who employ strategies that are quick to pun-
ish and reward (as noted by Axelrod). If such a strat-
egy is used by both players and a cycle of defection
is locked into on the first round, then each player’s
score at the end of a game will be 200. Whilst this
is individually fair, the final system score is relatively
low. For a player who wishes to achieve system fair-
ness and maximise the score of each player then the
best possible score that can be achieved is 600, which
is achieved by players immediately locking into a co-
operation cycle on the first round and maintaining this
for a full game. We observe that the only agent pairs
to do this are those that co-operate initially, those that
are quick to punish/defect and those that always co-
operate, no matter what for example:
Initially co-operative emotional agent with any
character and tit-for-tat agent.
Initially co-operative emotional agent with any
character and veracious agent.
However, as mentioned above, such behaviour
does not maximise the system’s score when agents
that seek an advantage, such as the random, tester and
joss agents, are also present (see table 4); from the
system’s view, achievement of a good system score
requires two goals to be achieved:
Co-operation must be established between the
members of the system.
Co-operation must be maintained between the
members of the system.
ICAART 2012 - International Conference on Agents and Artificial Intelligence
166
Table 5: Comparison of the average total scores of an initially co-operative emotional agent of character 7 and an initially
defective emotional agent of character 7.
Opponent
Ini Dis. Mendacious Veracious Random Tit-for-tat Tester Joss
Co-op 409 1200 1002.8 1200 1111 972.8
Defect 400 1199 1001.8 1198 400 968.6
Table 6: Comparison of the average total scores for intially co-operative emotional agents with characters 7, 8 and 9.
Opponent
Character Mendacious Veracious Random Tit-for-tat Tester Joss
7 409 1200 1002.8 1200 1111 972.8
8 409 1200 942 1200 1089 488.8
9 409 1200 902 1200 1036 488.8
The score of a system is increased if agents lock
into cycles of co-operation quickly and break them
slowly. Therefore, readiness to co-operate (respon-
siveness) and being slow to defection (tolerance) are
both important factors. If we compare the average to-
tal scores for an initially co-operative emotional agent
of character 7 to an initially lying emotional agent of
character 7 (see table 5) then the effect of being re-
sponsive becomes clear.
If an agent initially defects, co-operation cycle es-
tablishment is delayed, resulting in lower total system
scores as it becomes more likely that the players will
establish cycles of defection. Conversely, the quicker
an agent is to co-operate and forgive its opponent, the
quicker a co-operation cycle is established. There-
fore, by co-operating initially an agent is more likely
to find concurrent co-operation in a round and estab-
lish a co-operation cycle early in the game which is
important as the number of rounds in a game is finite;
table 6 clearly illustrates this point. The same pat-
tern also holds true for initially defective/co-operative
emotional agents with characters 1-3/4-6.
It is not enough to simply establish a cycle of co-
operation; in order to maximise the score of the sys-
tem then the established co-operation cycle must be
maintained, even when the other player temporarily
defects (as self interested agents will tend to do). If
we consider the scores of emotional agents with char-
acters 1, 4 and 7 displayed in table 4, we observe that
as an agent becomes more tolerant to defections, the
greater the aggregated average total system score be-
comes. If we then consider the individual scores that
are aggregated together for the initially co-operative
emotional agent of character 7 (see table 7) we can see
that character 7 sacrifices system fairness by taking a
reduced score in order to maximise the total system
score. This phenomenon of tolerance is the crucial
difference between character 7 and characters 1 and
4. Therefore, we can see that increased levels of toler-
ance are integral to maximising the total system score,
if playing against agents that periodically defect.
By being tolerant an agent enables the mainte-
nance of a co-operation cycle. Whilst the fairest sys-
tem possible entails the deployment of a strategy that
is quick to reward and quick to punish, such behaviour
breaks co-operation cycles quickly causing lower to-
tal system scores to be achieved. By one agent con-
tinuing to co-operate in the face of defection the sys-
tem scores five rather than two, so that when the de-
fector decides to co-operate again and it is met with
co-operation, a total system score of six is achieved.
A drawback to becoming more tolerant however, is
suffering a reduction in the tolerant agent’s individ-
ual score; table 8 illustrates the extent to which this
occurs.
Table 8 offers some interesting results, especially
if we consider those scores that pertain to the emo-
tional agent playing against the random agent. We ob-
serve that the average individual score of each agent
decreases as tolerance to defection increases, yet as
tolerance is increased the rate at which the average
individual score decreases slows; this can also be ob-
served in figure 1 and table 9. The salient point here
is: when the opponent is not a veracious or tit-for-
tat agent, there is a trade-off between fairness and to-
tal system score. From table 8 we can calculate this
trade-off exactly: for every point earned by the sys-
tem, the emotional agent must lose two points from its
individual score. This raises the question: how much
of a reduction in fairness is acceptable to achieve
these system gains?
It is worth mentioning that the situation is differ-
ent when the initially co-operative emotional agents
with characters 1, 4 and 7 play against a joss agent.
As the emotional agents become more tolerant, the
emotional agent’s average individual score increases
EMOTION AS AN ENABLER OF CO-OPERATION
167
Table 7: Average individual scores of initially co-operative emotional agents with characters 1 and 7.
Opponent
j
Character
i
Mendacious Veracious Random Tit-for-tat Tester Joss
1 199
i
, 204
j
600
i
, 600
j
449
i
, 451
j
600
i
, 600
j
533
i
, 533
j
228.4
i
, 233.4
j
7 197
i
, 212
j
600
i
, 600
j
372.4
i
, 630.4
j
600
i
, 600
j
443
i
, 668
j
449.4
i
, 523.4
j
Table 8: Average individual score of initially co-operative
emotional agents with character 1, 4 and 7 when played
against random, tester and joss agents.
Opponent
Character Random Tester Joss
1 449 533 228.4
4 398.2 465 417.2
7 372.4 443 449.4
Figure 1: Total system score achieved when initially co-
operative emotional agents of characters 1, 4 and 7 play
against a random agent, plotted against the individual score
of the initially co-operative emotional agents.
Table 9: Percentage of total system score owned by the ini-
tially co-operative emotional agents of characters 1, 4 and 7
when playing aginst the random agent.
%Total Score Owned
Character Emotional Random
1 49.9 50.1
4 40.9 59.1
7 37.1 62.9
(see figure 2 and table 8). This is due to the joss
agent’s behaviour, which enables the maintenance of
co-operation cycles in the face of rare, one-off, peri-
odic defections.
In order to determine when the trade-off between
an individual’s score and the system’s score becomes
unacceptable we need to make note of a number
of thresholds. To do this we consider a number
of various maximal and minimal scores that can be
achieved/tolerated for/by each entity in the Tileworld
Dilemma; table 10 below illustrates these values:
The best possible score that an individual agent
can achieve is 1000 whilst the worst is 0, achieved
Figure 2: Total system score achieved when initially co-
operative emotional agents of characters 1, 4 and 7 play
against a joss agent, plotted against the individual score of
the initially co-operative emotional agents.
Table 10: The differing threshold values present in the Tile-
world Dilemma along with how they are derived and their
maximum/minimum values.
Threshold
Value
How
Derived
Max. Min.
Average
Agent 1
Score (A1)
A1
Individual
Score
1000 0
Average
Agent 2
Score (A2)
A2
Individual
Score
1000 0
Average
System
Score
A1 + A2 1200 400
Average
Fairness
Score
A1/A2 1 0
when a mendacious strategy is played against a vera-
cious strategy. An individual score of 0 is the worst
scenario possible; yet, the lowest acceptable score
that can be achieved by a single agent is 200, caused
by two players locking into a defection cycle for a
whole game. The best possible score from the sys-
tem’s perspective is 1200, achieved when two agents
co-operate initially and lock into a co-operation cycle
for a whole game and the worst score is achieved by
two agents locking into a defection cycle for a whole
game, leading to a total system score of 400. The rat-
ing of fairness ranges from 0 to 1, the closer to 1 the
more equal the two player’s scores are.
Therefore, if we take the above discussions into
ICAART 2012 - International Conference on Agents and Artificial Intelligence
168
consideration we can say that an initially co-operative
emotional agent with character 7 is more successful
than an initially co-operative emotional agent with
character 1 due to its ability to quickly establish
and maintain co-operation. Granted, the total sys-
tem scores produced are not fairly distributed: against
a random agent the system/fairness value for an ini-
tially co-operative emotional agent of character 7 is
0.59, whereas for an initially co-operative emotional
agent of character 1 the system/fairness value is 0.99.
Despite this, the system total achieved by an initially
co-operative emotional agent of character 7 is much
higher than that achieved by its less tolerant peers. It
is conceivable that more tolerant agents will produce
greater total system scores at the expense of fairness,
but only until a certain point i.e. when their individual
score passes below the threshold of 200; after this the
trade-off becomes definitely unacceptable since con-
sistent defection produces a better result and there are
no individual gains from co-operating.
4 CONCLUSIONS
Our experiments have demonstrated that the rational
behaviour exhibited by the tit-for-tat strategy present
in (Axelrod, 1984) can be replicated by an initially
co-operative emotional agent with character 1 i.e. an
agent with a low anger threshold resulting in imme-
diate punishment in response to defection and a low
gratitude threshold resulting in immediate reward in
response to co-operation. Furthermore, we have also
shown that when playing against strategies that inter-
sperse co-operation with periodic defection, a degree
of responsiveness and tolerance are key to maximis-
ing the total score of the system. However, by becom-
ing increasingly tolerant and remaining just as respon-
sive, one must expect to suffer a loss with respect to
one’s individual score. Consequently, such altruism is
only demonstrated if it is worthwhile to do so.
We have implemented and begun testing an ex-
tension to the Tileworld Dilemma entitled Emotional
Population. This test-bed consists of a population of
emotional agents who are capable of being initialised
with individual characters in exactly the same way as
described in this paper. The Emotional Population
adds admiration to the existing emotion set consist-
ing of anger and gratitude. Admiration has the poten-
tial to be elicited when an agent’s neighbour obtains
the highest individual score after n number of rounds,
but, as with anger and gratitude, agents have varying
degrees of sensitivity with respect to admiration. If
admiration is elicited then the evaluating agent will
change its emotional character to become more like
the successful agent. Through this new scenario we
aim to analyse which emotional characters become
prevalent in a population and if these emotional char-
acters are successful at maximising the total score of
the system.
REFERENCES
Axelrod, R. (1984). The Evolution Of Cooperation. Basic
Books, Inc.
Baumeister, R. F., DeWall, C. N., Vohs, K. D., and Alquist,
J. L. (2009). Does Emotion Cause Behavior (Apart
from Making People Do Stupid, Destructive Things)?,
chapter 7, pages 119–136. OUP.
Berg, J., Dickhaut, J., and McCabe, K. (1995). Trust, reci-
procity and social history. Games and Economic Be-
haviour, 10:122–142.
Brooks, R. A. (1991). Intelligence without representation.
Artificial Intelligence, 47:139–159.
Fehr, E. and G
¨
achter, S. (2002). Altruistic punishment in
humans. Nature, 415:137–140.
Fessler, D. M. and Haley, K. J. (2003). Genetic and Cul-
tural Evolution of Cooperation. Cambridge, MA:
MIT Press.
Frank, R. H. (1988). Passions Within Reason: The Strategic
Role of the Emotions. W.W. Norton & Company.
Frijda, N. H. (1987). The Emotions. CUP.
James, W. (1884). What is an emotion? Mind, 9:188–205.
Lloyd-Kelly, M., Atkinson, K., and Bench-Capon, T.
(2011). Emotion as an enabler of co-operation. Tech-
nical Report ULCS-11-004, The University of Liver-
pool.
Ortony, A., Clore, G. L., and Collins, A. (1988). The Cog-
nitive Structure of Emotions. CUP.
Pollack, M. and Ringuette, M. (1990). Introducing the tile-
world: Experimentally evaluating agent architectures.
In Proc. of the 8th National Conference on Artificial
Intelligence, pages 183–189. AAAI Press.
Steunebrink, B. R., Dastani, M., and Meyer, J.-J. C. (2007).
A logic of emotions for intelligent agents. In Proc. of
the 22nd Conference on Artificial Intelligence, pages
142–147. AAAI Press.
EMOTION AS AN ENABLER OF CO-OPERATION
169