Magnus Johansson and Harko Verhagen
Department of Computer and Systems Sciences, Stockholm University, forum 100, Kista, Sweden
Keywords: Agent architectures and models, Computer game agents.
Abstract: Online worlds are complex places, where we have to know some of the rules of play to engage in the
interaction. These worlds are both inhabited by human players and artificial agents called “non player
characters” (NPCs). This is an article about how online worlds can contain a new level of interaction using
more humanlike NPCs. We propose a new way to describe social interaction in online worlds, where NPCs
are modelled to incorporate some of the traits that are more common to man. We also propose a way of
analysing current NPCs and a way to create more humanlike NPCs that can contribute to a more
unpredictable gaming experience, which seems to be the most promising aspect in the development of
online worlds.
Online games and online worlds have evolved from
the textual worlds of the 70s and 80s known as Multi
user dungeons (MUDs) into graphically rich worlds
that immerse players in these games for long periods
of time. Richard Bartle presents five ages of online
worlds in “Designing virtual worlds”[Bartle, 2003]
and there is reason to believe that we are on the
threshold of a sixth age with the games that are most
popular today. As an example of a highly popular
game world we can use World of Warcraft which
has over 12 million active subscriptions monthly. In
“Designing virtual worlds” a detailed picture is
given of what have been the key ingredients in the
evolution of online worlds. Some of the most
important aspects this far have been the development
on highly detailed graphics and the possibility for
people to connect their computer to the internet, but
what will happen from now? Edward Castronova
claims, “Of all the technological frontiers in world-
building, artificial intelligence (AI) holds the most
promise of change” (Castronova 2006, p. 93) and
this view is shared by Bartle: ”From the point of
view of world design, AI promises great thing. If
virtual worls could be populated by intelligent
NPCs, all manner of doors would open” (Bartle
2003, p. 616). If we are to believe Castronova and
Bartle one possible direction for the evolution of
online worlds is through smarter Non Player
It is a fact that online worlds, digital worlds, or
online games (depending on what we prefer to call
them) are popular and have many active players,
where World of Warcraft is just one example of their
success. It is hard to provide proof for what is the
one reason for these games to be as popular, but
intuitively social aspects is probably the one most
important reason. Humans like to solve problems
together, compete or just hang around. And that is
the beauty of online worlds; they provide a space
with possibilities not present in everyday life.
Our assumption is that the social aspects of
online games can be modelled into these worlds in a
far more complex ways than we see today. NPCs are
the focus of our research, and this is a proposal for
how we could create a new dynamic in online
If you spend enough time in an online world, you
will start to see patterns. You will understand what
kind of game mechanics is important and what affect
it will have on your characters development or your
game-play. You will sooner or later understand what
your possibilities are and what the limitations are.
Johansson M. and Verhagen H..
DOI: 10.5220/0003306903590364
In Proceedings of the 3rd International Conference on Agents and Artificial Intelligence (ICAART-2011), pages 359-364
ISBN: 978-989-8425-41-6
2011 SCITEPRESS (Science and Technology Publications, Lda.)
You will most certainly start to find common traits
in many quests or missions that you get involved in
and that the computer generated entities; usually
called NPCs (Non Player Characters) in the game
are not that complex or smart. They will start acting
in a predictable way and you will find yourself
anticipating much of their behaviour even before
they act.
There are typically two different kinds of NPCs in
online worlds and the following sections give a
further description of friendly and hostile NPCs
2.1 Friendly NPCs
There are many NPCs that assist the player in
MMOGs. Some of them are vendors where players
can buy equipment or repair items. Others distribute
quests for the player, where the quests most of the
time involve killing hostile NPCs and collecting
items that are essential in completing the quest. All
these NPCs have predetermined ways of interacting
with players and they are reduced to the function
that they are programmed to facilitate. They
typically have a scripted dialogue that follow a
storyline for different quests that are tailored to fit
players of a certain level.
A structural problem in most online worlds is
that they are designed to have special places where
players are meant to “socialize”. Most players will
sooner or later go to a city in these games where the
interaction between players is dense, and where
NPCs provide different services. Our point here is
that friendly NPCs could potentially have a more
dynamic role in MMOGs than being vendors or
quest givers.
2.2 Hostile NPCs (Mobs)
In most games, hostile NPCs are typically different
kind of monsters that are either part of a quest or
part of the wilderness outside of the city walls.
There are both villages where many NPCs of the
same type reside, to single NPCs that roam a certain
area. When a player is on a quest that involves
exploring a cave full of hostile NPCs the difference
between encouraging instrumental play or trying to
make every such quest a true adventure lies in how
the NPCs behave. As mentioned above, most NPCs
are fairly static and the ones that display some level
of dynamic behaviour will not change their
behaviour over time. The most dynamic NPCs will
run for help if their “health” reaches a certain
percentage of its maximum health, something that
could be explained as some type of “crisis
response”. Unfortunately NPCs that runs off to get
help do so randomly without even trying to find a
potential helping hand.
The limited dynamic and knowledge of NPCs
contributes to there always being a possibility for
players to easily find a strategy in order to maximize
their gain and minimize the cost of killing hostile
NPCs. If hostile NPCs could refine their tactics
through cooperation and change their behaviour in
response to players’ strategies, they would become
harder to predict. Depending on preferences there is
reason to believe that even the “achievers” from
Bartle’s “Player categorisation” (Bartle 2003),
would find NPCs with dynamic and unpredictable
behaviour a much more interesting counterpart since
it would demand skill and dynamic strategies to
succeed in killing them.
One important consideration is what do we gain by
introducing complex NPCs? Is it just a matter of
computational considerations that has influenced
game developers to hold back on the complexity of
NPCs? Or is it the case that NPCs just have to be
“smart enough” to create an illusion of being entities
that we need certain strategies to outsmart?
2.3 NPCs as Agents
This article will focus on what we believe can be a
solution on how to make NPCs more dynamic and
unpredictable, also providing a possibility for a
deeper interaction between players and NPCs, but in
order to create a different kind of NPCs we need a
way of measuring their present state.
We have chosen to look at NPCs as agents; with
a possibility to model interaction between NPCs in
what closely resembles Normative Multi Agent
Systems (NorMAS). One question that potentially
could pose a problem at this stage is: why the
analogy between NPCs and Agents? We strongly
believe that if we look at NPCs as social agents in
these worlds, we will have the possibility to tailor
their behaviour after the same principles that we
could use to describe player behaviour. If we treat
NPCs and players alike, we introduce a framework
to understand players at the same time that we can
cater to their needs as players when it comes to the
interaction with NPCs. We do not offer any proof or
further arguments that this is the only way to look at
NPCs but in order to create social NPCs we need to
create a possibility for them to adapt to the
population of players they are supposed to interact
ICAART 2011 - 3rd International Conference on Agents and Artificial Intelligence
Given that the behaviour repertoire of NPCs is
limited to just a few different types of almost
automatic reactions, we propose to have a look at
theories about human behaviour. Partly to see in
what way we can make NPCs behave more
“humanlike” and partly to get a feeling for the
important concepts involved.
Many competing theories on human behaviour
exist in (amongst others) psychology such as
theories about human needs, motivational processes,
social comparison theory, social learning theory,
theory of reasoned action, etc. Most of these theories
address a subset of all possible human behaviour and
situations in which this behaviour occurs. A meta-
model can be used to unite a collection of these
theories into one framework. An example of such
meta-model is the Consumat approach (Jager 2000,
Janssen and Jger 2000). The Consumat model
combines in an elegant way many of the leading
psychological theories on human behaviour
categorizes them into a 2*2 matrix based on the
level of need satisfaction (LNS) and behavioural
control (BC) on the one hand, and certainty, type of
needs, and cultural perspective (CP) on the other
hand. Concerning the amount of certainty perceived
by the agent, it is either confident in its decision
making (and thus adopting an individual based
perspective) or uncertain (thus turning towards
others for guidance). If the agent has a high need for
behavioural control and a high level of need
satisfaction it reduces the amount of processing
needed (using automated reactions) while a level on
both results in a need for cognitive processing. This
gives four general strategies humans follow, namely
repetition, deliberation, imitation and social
comparison (see table 1).
The Consumat model offers NPC developers the
opportunity to create a common framework of
concepts to create NPC behaviour and also a
solution for the problem on how to switch between
different behaviour modes. However, since it is a
meta-model based on psychological and social
psychological theories, it does not address social
aspects of behaviour in the way social sciences do. If
we really want to have NPC that behave as human
These theories include amongst others Maslows theory on
human needs, Festingers theory of social comparison, Pavlov
and Skinner operant conditioning theories, Banduras social
learning theory, and decision theories from Simon and Ajzen
amongast others
Table 1: The Consumat model (adapted from (Jager
Automated reactions
(high LNS, high BC)
(low LNS, low BC)
Classical and operant
conditioning theory
Decision and choice
theory, theory of
reasoned / planned
behaviour (attitude
and perceived
Social learning
theory, theory of
normative conduct
Social comparison
theory, relative
deprivation theory,
theory of reasoned
/planned behaviour
(social norm)
like as possible, we need to include a social theory
As in the behavioural sciences, theories are abundant
in the social sciences. Even here a meta-model can
be of help to structure our search. Carley and Newell
(1994) have created a matrix on social behaviour to
understand and explain the sociability and
complexity of agents and to illustrate the differences
between different agents based on a variety of
(mainly) sociological theories. Their goal is to
develop a “Model social agent” (MSA), or an agent
that would be an approximation of a human agent
which can be found in the bottom-right corner of the
matrix depicted in figure 1.
Compared to a “Model social agent”, most NPCs
would be limited both in knowledge and processing
abilities. If we start to look at the knowledge
situation (the x-axis in the matrix) NPCs does not
cooperate or communicate with other NPCs. What is
central for this situation is that if we remove other
agents, that leaves us with the “nonsocial task”
situation of figure 1, which is devoid of social
content. This severely limits the capabilities of
NPCs are a bit harder to locate on the Y-axis. We
have to bear in mind that the environment limits the
Figure 1: Social behaviour matrix (adapted from(Carley and Newell 1994)).
possible actions for an agent, and that the mental
model of the agent enables more complex goals.
Some of the traits of NPCs are similar to the
bounded rational agent, in terms of being rational in
their attempts to achieve their goals, and that their
attention is limited, making it hard for the agent to
process all information in its task environment. But
NPCs lack some of the components of the “bounded
rational agent”, the “cognitive agent” and the
“emotional cognitive agent”. NPCs typically lack a
memory function and that would make NPCs a class
of its own in the matrix. If we consider some of the
most typical scenarios when interacting with NPCs
we can distinguish patterns that are similar to the
behaviour of state machines, where the NPCs
typically behaves accordingly to different kind of
stimuli. Most NPCs however could be described
with some or the traits common to both the
“cognitive agent” and the “bounded rational agent”
and this motivates the placement of NPCs in the
matrix (see figure 1). This spot coincides with the
placement of the Consumat model and the theories
behind it.
It is obvious that there are many components that
NPCs lack in order to display a higher level of
complexity. Even for simple tasks that NPCs cannot
yet perform, a memory function with limited
learning abilities would improve their capacity for
interaction manifold. In order to enrich the NPCs
and thus moving them from Nonsocial task situation
in the Carley and Newell matrix, we need a
possibility for the agents to know more about their
task environment, since this influence the
complexity of an agents goals. In order to create a
social system of agents we need a model to
understand what aspects of interaction we should
build into the system. And finally we need a
mechanism of trust and norms between agents to
make them believable in terms of choosing other
agents to cooperate with.
5.1 Social Aspects and Reputation
When players interact in online worlds there are
certain aspects of their interaction that is easier to
observe. One such thing is what kind of sanctions
that are being used in order to punish players that
break the rules or does not comply with the norms of
that particular group. In (Verhagen and Johansson
2009) some of these mechanisms are studied at some
depth focusing on monetary loss and ostracism of
players that does not comply with the rules.
However much of the interaction in computer
games and online worlds revolves around
ICAART 2011 - 3rd International Conference on Agents and Artificial Intelligence
Figure 2: The road to future NPCs.
reputation as a value for players. Reputation in
online worlds can be something of great importance
when it comes to what groups a player will have
access to. As discussed in (Jacobsson and Taylor
2003), a good reputation or knowing a player with a
good reputation can be the difference of gaining
access to a prestigious guild or being denied
membership. We therefore need to look at reputation
as a value that is important in order for NPCs and
players in their evaluation of which players to
cooperate with.
For better understanding of the steps to take (i.e.
which models and theories to include in agents) we
have projected the future NPC or Model Social NPC
in on our version of Carley and Newells matrix
(figure 2).
We have described the needs for more intelligent
NPCs, analyzed the current behaviour set of NPCs
and proposed a framework to make this behaviour
set more flexible. After this we have proposed a
matrix to add social behaviour to NPCs in order to
create a Model Social NPC.
With the ideas and models suggested above,
there is reason to believe that we could create
dynamic NPCs mimicking player dynamics and
create a totally new experience for players. However
what is still missing is a full implementation of all
the components we argue would be beneficial for the
player experience in Online Worlds for us to know
for sure if our ideas are too ambitious or even
possible. Implementations of reputation system as
mentioned in the Consumat section and even in
other systems such as RePage have been successful
in terms of introducing trust mechanisms between
agents (Sabater et al. 2006) and Normative Multi
agent systems have been implemented on a second
life server (Savarimuthu et al. under publication) our
approach is just slightly more encompassing. But to
answer the real question whether or not NPCs
coordinated by a social system, norms and trust will
create the dynamics we are looking for in computer
games is yet to be answered.
Future work will be focused at developing a
conceptual model with enough detail that makes it
possible to implement the key elements for
improving NPCs. That conceptual model should
incorporate the parts of Parsons social system on a
meta-level, the reputation mechanisms from the
consumat system, a memory function that enables
the NPC to remember and learn from previous
interactions, models dealing with psychological
traits, and we also need to implement a norm
typology to complement the trust mechanism so that
socially not accepted behaviour between NPCs can
be detected and sanctioned. Agents must have means
to communicate, alter and recognize norms as
discussed (Verhagen and Johansson 2009) for norms
to be a communicational tool for agents in the social
Bartle, R., 2003. Designing virtual worlds. New Riders.
Carley, K. M. and Newell, A., 1994. The nature of the
social agent. Journal of Mathematical Sociology, 19
(4): 221—262.
Castronova E., 2006. Synthetic worlds: the business and
culture of online games. University of Chicago Press.
Jacobsson, M. and Taylor, T. L., 2003. The Sopranos
meets Everquest – Social networking in massively
multiplayer online games. Melbourne DAC.
Jager, W., 2000. Modelling consumer behaviour, Ph.D.
thesis, Universal Press, Veenendaal.
Janssen, M. A. and Jager, W., 2000. An integrated
approach to simulating behavioural processes: A case
study of the lock-in of consumption patterns, Journal
of Artificial Societies and Social Simulation, 2 (2).
Online at http://www.soc.surrey.ac.uk/JASSS/
2/2/2.html. Last checked March 30, 2010.
Sabater, I., Paolucci, M., and Conte, R., 2006. Repage:
Reputation and ImAGE Among Limited Autonomous
Partners.: Journal of artificial societies and social
simulation, 9 (2). Online at http://www.soc.surrey.
ac.uk/JASSS/9/2/3.html. Last checked March 30,
Savarimuthu, T., Cranefield, S., Purvis, M. A., Purvis, M.
K. (under publication). Norm identification in multi-
agent societies.
Verhagen, H. and Johansson, M., 2009. Demystifying
Guilds: MMORPG- playing and norms. Proceedings
of DiGRA 2009.
ICAART 2011 - 3rd International Conference on Agents and Artificial Intelligence