“WHERE IS MY MIND”- THE EVOLUTION OF NPCS

IN ONLINE WORLDS

Magnus Johansson and Harko Verhagen

Department of Computer and Systems Sciences, Stockholm University, forum 100, Kista, Sweden

Keywords: Agent architectures and models, Computer game agents.

Abstract: Online worlds are complex places, where we have to know some of the rules of play to engage in the

interaction. These worlds are both inhabited by human players and artificial agents called “non player

characters” (NPCs). This is an article about how online worlds can contain a new level of interaction using

more humanlike NPCs. We propose a new way to describe social interaction in online worlds, where NPCs

are modelled to incorporate some of the traits that are more common to man. We also propose a way of

analysing current NPCs and a way to create more humanlike NPCs that can contribute to a more

unpredictable gaming experience, which seems to be the most promising aspect in the development of

online worlds.

1 INTRODUCTION

Online games and online worlds have evolved from

the textual worlds of the 70s and 80s known as Multi

user dungeons (MUDs) into graphically rich worlds

that immerse players in these games for long periods

of time. Richard Bartle presents five ages of online

worlds in “Designing virtual worlds”[Bartle, 2003]

and there is reason to believe that we are on the

threshold of a sixth age with the games that are most

popular today. As an example of a highly popular

game world we can use World of Warcraft which

has over 12 million active subscriptions monthly. In

“Designing virtual worlds” a detailed picture is

given of what have been the key ingredients in the

evolution of online worlds. Some of the most

important aspects this far have been the development

on highly detailed graphics and the possibility for

people to connect their computer to the internet, but

what will happen from now? Edward Castronova

claims, “Of all the technological frontiers in world-

building, artificial intelligence (AI) holds the most

promise of change” (Castronova 2006, p. 93) and

this view is shared by Bartle: ”From the point of

view of world design, AI promises great thing. If

virtual worls could be populated by intelligent

NPCs, all manner of doors would open” (Bartle

2003, p. 616). If we are to believe Castronova and

Bartle one possible direction for the evolution of

online worlds is through smarter Non Player

Characters.

It is a fact that online worlds, digital worlds, or

online games (depending on what we prefer to call

them) are popular and have many active players,

where World of Warcraft is just one example of their

success. It is hard to provide proof for what is the

one reason for these games to be as popular, but

intuitively social aspects is probably the one most

important reason. Humans like to solve problems

together, compete or just hang around. And that is

the beauty of online worlds; they provide a space

with possibilities not present in everyday life.

Our assumption is that the social aspects of

online games can be modelled into these worlds in a

far more complex ways than we see today. NPCs are

the focus of our research, and this is a proposal for

how we could create a new dynamic in online

worlds.

2 CURRENT STATUS OF NPCS

If you spend enough time in an online world, you

will start to see patterns. You will understand what

kind of game mechanics is important and what affect

it will have on your characters development or your

game-play. You will sooner or later understand what

your possibilities are and what the limitations are.

359

Johansson M. and Verhagen H..

“WHERE IS MY MIND”- THE EVOLUTION OF NPCS IN ONLINE WORLDS.

DOI: 10.5220/0003306903590364

In Proceedings of the 3rd International Conference on Agents and Artiﬁcial Intelligence (ICAART-2011), pages 359-364

ISBN: 978-989-8425-41-6

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

You will most certainly start to find common traits

in many quests or missions that you get involved in

and that the computer generated entities; usually

called NPCs (Non Player Characters) in the game

are not that complex or smart. They will start acting

in a predictable way and you will find yourself

anticipating much of their behaviour even before

they act.

There are typically two different kinds of NPCs in

online worlds and the following sections give a

further description of friendly and hostile NPCs

respectively.

2.1 Friendly NPCs

There are many NPCs that assist the player in

MMOGs. Some of them are vendors where players

can buy equipment or repair items. Others distribute

quests for the player, where the quests most of the

time involve killing hostile NPCs and collecting

items that are essential in completing the quest. All

these NPCs have predetermined ways of interacting

with players and they are reduced to the function

that they are programmed to facilitate. They

typically have a scripted dialogue that follow a

storyline for different quests that are tailored to fit

players of a certain level.

A structural problem in most online worlds is

that they are designed to have special places where

players are meant to “socialize”. Most players will

sooner or later go to a city in these games where the

interaction between players is dense, and where

NPCs provide different services. Our point here is

that friendly NPCs could potentially have a more

dynamic role in MMOGs than being vendors or

quest givers.

2.2 Hostile NPCs (Mobs)

In most games, hostile NPCs are typically different

kind of monsters that are either part of a quest or

part of the wilderness outside of the city walls.

There are both villages where many NPCs of the

same type reside, to single NPCs that roam a certain

area. When a player is on a quest that involves

exploring a cave full of hostile NPCs the difference

between encouraging instrumental play or trying to

make every such quest a true adventure lies in how

the NPCs behave. As mentioned above, most NPCs

are fairly static and the ones that display some level

of dynamic behaviour will not change their

behaviour over time. The most dynamic NPCs will

run for help if their “health” reaches a certain

percentage of its maximum health, something that

could be explained as some type of “crisis

response”. Unfortunately NPCs that runs off to get

help do so randomly without even trying to find a

potential helping hand.

The limited dynamic and knowledge of NPCs

contributes to there always being a possibility for

players to easily find a strategy in order to maximize

their gain and minimize the cost of killing hostile

NPCs. If hostile NPCs could refine their tactics

through cooperation and change their behaviour in

response to players’ strategies, they would become

harder to predict. Depending on preferences there is

reason to believe that even the “achievers” from

Bartle’s “Player categorisation” (Bartle 2003),

would find NPCs with dynamic and unpredictable

behaviour a much more interesting counterpart since

it would demand skill and dynamic strategies to

succeed in killing them.

One important consideration is what do we gain by

introducing complex NPCs? Is it just a matter of

computational considerations that has influenced

game developers to hold back on the complexity of

NPCs? Or is it the case that NPCs just have to be

“smart enough” to create an illusion of being entities

that we need certain strategies to outsmart?

2.3 NPCs as Agents

This article will focus on what we believe can be a

solution on how to make NPCs more dynamic and

unpredictable, also providing a possibility for a

deeper interaction between players and NPCs, but in

order to create a different kind of NPCs we need a

way of measuring their present state.

We have chosen to look at NPCs as agents; with

a possibility to model interaction between NPCs in

what closely resembles Normative Multi Agent

Systems (NorMAS). One question that potentially

could pose a problem at this stage is: why the

analogy between NPCs and Agents? We strongly

believe that if we look at NPCs as social agents in

these worlds, we will have the possibility to tailor

their behaviour after the same principles that we

could use to describe player behaviour. If we treat

NPCs and players alike, we introduce a framework

to understand players at the same time that we can

cater to their needs as players when it comes to the

interaction with NPCs. We do not offer any proof or

further arguments that this is the only way to look at

NPCs but in order to create social NPCs we need to

create a possibility for them to adapt to the

population of players they are supposed to interact

with.

ICAART 2011 - 3rd International Conference on Agents and Artificial Intelligence

360

3 THEORIES ON HUMAN

DECISION MAKING

Given that the behaviour repertoire of NPCs is

limited to just a few different types of almost

automatic reactions, we propose to have a look at

theories about human behaviour. Partly to see in

what way we can make NPCs behave more

“humanlike” and partly to get a feeling for the

important concepts involved.

Many competing theories on human behaviour

exist in (amongst others) psychology such as

theories about human needs, motivational processes,

social comparison theory, social learning theory,

theory of reasoned action, etc. Most of these theories

address a subset of all possible human behaviour and

situations in which this behaviour occurs. A meta-

model can be used to unite a collection of these

theories into one framework. An example of such

meta-model is the Consumat approach (Jager 2000,

Janssen and Jger 2000). The Consumat model

combines in an elegant way many of the leading

psychological theories on human behaviour

and

categorizes them into a 2*2 matrix based on the

level of need satisfaction (LNS) and behavioural

control (BC) on the one hand, and certainty, type of

needs, and cultural perspective (CP) on the other

hand. Concerning the amount of certainty perceived

by the agent, it is either confident in its decision

making (and thus adopting an individual based

perspective) or uncertain (thus turning towards

others for guidance). If the agent has a high need for

behavioural control and a high level of need

satisfaction it reduces the amount of processing

needed (using automated reactions) while a level on

both results in a need for cognitive processing. This

gives four general strategies humans follow, namely

repetition, deliberation, imitation and social

comparison (see table 1).

The Consumat model offers NPC developers the

opportunity to create a common framework of

concepts to create NPC behaviour and also a

solution for the problem on how to switch between

different behaviour modes. However, since it is a

meta-model based on psychological and social

psychological theories, it does not address social

aspects of behaviour in the way social sciences do. If

we really want to have NPC that behave as human

These theories include amongst others Maslows theory on

human needs, Festingers theory of social comparison, Pavlov

and Skinner operant conditioning theories, Banduras social

learning theory, and decision theories from Simon and Ajzen

amongast others

Table 1: The Consumat model (adapted from (Jager

2000)).

Automated reactions

(high LNS, high BC)

Reasoned

(re)actions

(low LNS, low BC)

Individual

determined

behaviour

(certainty,

personal

needs,

private

individualist

CP)

REPETITION

Classical and operant

conditioning theory

DELIBERATION

Decision and choice

theory, theory of

reasoned / planned

behaviour (attitude

and perceived

control)

Socially

determined

behaviour

(uncertainty,

social

needs,

egalitarian

CP)

IMITATION

Social learning

theory, theory of

normative conduct

SOCIAL

COMPARISON

Social comparison

theory, relative

deprivation theory,

theory of reasoned

/planned behaviour

(social norm)

like as possible, we need to include a social theory

perspective.

4 THEORIES ON SOCIAL

BEHAVIOUR

As in the behavioural sciences, theories are abundant

in the social sciences. Even here a meta-model can

be of help to structure our search. Carley and Newell

(1994) have created a matrix on social behaviour to

understand and explain the sociability and

complexity of agents and to illustrate the differences

between different agents based on a variety of

(mainly) sociological theories. Their goal is to

develop a “Model social agent” (MSA), or an agent

that would be an approximation of a human agent

which can be found in the bottom-right corner of the

matrix depicted in figure 1.

Compared to a “Model social agent”, most NPCs

would be limited both in knowledge and processing

abilities. If we start to look at the knowledge

situation (the x-axis in the matrix) NPCs does not

cooperate or communicate with other NPCs. What is

central for this situation is that if we remove other

agents, that leaves us with the “nonsocial task”

situation of figure 1, which is devoid of social

content. This severely limits the capabilities of

NPCs.

NPCs are a bit harder to locate on the Y-axis. We

have to bear in mind that the environment limits the

"WHERE IS MY MIND"- THE EVOLUTION OF NPCS IN ONLINE WORLDS

361

Figure 1: Social behaviour matrix (adapted from(Carley and Newell 1994)).

possible actions for an agent, and that the mental

model of the agent enables more complex goals.

Some of the traits of NPCs are similar to the

bounded rational agent, in terms of being rational in

their attempts to achieve their goals, and that their

attention is limited, making it hard for the agent to

process all information in its task environment. But

NPCs lack some of the components of the “bounded

rational agent”, the “cognitive agent” and the

“emotional cognitive agent”. NPCs typically lack a

memory function and that would make NPCs a class

of its own in the matrix. If we consider some of the

most typical scenarios when interacting with NPCs

we can distinguish patterns that are similar to the

behaviour of state machines, where the NPCs

typically behaves accordingly to different kind of

stimuli. Most NPCs however could be described

with some or the traits common to both the

“cognitive agent” and the “bounded rational agent”

and this motivates the placement of NPCs in the

matrix (see figure 1). This spot coincides with the

placement of the Consumat model and the theories

behind it.

5 STEPS TO TAKE

It is obvious that there are many components that

NPCs lack in order to display a higher level of

complexity. Even for simple tasks that NPCs cannot

yet perform, a memory function with limited

learning abilities would improve their capacity for

interaction manifold. In order to enrich the NPCs

and thus moving them from Nonsocial task situation

in the Carley and Newell matrix, we need a

possibility for the agents to know more about their

task environment, since this influence the

complexity of an agents goals. In order to create a

social system of agents we need a model to

understand what aspects of interaction we should

build into the system. And finally we need a

mechanism of trust and norms between agents to

make them believable in terms of choosing other

agents to cooperate with.

5.1 Social Aspects and Reputation

When players interact in online worlds there are

certain aspects of their interaction that is easier to

observe. One such thing is what kind of sanctions

that are being used in order to punish players that

break the rules or does not comply with the norms of

that particular group. In (Verhagen and Johansson

2009) some of these mechanisms are studied at some

depth focusing on monetary loss and ostracism of

players that does not comply with the rules.

However much of the interaction in computer

games and online worlds revolves around

ICAART 2011 - 3rd International Conference on Agents and Artificial Intelligence

362

Figure 2: The road to future NPCs.

reputation as a value for players. Reputation in

online worlds can be something of great importance

when it comes to what groups a player will have

access to. As discussed in (Jacobsson and Taylor

2003), a good reputation or knowing a player with a

good reputation can be the difference of gaining

access to a prestigious guild or being denied

membership. We therefore need to look at reputation

as a value that is important in order for NPCs and

players in their evaluation of which players to

cooperate with.

For better understanding of the steps to take (i.e.

which models and theories to include in agents) we

have projected the future NPC or Model Social NPC

in on our version of Carley and Newells matrix

(figure 2).

6 CONCLUSIONS AND FUTURE

WORK

We have described the needs for more intelligent

NPCs, analyzed the current behaviour set of NPCs

and proposed a framework to make this behaviour

set more flexible. After this we have proposed a

matrix to add social behaviour to NPCs in order to

create a Model Social NPC.

With the ideas and models suggested above,

there is reason to believe that we could create

dynamic NPCs mimicking player dynamics and

create a totally new experience for players. However

what is still missing is a full implementation of all

the components we argue would be beneficial for the

player experience in Online Worlds for us to know

for sure if our ideas are too ambitious or even

possible. Implementations of reputation system as

mentioned in the Consumat section and even in

other systems such as RePage have been successful

in terms of introducing trust mechanisms between

agents (Sabater et al. 2006) and Normative Multi

agent systems have been implemented on a second

life server (Savarimuthu et al. under publication) our

approach is just slightly more encompassing. But to

answer the real question whether or not NPCs

coordinated by a social system, norms and trust will

create the dynamics we are looking for in computer

games is yet to be answered.

Future work will be focused at developing a

conceptual model with enough detail that makes it

possible to implement the key elements for

"WHERE IS MY MIND"- THE EVOLUTION OF NPCS IN ONLINE WORLDS

363

improving NPCs. That conceptual model should

incorporate the parts of Parsons social system on a

meta-level, the reputation mechanisms from the

consumat system, a memory function that enables

the NPC to remember and learn from previous

interactions, models dealing with psychological

traits, and we also need to implement a norm

typology to complement the trust mechanism so that

socially not accepted behaviour between NPCs can

be detected and sanctioned. Agents must have means

to communicate, alter and recognize norms as

discussed (Verhagen and Johansson 2009) for norms

to be a communicational tool for agents in the social

system.

REFERENCES

Bartle, R., 2003. Designing virtual worlds. New Riders.

Carley, K. M. and Newell, A., 1994. The nature of the

social agent. Journal of Mathematical Sociology, 19

(4): 221—262.

Castronova E., 2006. Synthetic worlds: the business and

culture of online games. University of Chicago Press.

Jacobsson, M. and Taylor, T. L., 2003. The Sopranos

meets Everquest – Social networking in massively

multiplayer online games. Melbourne DAC.

Jager, W., 2000. Modelling consumer behaviour, Ph.D.

thesis, Universal Press, Veenendaal.

Janssen, M. A. and Jager, W., 2000. An integrated

approach to simulating behavioural processes: A case

study of the lock-in of consumption patterns, Journal

of Artificial Societies and Social Simulation, 2 (2).

Online at http://www.soc.surrey.ac.uk/JASSS/

2/2/2.html. Last checked March 30, 2010.

Sabater, I., Paolucci, M., and Conte, R., 2006. Repage:

Reputation and ImAGE Among Limited Autonomous

Partners.: Journal of artificial societies and social

simulation, 9 (2). Online at http://www.soc.surrey.

ac.uk/JASSS/9/2/3.html. Last checked March 30,

2010.

Savarimuthu, T., Cranefield, S., Purvis, M. A., Purvis, M.

K. (under publication). Norm identification in multi-

agent societies.

Verhagen, H. and Johansson, M., 2009. Demystifying

Guilds: MMORPG- playing and norms. Proceedings

of DiGRA 2009.

ICAART 2011 - 3rd International Conference on Agents and Artificial Intelligence

364