AGENT-BASED SIMULATION TO SUPPORT DECISION MAKING

IN HEALTHCARE MANAGEMENT PLANNING

∗

Hayden Stainsby

, Manel Taboada

and Emilio Luque

Computer Architecture and Operating Systems Department (CAOS),University Autonoma of Barcelona, Spain

“Tom`as Cerd`a” Computer Science School, University Autonoma of Barcelona, Spain

Keywords:

Healthcare operational management, Agent-based modelling, Individual oriented simulation, Emergency de-

partment, Decision support systems.

Abstract:

Simulation has proved to be a useful tool in healthcare operational management, although up until now does

not appear to have reached its full potential within this area. An area in which simulation is increasingly

useful is as in aiding decision making of healthcare managers when planning constructing new Emergency

Departments or making changes to existing ones. A simulation based on Agent Based Modelling techniques

is proposed with an aim to produce a Decision Support System that takes into account the human and social

factors present within such departments and that can also be generalised to be used in multiple hospitals. Work

on the creation of this model has already began, with many of its concepts and structures presented.

1 INTRODUCTION

Throughout numerous years and a multitude of pub-

lished work, simulation has been shown to be a very

useful tool in healthcare operational management.

Why is it then that it has seemingly failed to rise to

the same level of importance as in other equally large

ﬁelds such as supply chain management and the man-

ufacturing industry. While many papers have been

published on the topic, few have given detailed re-

ports on the completed implementation of the simula-

tion discussed(Fone et al., 2003).

A number of theories have been presented as to

the reason for the current state of simulation in health-

care operational management. Many of these reasons

are social or educational barriers rather than tech-

nical issues, some healthcare modellers have gone

so far as to say that constructing the model may be

the easiest part of implementing simulation in health-

care(Lowery, 1996).

One of the reasons simulation may have failed

to gain a strong foothold in this area is the apparent

dehumanising factor of simulation. Healthcare man-

agers are also trained doctors, and the idea of reduc-

ing patients to numbers in a pool or tokens in a queue

goes strongly against their training.

∗

Supported by the MEC-Spain under contract TIN2007-

64974.

Another reason is the so-called “not invented

here” syndrome(Brailsford, 2007). The vast major-

ity of models described in the literature are created in

conjunction with staff from the relevant department of

a hospital, a practice that is advocated by most if not

all healthcare modellers. However this can leave the

model with a history that may give other institutions

the impression that they’re using a rehashed, second

hand model. This can restrict the spread of what may

be a valid generalised model further than the institu-

tions for which is was originally created.

When it is considered that in many countries

healthcare is nationalised and ultimately all hospitals

are part of a single system under a single strategic

leadership, it would appear unlikely that every one of

these institutions is so different as to require its own

individual model. To be certain, every institution has

its differences, but these differences should be able to

be described as differences in parameters and using

a simulation as a component of a Decision Support

System (DSS) in a new hospital should be a case of

tuning it for its new environment rather than begin-

ning the system analysis from scratch.

Emergency Departments, just like the hospitals

they are within, are under increasing pressure to han-

dle additional patients with the same or a reduced

level of stafﬁng. At the same time both technolog-

ical and organisational changes are being proposed

436

Stainsby H., Taboada M. and Luque E. (2010).

AGENT-BASED SIMULATION TO SUPPORT DECISION MAKING IN HEALTHCARE MANAGEMENT PLANNING.

In Proceedings of the Third International Conference on Health Informatics, pages 436-441

DOI: 10.5220/0002739304360441

 SciTePress

to improve the efﬁciency of these departments, but

implementing these changes can be both costly and

dangerous if their full effects are not completely un-

derstood. One recent study of the impact of techno-

logical changes on the running of an emergency de-

partment(Ayatollahi et al., 2009) showed that while

the effects had been mostly positive, the introduc-

tion of computerised systems into the department had

also caused additional problems, speciﬁcally in the

learning phases and when the computer system ex-

perienced downtime.

This example highlights the need for modelling

and simulation to try and measure the impact of these

kinds of work ﬂow changes in a speciﬁc department

before they are implemented, not just technological

changes but changes to stafﬁng levels, patient num-

bers and arrival distribution, or operational proce-

dures would all be cases where a DSS could be used

to assist in planning for such occurrences. The type

of Decision Support System that can be provided by

computer simulation, and more speciﬁcally by Agent-

Based Simulation, is perfectly suited to balancing the

advantages and disadvantages of proposed changes,

giving healthcare managers a better position from

which to make these important decisions.

2 PRIOR WORK

As mentioned in section 1 there are a number of

examples of the use of simulation in healthcare op-

erational management in the literature. Within this

area the types of models the simulations are based on

can be split into two categories; department focussed

models and hospital or region wide models.

The department focussed models most often use

a technique called Discrete Event Simulation (DES),

which involves simulating only the important events

with a system. This is often characterised by a series

of tokens that wait in turn for a server which process

them before they continue on to the next part of the

system.This method can be particularly well suited to

predicting average waiting times of patients in emer-

gency department settings(Connelly and Bair, 2004).

The hospital or region wide models are more gen-

eralised simulations that focus on the system as a

whole. A popular simulation technique whose use

in healthcare operational management began more re-

cently than DES is System Dynamics (SD), which is

often used to simulate these larger systems. An SD

simulation does not model individuals, but rather the

quantity of patients or resources in pools which repre-

sent combined waiting rooms, hospital beds, or some

other place of rest for that entity. SD simulations can

illustrate how changes in one department can have

negative effects on other, apparently unrelated depart-

ments(Taylor and Dangerﬁeld, 2005).

Another increasingly popular modelling tech-

nique is Agent-Based Modelling (ABM). This tech-

nique is used in biological simulations at both the

macro level - for example the movement of a ﬁsh

school(Gonz´alez et al., 2009) - and the micro level -

cell behaviour(Walker et al., 2004) - as well as in the

social sciences(Cederman, 2002). While ABM shows

a lot of promise in being able to model the individual

aspects of patients and hospital staff, up to this point

the authors have been unable to ﬁnd any reference to

prior work involving the use of ABM in healthcare

operational management.

3 DECISION SUPPORT SYSTEM

Making decisions about the current and future opera-

tions of even a small sized emergency department is

a complicated and ever-changing task. One assistive

measure in these circumstances is a Decision Support

System(Wang et al., 2009). Such a tool, supplied with

the correct input data can provide insight into the in-

ner workings of a real world system that can be difﬁ-

cult, if not impossible to glean using traditional meth-

ods.

A DSS can take many forms, however its purpose

is the same; given information from a current or pro-

posed system and conﬁgured based upon the expec-

tations of the user, a DSS should be able to process

that information and present it in a form which aids

the decision making process. This tool may be com-

prised of a single programatic aid, or a number of

tools working in consort, although in this ﬁnal case an

additional layer may be required to package the infor-

mation from all tools into a single coherent interface.

One example of a tool that is often employed as a

Decision Support System component is simulation. A

simulation, given correct conﬁguration and input data

can allow the user to discover the interrelated effects

of changes in various parts of the system without the

need to try these changes in a live environment, a def-

inite advantage when the live system is so important.

The most effective Decision Support Systems are

those were the users are the ones who gain the most

advantages from the tool being in place.

The measure of the effectiveness of a Decision

Support System is the advantages that it gives to the

user of the system. In the case of a hospital emer-

gency department this would be the manager or de-

partment head who is planning improvements to an

existing department, or possibly even designing a new

AGENT-BASED SIMULATION TO SUPPORT DECISION MAKING IN HEALTHCARE MANAGEMENT

PLANNING

437

department. The goal of such a tool is to allow the in-

dividual to be able to adjust the the input parameters

themselves and receive timely feedback on the effects

of these changes in a form that they can easily inter-

pret.

In order to provide meaningful output, a simula-

tion based DSS must be able to describe a number of

physical and abstract entities. Physical entities can be

divided into two groups, the individuals, or agents,

and the environment in which they interact. Indi-

viduals represent the patients and hospital personnel,

while the environment is the emergency department

itself and possibly operationally adjacent sections of

the hospital. The abstract entities refer to the social

and economic factors that are as just importantas met-

rics such as how many patients can be processed by

the triage department per hour.

A Decision Support System centres around an-

swering questions in order to aid healthcare managers

make the best informed decisions possible. Obviously

asking the correct questions is an important part of the

process, and in most cases these questions can be di-

vided into two groups.

The ﬁrst group are the simplest and require more

interpretation on the part of the user of the DSS. These

are the “what if...?” questions. Answering these ques-

tions is an attempt to divine what will happen to the

system as a whole if one or more changes are made

to the parameters that deﬁne it. In this case examples

may be:

• what if an additional triage nurse is present dur-

ing the busiest period of the day?; or

• what if a ﬂu epidemic raises the number of pa-

tients seeking treatment by 40%?

While a manager may be able to answer some of

these questions based on their own experience, there

is the possibility that these changes will have addi-

tional and unexpected consequences which are more

likely to be revealed by the process of simulation.

The second group of questions are much more

complex, requiring both a better understanding of the

system as well as additional deﬁnitions within the

Decision Support System. These questions involve

the optimisation of the system given certain param-

eters and indices. The additional deﬁnitions in the

DSS are the indices involved, these may be concrete

numbers such as total cost of personnel or may be

much more fuzzy concepts such as patient satisfac-

tion. Once these indices have been deﬁned it is im-

portant to also deﬁne what value for these indices is

considered optimal. Then the user of the DSS may

ask how to optimise the system based on a group of

indices, or how to optimise a given index while keep-

ing another above a certain threshold.

These optimisation tasks can be incredibly time

consuming on the part of the Decision Support Sys-

tem, especially when some of the indices under ex-

amination are represented differently from others. A

system comprising of both quantitative and qualita-

tive indices makes a simple formula based solution

impractical and the required processing power usu-

ally increases dramatically. It is important to take into

consideration that a DSS must provide timely feed-

back to a user.

When seen in the realm of healthcare operational

management the system response time must allow the

user to adjust variables and receive feedback while

they are working, this will allow them to more eas-

ily understand the relation between what they have

changed and the effects that change has on a system.

To achieve this degree of responsiveness when sim-

ulating a large complex system it is quite likely that

High Performance Computing (HPC) techniques, as

provided by Cloud Computing, will form an integral

part of the solution.

Decision Support Systems can be useful tools in

aiding the decision making process, however it is im-

portant that they are easily usable by the people who

are seeking answers from them and also must be able

to produce results within acceptable time frames.

4 MODEL OF AN EMERGENCY

DEPARTMENT

In order to solve the previously discussed problems a

number of strategies are being used to create a general

and validate-able model. Firstly the model is being

constructed in a modular fashion. The model itself

is comprised of a number of smaller sub models that

comprise each of the main arenas of interaction for

the individuals present in the system. The orientation

of these sub models within the larger model is shown

in ﬁgure 1.

Within the emergency department itself there are

four main areas for interaction as shown in ﬁgure 1;

Administration, Triage, Diagnosis / Treatment, and

the Waiting room. There is also a ﬁfth sub model that

represents the laboratory tests, although this need not

be modelled with the same level of precision at this

time as it’s unlikely to involve such complex human-

human interactions, this multi-paradigm model type

is explored further in section 4.1.

Five classes of individuals have been deﬁned.

These ﬁve classes are patients (ρ), companions of

patients (ς), administrative staff (α), nurses (η), and

doctors (δ). The individuals within these classes are

also heterogeneous with respect to one another, which

HEALTHINF 2010 - International Conference on Health Informatics

438

Emergency Department

voluntary exit

data

Legend

ρ - patient

ς - companion of

patient

α - administration

staff

η - nurse

δ - doctor

I( ) - interaction

between individuals

Laboratory

tests

Waiting room

I(ρ, ρ, Wai), I(ρ, ς, Wai)

Main Emergency

Department Entrance

Entrance via

Ambulance

inpatient

admission

Administration

I(ρ, α, Adm); I(ρ, ρ, Adm); I(ς, a, Adm)

Triage

I(η, ρ, Tri), I(ρ, ς, Tri), I(η, ς, Tri)

Diagnosis / Treatment

I(ρ, η, Dia); I(δ, ρ, Dia)

Exit Hospital

Figure 1: Overview of agent interaction stages within an emergency department.

is to say that each individual classiﬁed as a nurse

also has their own properties unique to themselves

amongst other nurses.

Interactions between two individuals is denoted

by the function I(x, y), where x and y represent the

class of individual. The order is meaningful in the

case of the interaction, the individual stated ﬁrst (x in

this example) is the initiator of the interaction. So an

interaction between a doctor and a patient, initiated

by the doctor would be denoted by I(δ, ρ).

The environment where an interaction takes place

may well change the form and outcome of the interac-

tion. When describing an interaction within the entire

emergencydepartment as a whole we extend the inter-

action function to include a location attribute as well,

I(x, y, hloci). To extend our previous example of an

interaction between a doctor and a patient and specify

that it is occurring within the diagnosis and treatment

sub model we would deﬁne it like this, I(δ, ρ, Dia).

It is important at this stage to remember that what

is being represented is the interactions between real

people in real environments, a less technical descrip-

tion of the function deﬁned here would be to say:

A doctor approaches a patient waiting in a

bed in the diagnosis area to begin a discussion

of the level of pain they are experiencing due

to what appears to be a broken leg.

The extra information given here is not lost in the

model, nor the representation in function form, it is

provided within the agents. It is an attribute of the

patient that they have a broken leg and an attribute

of the doctor that they are approaching the patient in

order to discuss the level of pain resulting from this

injury. Further details of the models of individuals

are discussed in section 5.

4.1 Modular Design and the

Multi-paradigm Approach

As stated previously, the model of the emergency de-

partment is made up of a number of smaller sub mod-

els. These design structure provides a number of ad-

vantages. The ﬁrst is in breaking down the problem

into smaller and more manageable sections. Each sec-

tion has been separated out based on the interactions

involved within it, more speciﬁcally which hospital

staff are involved. While all staff will have a gener-

alised idea of how the emergency department works

the bulk of their knowledge is necessarily focussed

on their own area. By dividing the sub models in

this way the process of system analysis is simpliﬁed

because staff from different areas can be interviewed

separately, constructing each sub model in turn rather

than trying to build everything at once.

This scheme of building each sub model sepa-

rately also offers the advantages of unit testing, a pop-

ular method for separately verifying individual mod-

ules of code used in software engineering. Each sub

model is built and then tested using sample or sta-

tistical data to verify that it is accurate within itself.

This greatly reducing the time spent hunting for erro-

neous parts of the model if the whole model output is

not what is expected once all the separate sub models

have been attached to one another.

The third advantage of using a modular design for

the model is the ability to create a multi-paradigm

model in order to take advantage of modelling tech-

niques that best suit each part of the system(Stainsby

et al., 2009). The laboratory tests are a good example

of where this brings an advantage. Because the focus

of the model is the interactions the patient has while

AGENT-BASED SIMULATION TO SUPPORT DECISION MAKING IN HEALTHCARE MANAGEMENT

PLANNING

439

in an emergency department the laboratory is lacking

in any signiﬁcant form of interaction. From a patient’s

point of view, a sample is taken and at some pointlater

it will be ready for collection. Each of these samples

could be modelled as an individual, but this is really

an unnecessary use of resources given that a labora-

tory sample has neither personality nor memory. In

this case it may make much more sense to create a

sub model deﬁned by tokens waiting in a queue for

their turn to be processed, the output of this is then re-

turned to another part of the model where it is likely

to invoke the action of an individual.

Each sub-model has various attributes and outputs

which together form the larger picture of the reality of

an Emergency Department. Therefore the sub-models

may be made up of economic, logistic, and social

models, as well as other information required to solve

the optimisation questions of the DSS.

4.2 Social and Human Behavioural

Factors

Social factors are an important part of all human in-

teraction. A person may react differently to the ex-

act same situation depending on their current state of

mind, the environment they’re in or a range of other

factors. Many simulations do not take human factors

into account, this can lead to results that are unable to

fully represent all parts of an emergency department

that are effected by human decisions.

Research has shown that social and behavioural

factors can play an important role patient decision

making with regard to healthcare, from as early in the

process as whether or not they choose to seek out care

to begin with(Brailsford et al., 2006). The complex

social and psychological factors that drive the desires

and beliefs of people are made many times more com-

plex when they interact with others around them, it is

these interactions that are important during the time a

patient spends in an emergency department.

By using an agent-based model human behaviour

can be modelled at the individual level, rather than

the combination and average of all human behaviour

across the simulation. This becomes more important

when the interactions at one stage may effect the man-

ner in which an individual acts in another stage, pos-

sibly creating behaviour changing feedback loops.

4.3 Generality of the Model

One of the possible reasons for the lack of mainstream

take-up of simulation as a decision support tool in

hospital emergency departments discussed previously

is the idea that a model built for one department is not

applicable to another. For this reason the model being

developed is being done so in concert with more than

one partner hospital. This has a two fold effect on the

same issue.

The ﬁrst is that by working with more than one de-

partment the differences and similarities will quickly

become obvious, suggesting which parts of the model

may be given default values and which need to be im-

plemented as parameters so as to be easily change-

able by the user at a given site. When it is said that

some parts may be given default values, that may be

easily changed in the future to accommodate different

emergency departments in different hospitals. When

it comes to creating the simulator this is already good

programming practice and should not cause too many

problems.

The effect is that the model can be shown to be

applicable to more than one department from the be-

ginning of its lifecycle. This reduces the perceived

ownership and single use nature of the model just by

it being used by multiple departments at the beginning

of its life cycle.

5 MODELS OF AN INDIVIDUAL

While the modular nature of the model described in

the previous section creates an architecture of mod-

els within a model, the use of Agent-Based Modelling

creates yet another level. When using ABM each indi-

vidual is essentially an instance of a model. The num-

ber of models used to describe the agents and their

complexity depends on the level of detail required.

Each individual may be modelled using almost

any modelling or control technique available, from

simple algorithmic or boolean decision making to

methods often not seen in modelling such as sub-

sumption architecture. In fact it is quite possible

to create individuals that are in fact controlled by

their own Agent-Based Models, although logically

this cannot go on forever.

In the model described in section 4, the actions of

each individual are controlled by a Finite State Ma-

chine (FSM). The form of each FSM is dependent on

the class of the individual (patient, companion of a

patient, administrative staff, nurse, or doctor). Within

each of these ﬁve classes the behaviour of an indi-

vidual is determined by certain attributes which are

assigned to them, giving each one their own individu-

ality.

These models of individuals also govern the man-

ner in which individuals interact, both with other in-

dividuals and with the environment in which they re-

side.

HEALTHINF 2010 - International Conference on Health Informatics

440

In the same manner as the indices used to deﬁne

aspects of the real world system when using a Deci-

sion Support System, the attributes of an individual

may not all be of the same type, nor may they all

be static during the course of the simulation. Some

values may be numeric in nature, such as how much

blood a patient has lost, others may require fuzzy

states to more accurately represent naturally occur-

ring, non-numeric conditions such as human concen-

tration.

These factors can be deﬁned as either mutable or

immutable attributes, representing factors that may or

may not change during the course of the simulation

respectively. Mutable attributes may represent such

factors as the level of pain or stress experienced by

an individual at a given moment or the short term

memory of an individual with regard to their expe-

riences within the simulated period. Immutable at-

tributes may represent the level of training a nurse or

doctor has, or the individuals ability to communicate

based on their competence in the local language, these

form the properties of the agent which are unlikely to

change during the course of the simulation.

6 CONCLUSIONS

In the ﬁeld of healthcare operational management it

has been shown that simulation can be a useful tool.

A number of examples exist in the literature, although

the lack of results from complete implementations

shows that simulation in this area has not reached its

full potential.

A number of requirements for a general and ver-

iﬁable simulation are presented. Starting from these

requirements a new model has been created speciﬁ-

cally to work towards creating a tool that can be used

as a basic component of a Decision Support System

which will be easily usable to healthcare managers

to gain insight into the inner workings of current or

planned emergency departments.

A key feature of this project is the use of Agent-

Based Modelling techniques to both take into account

the important social and psychological factors that

come into play during human interactions in high

stress environments such as emergency departments,

but also to aid to reducing the dehumanising appear-

ance of simulation tools from the point of view of

healthcare managers.

The work on this model continues to proceed

based on ongoing investigation and partnership with

a number of hospitals.

ACKNOWLEDGEMENTS

We would like to thank Montse Edo for the detailed

and invaluable assistance provided in describing the

operations of the triage department where she works.

Without the help afforded, this work would not have

been possible.

REFERENCES

Ayatollahi, H., Bath, P. A., and Goodacre, S. (2009). It

in the emergency department - what is the impact of

technology? Proceedings of the International Con-

ference of Health Informatics (HEALTHINF), pages

454–457.

Brailsford, S. (2007). Tutorial: Advances and challenges

in healthcare simulation modeling. Proceedings of the

Winter Simulation Conference, pages 1436 – 1448.

Brailsford, S., Sykes, J., and Harper, P. (2006). Incorpo-

rating human behavior in healthcare simulation mod-

els. Proceedings of the Winter Simulation Conference,

pages 466–472.

Cederman, L. (2002). Endogenizing geopolitical bound-

aries with agent-based modeling. Proceedings of the

National Academy of Sciences of the United States of

America, 9(3):7296–7303.

Connelly, L. and Bair, A. (2004). Discrete event simu-

lation of emergency department activity: a platform

for system-level operations research. Academic Emer-

gency Medicine, 11(11):1177–1185.

Fone, D., Hollinghurst, S., Temple, M., and Round, A.

(2003). Systematic review of the use and value of

computer simulation modelling in population health

and health care delivery. Journal of Public Health,

25(4):325–335.

Gonz´alez, J., Dalforno, C., Suppi, R., and Luque, E. (2009).

A fuzzy logic ﬁsh school model. Procc. ICCS 2009,

LNCS, 5544:13–22.

Lowery, J. (1996). Introduction to simulation in health

care. Proceedings of the Winter Simulation Confer-

ence, pages 78 – 84.

Stainsby, H., Taboada, M., and Luque, E. (2009). Towards

an agent-based simulation of hospital emergency de-

partments. Services Computing, 2009. SCC ’09. IEEE

International Conference on, pages 536 – 539.

Taylor, K. and Dangerﬁeld, B. (2005). Modelling the feed-

back effects of reconﬁguring health services. Journal

of the Operational Research Society, 56(6):659 – 675.

Walker, D., Southgate, J., Hill, G., and ..., M. H. (2004).

The epitheliome: agent-based modelling of the social

behaviour of cells. Biosystems, 76:89–100.

Wang, J., Gwebu, K., Shanker, M., and Troutt, M. (2009).

An application of agent-based simulation to knowl-

edge sharing. Decision Support Systems, 46:532–541.

AGENT-BASED SIMULATION TO SUPPORT DECISION MAKING IN HEALTHCARE MANAGEMENT

PLANNING

441