Supported by the MEC-Spain under contract TIN2007-64974

AN AGENT-BASED DECISION SUPPORT SYSTEM

FOR HOSPITAL EMERGENCY DEPARTMENTS

Manel Taboada

, Eduardo Cabrera

and Emilio Luque

Tomas Cerda Computing Science School, Universidad Autonoma of Barcelona (UAB), Barcelona, Spain

Computer Architecture and Operating Systems Department (CAOS)

Universidad Autonoma of Barcelona (UAB), Barcelona, Spain

Keywords: Healthcare operational management, Agent-based modelling, Individual oriented simulation, Emergency

department, Decision support systems.

Abstract: Healthcare operational management provides many areas where modelling and simulation have been shown

to be useful tools, and within this field there is probably no area more fluid or dynamic than hospital

emergency departments (ED). This paper presents the results of an ongoing project that is being carried out

by Individual Oriented Modelling (IoM) research group of the UAB, with the participation of Hospital ED

Staff Teams. The general objective is creating a simulator that, used as decision support system (DSS), aids

the heads of the ED to answer both, “what if...” questions in order to make the best informed decisions

possible, and more complex questions involving the optimisation of the system. The defined ED model is a

pure Agent-Based Model, formed entirely of the rules governing the behaviour of the individual agents

which populate the system. Two distinct types of agents have been identified, active and passive. Active

agents represent human actors, meanwhile passive agents represent services and other reactive systems.

Active agents are described by Moore state machines extended to include probabilistic transitions. With the

aim of verifying the proposed model a simulation has been created using NetLogo.

1 INTRODUCTION

Healthcare is one of the most important services in

modern civilisation. In a hospital there are many

complex, independent, but interrelated departments

(Decker, 1998). The Emergency Department (ED)

may well be one of the most complex and fluid

healthcare systems that exists, consuming a large

portion of economic budgets for health services.

However, patients often feel neglected and that the

service is saturated.

The simulation of complex systems is of

considerable importance and is used in a broad

spectrum of fields such as engineering, biology,

economy and health care. There are no standard

models to describe these complex systems, but they

may share many common traits. Agent-Based

Modelling (ABM) is an efficient and well utilised

technique that has many advantages, amongst them

is increased detail in simulation based experiments,

a transparent learning process, and the ability to a

control and easily modify individual behaviour.

This paper presents the results of an ongoing

project that is being carried out by Individual

Oriented Modelling (IoM) research group of the

UAB, with the participation of the ED Staff Team of

the Hospital of Sabadell. Its general objective is

create a simulator that, used as decision support

system (DSS), aids the heads of the ED to allow

additional knowledge of patient admission

scheduling (Hutzschenreuter et al., 2008), physician

staff (Jones and Evans, 2008), resource optimisation,

and decreased patient waiting time (Laskowski and

Mukhi, 2008).

Following Macal and North (2006), and also

making use of the considerable expertise existing

within the IoM research group, a concrete and

continuous development methodology has been

devised for the construction of the tool, following an

iterative & spiral process. Each cycle involves 5

phases: 1) system analysis; 2) model design; 3)

simulator implementation; 4) simulator execution

and results analysis; 5) simulator validation. Once

417

Taboada M., Cabrera E. and Luque E..

AN AGENT-BASED DECISION SUPPORT SYSTEM FOR HOSPITAL EMERGENCY DEPARTMENTS.

DOI: 10.5220/0003291804170422

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2011), pages 417-422

ISBN: 978-989-8425-34-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

the first iteration has been completed, based on the

conclusions obtained during the analysis and

validation phase, the model is updated and a new

cycle is carried out. The process will be repeated

until the objectives are achieved.

An Agent-Based Model for Emergency

Departments is being designed, in which all rules

within the model concern the agents, no higher level

behaviour is modelled; it emerges as a result of local

level actions and interactions. This model describes

the complex dynamics found in a hospital ED,

representing each individual and system as an

individual agent. After ending the first cycle two

distinct kinds of agents have been identified, active

and passive. Active agents represent the individuals

involved in the ED, in this case all human actors,

such as patients, nurses or doctors. Passive agents

represent services and other reactive systems, such

as the information technology (IT) infrastructure or

central services used for performing tests.

In order to simulate the model state machines are

used to represent the actions of each agent and the

communication between agents. This takes into

consideration all the variables that are required to

represent the many different states that an individual

may be in throughout the course of their time in a

hospital emergency department, be that individual a

patient, a member of hospital staff, or any other role.

The change in these variables, invoked by an input

from an external source, is modelled as a transition

between states. The communication between

individuals is modelled as the inputs that agents

receive and the outputs they produce, both implicitly

and explicitly. In order to control the agent

interaction, the physical environment in which these

agents interact also have to be modelled, being

sufficient do it as a series of interconnected areas,

such as admissions, the waiting room, or

consultation suits.

In the next cycles new agents and state variables

will be added gradually, until the simulator behaves

as similar as possible to the real system, although

being less complex. After this, data assimilation and

optimisation techniques will be used for new

improvements of the tool. Parallel simulations with

different parameters will be performed, in order to

make adjustments to the model based on the

comparison of the results of these simulations with

data from real systems. High Performance

Computing will be necessary due to the high amount

of data and computation inherent to these both

phases.

The remainder of this article is organised as

follows; section 2 describes the related work in

healthcare operational management and simulation.

The proposed emergency department model is

detailed in section 3, while the corresponding

simulation is given in section 4. In section 5 the

future work is pointed out. Finally, section 6 closes

with conclusions.

2 RELATED WORK

The modelling and simulation of hospital emergency

departments sits at the intersection of a number of

distinct fields. In addition Agent-based techniques

have been used in the modelling of healthcare

operational management, but there are few pure

agent-based models to be found in the literature that

have been rigorously validated against their real

world counterparts.

Economics, biology, and social sciences are the

three fields in which agent-based models are most

utilised (Jones and Evans, 2008). Modelling

techniques using agents can bring the most benefit

when applied to human systems where agents

exhibit complex and stochastic behaviour, the

interaction between agents are heterogeneous and

complex, and agent positions are not fixed

(Bonabeau, 2002). In the particular case of social

sciences ABMs are used in situations where human

behaviour cannot be predicted using classical

methods such as qualitative or statistical analysis

(Norling and Sonenberg, 2000). Human behaviour is

also modelled with ABMs in the fields of

psychology (Smith and Conrey, 2007) and

epidemiology (Epstein, 2009) amongst others.

Agent technology is a useful tool when applied

to healthcare applications. Previous works modelling

healthcare systems have focused on patient

scheduling under variable pathways and stochastic

process durations, the selection of an optimal mix

for patient admission in order to optimise resource

usage and patient throughput (Hutzschenreuter et al.,

2008). Work has been performed using differing

degrees of agent-based modelling for evaluating

patient waiting times under the effects of different

ED physician staffing schedules (Jones and Evans,

2008) or patient diversion strategies (Laskowski and

Mukhi, 2008).

This proposal addresses many of the issues

surrounding the modelling and simulation of a

hospital emergency department using agent-based

technologies. The basic rules governing the actions

of the individual agents are defined, in an attempt to

understand micro level behaviour. The macro level

behaviour, which means the system as a whole,

HEALTHINF 2011 - International Conference on Health Informatics

418

emerges as a result of the actions of these basic

building blocks, from which an understanding of the

reasons for system level behaviour can be derived as

shown Staindsby, Taboada and Luque (2009).

3 EMERGENCY DEPARTMENT

MODEL

The Emergency Department model defined in this

work is a pure Agent-Based Model, formed entirely

of the rules governing the behaviour of the

individual agents which populate the system.

Through the information obtained during interviews

carried out with ED staff at the Hospital of Mataro

and the Hospital of Sabadell, two kinds of agents

have been identified; these are active and passive

agents. The active agents represent people and other

entities that act upon their own initiative (patients,

companions of patients, admission staff, sanitarian

technicians, triage and emergency nurses, staff

emergency doctors, specialists, and social workers).

The passive agents represent systems that are solely

reactive, such as the loudspeaker system, patient

information system, pneumatic pipes, and central

diagnostic services (radiology service and

laboratories).

This section is dedicated to describe the various

components of the general model in detail. Section

3.1 explains the manner in which active agents are

modelled. Passive agents are discussed in section

3.2. The communication model is defined in section

3.3. Finally in section 3.4 the details of the

environment where the agents move and interact are

outlined.

3.1 Active Agents

Active agents are described by state machines,

specifically Moore machines. A Moore machine has

an output for each state; transitions between states

are specified by the input. Considering this, the

current state of an active agent is represented by a

collection of “state variables”, known as the state

vector (T). Each unique combination of values for

these variables defines a distinct state. In each time

step the state machine moves to the next state as

defined by the current state and the input vector as

described below.

3.1.1 State Variables

In order for the state machine to function, all state

variables must be enumerable in some manner. This

may be discrete variables or variables representing

continuous quantities which have been divided into

ranges.

An initial set of state variables has been defined

through the round of interviews performed, based on

the minimum amount of information required to

model each patient and member of staff. Such state

variables are: name/identifier (for identifying each

individual), personal details (any individual

information which is important in relation with his

stay in the ED, such as age, medical history, origin,

etc.), location (the area of the ED where the

individual currently is), action (what the individual

in a particular state is doing), physical condition

(haemodynamic-constants and the degree of

dependence following the Barthel Index), symptoms

(healthy, cardiac/respiratory arrest, severe/moderate

trauma, etc. Information reported by the patient, and

classified by ED staff following the relevant triage

and acuity scale, the Canadian or the Andorran

scale, depending on the country of the Hospital),

communication skills (The time spent during the

process depends on the agent’s ability to

communicate. The possible values are low, medium

or high), and level of experience of the ED staff

(None, low, medium or high), which also influences

the processing time. Some of the state variables will

have a potentially very large set of possible values,

e.g. the symptoms or physical condition.

3.1.2 Inputs, Outputs & State Transitions

Upon each time step the state machine moves to the

next state. This may be another state or the same one

it was in before the transition. The next state the

machine takes is dependent on the input during that

state. The input may be more accurately described as

an input vector (I) that contains a number of input

variables, each one of which may take a number of

different values. As this is a Moore machine, the

output depends only on the state, so each state has its

own output, although various states may have

outputs that are identical.

Table 1: State transition table.

AN AGENT-BASED DECISION SUPPORT SYSTEM FOR HOSPITAL EMERGENCY DEPARTMENTS

419

Again, the output is more accurately described as an

output vector (O), a collection of output variables,

each with a number of defined possible values.

Transitions between states are dependent on the

current state at time t (St) and the input at time t (It).

Following the transition the state machine will be in

a new state (St+1). The state machine can be

represented as a state transition table, as shown in

Table 1, where each row represents a unique state

input combination, showing the output and the state

in the next time step (defined by the current state

and the input).

3.1.3 Probabilistic State Transitions

In dynamic and complex systems such as hospitals,

there exists the necessity for a model not to be

entirely deterministic. In these cases a state machine

can be modelled with more than one possible next

state given a current state and input combination.

Which transition is made is chosen at random at the

time of the transition, weights on each transition

provide a means for specifying transitions that are

more or less likely for a given individual. Each one

of the input variable of the input vector (I) may take

a number of different values. In these cases the state

transition table is defined with probabilities on the

“effect” of the input. An agent in state S

receiving

input I

may move to either state S

, state S

, or

remain in the same state, with a probability of p

, p

and p

respectively. One of these transitions will

always occur, which is to say p

+ p

= 1.

Figure 1 shows the three different transitions for the

“current state-input” combination of this specific

example.

Figure 1: Probabilistic state transitions graph

representation.

The exact probabilities may be different for each

agent, in this way agent behaviour can be

probabilistically defined external to their state,

representing personality characteristics in different

people.

3.2 Passive Agents

Passive agents represent services within the hospital

system such as the IT infrastructure that allows

patient details to be stored, radiology services and

other laboratory tests as well as specialist systems

such as the pneumatic tube networks that some

larger hospitals use to quickly transfer samples from

one part of a building to another.

Not all passive agents are modelled using state

machines. In some cases this may be overly

complicated, due to systems with register like

memory capacities; in others a simple store and

forward messaging system is sufficient.

3.3 Communication Model

To model the interaction between agents 3 types of

communication are proposed:

 1-to-1 between two individuals;

 1-to-n is representing an individual addressing a

group;

 and 1-to-location a type of communication

where an individual speaks to all occupants of a

specific area, for instance using a speaker

system.

Messages contain three parts. The message

source is the individual who is communicating,

speaking in many cases. The message destination

would then be to whomever this individual is

speaking to, and thirdly the content, what is being

said. These three parts form the message tuple

(<src>, <dst>, <content>). In the case of a 1-to-

location message, the destination of the message is

an entire location, so the content may need to

include the actual indented recipient of the message.

This could represent a patient’s name being called

over the loudspeaker system.

3.4 Environment

All actions and interactions modelled take place

within certain locations, collectively known as the

environment. The environment itself can be defined

to different levels depending on the positional

precision required of the model.

The environment in which the agents move and

interact is passive and discrete. There is little

distinction made between agents in the same

location. A patient in the waiting room does not

have any more specific sense of position than they

are in the waiting room. Certain locations may be

physically distinct, but functionally identical, for

instance there are usually a number of triage rooms,

HEALTHINF 2011 - International Conference on Health Informatics

420

Figure 2: Simplified Emergency Department layout.

an agent in any one of these will act as if they are in

any triage room, however they Simplified ED layout

are distinct in order to represent that each available

room may only be used by one nurse-patient group

at a time.

The environment also contains representations of

the relative distances between different discrete

locations. Figure 2 shows a representation of

topographical distribution of the Emergency

Department.

4 SIMULATION

With the aim of verifying the proposed model

against real data sets from a hospital emergency

department a simulation has been created using the

agent based simulation tool NetLogo (Wilensky,

1999).

The tool initially uses a simplified set of patient

attributes and a less complicated patient flow in

order to make a preliminary demonstration of how

accurate a simulation can be produced using only

reduced parameters. The four primary areas are

accounted for: admissions, triage (3 boxes), waiting

rooms (one for patients before triage, and the second

for patients who have passed the triage process, and

are waiting for treatment), and diagnosis and

treatment areas conjoined (that include four boxes).

The types of individuals represented in this simula-

Figure 3: Simulation display in NetLogo.

tion are patients, admission staff, triage nurses, and

doctors.

In this model the patients are shown following

the same path through the ED, even though in reality

they are been treated differently depending on the

level of severity of their condition. The time spent at

each stage may also represent laboratory tests, which

are not shown explicitly.

Since a primary use of the simulation tool is to

garner feedback from the professionals who work in

emergency departments, a graphical representation

of the process is considered a very important aspect

of the simulation. NetLogo allows visualisations of

agent actions and interactions; a sparse patient load

is shown in figure

There is further validation to be performed.

However initial tests using the simulation are

showing very promising results.

5 FUTURE WORK

Following the iterative & spiral process discussed in

the introduction, after this first cycle, the work will

continue with the purpose of improving the model

and the simulation, applying assimilation techniques

for that.

In the next iterations new agents and state

variables will be added gradually, until the simulator

behaves as similar as possible to the real system,

although being less complex. After this, parallel

simulations with different parameters will be

performed, in order to make adjustments based on

the comparison of the results obtained in these

simulations with data from real system.

Once the improving of the model achieves a

proper adjustment level, simulation will be used as

DSS, with the objective of answering “what if...”

questions in order to aid healthcare managers make

the best informed decisions possible. The tool will

let to divine what will happen to the system as a

whole if one or more changes are made to the

AN AGENT-BASED DECISION SUPPORT SYSTEM FOR HOSPITAL EMERGENCY DEPARTMENTS

421

parameters that define it. After additional definitions

of quantitative and qualitative indices, more

complex questions involving the optimisation of the

system could be evaluated. In these specific cases

simulation will be carried out with the objective of

minimising (patient waiting times, total service cost,

etc) or maximising (patient’s satisfaction) indices,

subject to specific constraints of human and material

resources available.

The distribution pattern of patients’ arrival to the

ED varies among the day, but also over a week or a

year. For this reason, and having into account the

objectives of the project, is desirable to run

simulations for an annual period of time. In addition,

as a result of the potential number of individuals to

include in the simulation and the number of states in

the state machine of each individual, a great amount

of values should be computed. Considering also the

parallel simulations that will have to be performed

during the adjustment process, it can be concluded

the need of using High Performance Computing.

6 CONCLUSIONS

A concrete example of an Agent-Based Model for

Hospital Emergency Departments has been

presented, which represents a hospital ED following

system analysis performed at a number of different

hospitals, under the advice of healthcare

professionals with many years of experience. The

model uses state machine based agents which act

and communicate within a defined environment,

providing the ability to study the dynamic of

complex systems without the difficulty of obtaining

exhaustive system descriptions required by other

modelling paradigms. An initial simulation has been

created in order to demonstrate the validity of the

model.

Future improvements of the model will be made

using data assimilation and optimisation techniques.

The former will involve performing parallel

simulations with different parameters, and after

comparing data from simulation and real system,

making the proper adjustments of the model. The

latter refers to optimisation through simulation runs

in order to minimise or maximise indices such as

patient waiting times, total service cost or patient

satisfaction, while adhering to constraints in the

number of doctors, nurses and hospital beds

available. In both parallelisation and High

Performance Computing will be necessary.

From this point, the simulation will be able to be

used as the core component of a decision support

system to aid hospital administrators make better use

of resources, achieving a more efficient and

improved patient care cycle. This in turn will allow

better management of dynamic patient flow, either

as a result of specific circumstances or seasonal

fluctuation.

REFERENCES

Bonabeau E., 2002. Agent-based modeling: Methods and

techniques for simulating human systems. In

Proceedings of the National Academy of Sciences.

Decker K, Li J., 1998. Coordinated Hospital Patient

Scheduling. In: Proceedings of the 3rd International

Conference on Multi Agent Systems. Washington, DC,

USA: IEEE Computer Society;

Epstein J. M., 2009. Modelling to contain pandemics. In

Nature, 460(7256):687.

Heath B, Hill R, Ciarallo F., 2009. A Survey of Agent-

Based Modeling Practices In Journal of Artificial

Societies and Social Simulation;12(4):9.

Hutzschenreuter A. K., Bosman P. A. N., Blonk-Altena I.,

van Aarle J., La Poutr´e H., 2008. Agent-based patient

admission scheduling in hospitals. In: Proceedings of

the 7th international joint conference on Autonomous

agents and multiagent systems. Richland.

Jones S. S., Evans R. S., 2008. An Agent Based

Simulation Tool for Scheduling Emergency

Department Physicians. In: AMIA Annual Symposium

proceedings,

Laskowski M., Mukhi S., 2008. Agent-Based Simulation

of Emergency Departments with Patient Diversion. In:

eHealth; p. 25–37.

Macal C., and North M., 2006. “Tutorial on agent-based

modeling and simulation part 2: how to model with

agents,” In: Proceedingsof the Winter Simulation

Conference, pp. 73 – 83.

Norling E., Sonenberg L., Ronnquist R., 2000. Enhancing

Multi-Agent Based Simulation with Human-Like

Decision Making Strategies. In: MABS; p. 214–228.

Smith E. R., Conrey F. R., 2007 Agent-Based Modeling:

A New Approach for Theory Building in Social

Psychology. In: Pers Soc Psychol Rev. 11(1):87–104.

Stainsby H., Taboada M., Luque E., 2009. Towards an

Agent-Based Simulation of Hospital Emergency

Departments. In: Proceedings of the 2009 IEEE

International Conference on Services Computing.

Washington, DC, USA: IEEE Computer Society

Wilensky U., 1999. NetLogo. Center for Connected

Learning and Computer-Based Modeling,

Northwestern University Evanston, IL.

HEALTHINF 2011 - International Conference on Health Informatics

422