Agent-based Modelling for Simulating Patients Flow

in a Community Hospital

Thomas Ostermann

Institute of Integrative Medicine, Witten/Herdecke University, 58313 Herdecke, Germany

Keywords: Agent-based Modelling, Hospital Management, Waiting times, Queueing.

Abstract: One of the most innovative tools in health care informatics is agent-based modelling. Such models change

dynamically and help to understand interactions in complex systems especially when simulating competitive

and cooperative behaviors in human systems. In our approach we use multi-agent modelling for simulating

and evaluating patients flow in a community hospital. The model proposed in this context consists of three

different types of agents: the hospital agent, the unit-agent and the patient-agent. Calculation of waiting

times was performed using previously collected data from elective patients entering the community hospital

ambulance. Poisson distribution was used to model waiting times. The simulation was carried out using the

JAVA-based multi-agent-modelling environment Quicksilver. After solving convergence problems, we

found, that the simulation especially for the ambulance entrance unit did show completely unexpected

results. We were able to prove that the waiting times did not solely refer to the service times of the modelled

units. To assure an unobstructed patient flow, we also showed that the mean service time at the entrance unit

should not exceed 25 min. Although no evidence was given by the isolated analysis of waiting times, the

simulation gave hints for a “hidden patient queue”, which after presenting the results in the quality circle

meeting was confirmed by the ambulance staff.

1 INTRODUCTION

Waiting times for elective (non-urgent) in- and

outpatient hospital treatment are a major health

policy concern generating dissatisfaction of patients

before their treatment has started. A recent study

from the Kopanong Hospital, South Africa found

waiting times of up to five hours due to bottlenecks

at the reception and the treatment areas (Nhlapo,

2012).

Additionally, waiting times are also economic

factors which may cause extra costs and loss of

utility (Olukotun and Ogbadu, 2013).

Especially in hopitals with interdisciplinary

ambulances which act as an interface in the patient

flow of different departments (i.e. surgery, internal

medicine) the managing of the clinic wait is an

important quality of care challenge (Rondeau, 1998).

To describe and optimize the process of patients

waiting in a hospital, methods of stochastic queuing

theory show possibilities for hospitals to optimize

clinical pathways of patients in time (Brahma, 2013;

Schwierz et al., 2011). A mathematical analysis of

waiting times of elective patients in a previous study

(Ostermann et al., 2003) found a good qualitative

agreement with the theoretically assumed results of

waiting-times-distribution. Within this theoretical

model, six homogenous groups of patients were

identified and their total waiting times were

calculated ranging from 24 to 53 minutes in the

mean which confirms the results of similar studies

(Kadivec et al., 2001; Sonnenberg, 2000).

However, this model did not explain how waiting

times of patients emerge and in which way they

depend on treatment times in the different units.

Moreover, it did not answer the question if there is

an interaction of different units within the patient

flow in the hospital.

One way of answering these question is the use

of simulation techniques to model different types of

environments. Especially for patients waiting times,

several simulation approaches do exist in the

literature. For example, Garcia et al. (1995)

presented a simulation model focussed on reduction

of waiting time in the emergency room of Mercy

Hospital in Miami. Another simulation used

complex optimization techniques of operation

research to model hospital emergency departments

queueing systems (Panayiotopoulos and

Vassilacopoulos, 1984).

Ostermann T..

Agent-based Modelling for Simulating Patients Flow in a Community Hospital.

DOI: 10.5220/0005178100140019

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2015), pages 14-19

ISBN: 978-989-758-068-0

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

This article presents an approach to simulate the

waiting process of patients using a combination of

queuing theory and agent-based modelling.

Agent-based modelling is a fairly new paradigm

in health care information science. Although there is

no common definition of what is called an artificial

agent, it can be described as an autonomous, social,

reactive and proactive entity, with a behaviour

predictable by attributing abstractions of

anthropomorphic attitudes (i.e. intention, obligation,

commitment, choice) to describe its behaviour

(Della Mea, 2001). Hence, a multi-agent system is a

open system, which may change dynamically, when

its agents change their behaviour or interact in

special ways. It therefore helps to understand

interactions in complex systems especially when

simulating competitive and cooperative behaviors in

human systems, which are difficult to capture in

other mathematical formalisms (Bonabeau, 2002).

Thus, agent-based modeling has become a powerful

simulation technique that has seen a number of

applications in the last few years.

In the health care sector, first attempts of using

multi agent systems concentrated in the scheduling

of operations especially in the field of organ

transplants (Moreno et al., 2001; Becker et al.,

2003). Meanwhile several applications have

demonstrated the power of agent based models in

health services research (Maglio and Mabry, 2011).

A first attempt to use agent based modeling for

hospital management simulation was presented by

Sibbel & Urban (2001).

In our approach we used multi-agent modelling

for simulating and evaluating patients flow in a

community hospital. The main question in our

approach is, whether the process of patients’ flow

itself induces the waiting times or if interactions

outside this process may be responsible for the

generation of waiting times. Therefore, we carried

out a simulation based only on the arrival times and

the service times distributions in the particular units.

On this bases we compared empirical waiting times

with the results of the multi-agent simulation.

2 MATERIAL AND METHODS

The model proposed in this context consists of three

different types of agents: the hospital agent, the unit-

agent and the patient-agent (Figure 1). The hospital

agent represents the hospital and is the models‘ root.

Here patients arrive and are sent to the “Outpatient

office”-unit. The time of the patient’s arrival is

determined by empirical data derived from

(Ostermann et al., 2003) given in Figure 2. With this

distribution a timetable is created which also defines

the exact times of arrival of the patient. As soon as it

is time for a new patient to appear, he is created by

the hospital agent. The patient’s type and his time of

arrival are determined.

The unit-agent describes a unit during the

admission. In this entity the units outpatient office,

radiology, ECG, admission-unit and departments are

described. The first four units are transient, since a

patient can be sent to another unit after passing this

one. The unit-agent holds two queues, one for the

patients waiting and one for the patients being

treated.

Figure 1: Interacting Agents of this Model.

At every moment there is exactly one patient

being treated. Each unit i is associated with a

lambda-value 

, which is the reciprocal of the mean

service time at this unit:





It is derived from the empirical data given in

Table. 1 and generates an exponential distributed

service time, from which an individual patient’s

service time is randomly assigned.

0 2 4 6 8 1012141618202224

tim e in h

Figure 2: Empirical distribution of patients-arrivals during

the day measured for ten days.

Agent-basedModellingforSimulatingPatientsFlowinaCommunityHospital

The unit-agent also sends the patient to the next unit,

as soon as his treatment is completed. For this

purpose, it is necessary to find out, whether the

patient has to visit another unit. This can be decided

by the patient’s type and the units he has already

been sent to.

If the patient does not have to go to another unit,

he is inserted in the queue of treated patients and it is

decided whether he is hospitalized or discharged.

When a patient enters a unit, he is appended to

the end of the queue of already waiting patients

unless he is an emergency patient. Such patients are

treated as an exception and hence they are inserted at

the beginning of the queue. The time he has to wait

until his treatment begins, is measured and stored

into the patient-agent.

Table 1: Mean service times for the units.

Unit

Waiting time

(mean ± sd)

Outpatient office 29.2 ± 18.7

320

ECG 11.4 ± 8.7

Radiology 9.7 ± 7.9

197

Admission 9.2 ± 9.2

Department 25.8 ± 18.3

The patient-agent in this model is an agent

without any methods for acting individually i.e.

skipping a unit. Similar as in real life, he is sent from

unit to unit and just remembers his times of being

treated and of waiting. Each patient belongs to a

special type. This type specifies, whether the patient

is to be treated as an emergency, whether he is a

patient with or without transient units or if he needs

an inpatient or an outpatient treatment. The

Table 2: Groups of patients and their characteristics.

Group of patients N %

min

max

Stationary patients

directly transferred to a

hospital department

35 12,2% 37,4 37,4

Stationary patients with

one intermediate unit

27 9,4% 40,9 46,6

Stationary patients with

two intermediate units

40 13,9% 47,8 53,5

Stationary patients with

more than two

intermediate units

56 19,4% * *

Ambulant patients with

one intermediate unit

97 33,6% 24,5 27,9

Ambulant patients with

two or more

intermediate units

13 4,5% 31,4 31,4

distribution of the types was taken from the

empirical data of (Ostermann et al., 2003) and is

shown in Table 2.

The simulation was carried out using the JAVA-

based multi-agent-modelling environment Quick-

silver (Burse, 2000). Calculation of waiting times

were performed using SPSS with a Poisson-

distribution

etW





 1)(

W(t) expresses the possibility of being served

after t minutes.

3 RESULTS

For this evaluation forty patients were simulated,

arriving at the hospital during a whole day from 0.00

h to 23.59 h. One minute is the smallest interval in

the model, since the empirical data also measure

time in minutes. Astonishingly, with the original

lambda values for the service times at the entrance

unit “Outpatient office” the model did not converge,

which means that after finishing one simulation

patients still were waiting at the entrance.

Therefore, simulations for different lambda-

values were made, with the value of lambda varying

from slightly bigger to slightly smaller than the

empirically quantified lambda. To further reduce

uncertainty, ten days with different initializations of

the random numbers were simulated for each

lambda-value. Figure 3 shows the convergence of

the model depending on the chosen lambda.

Figure 3: Convergence of the model depending on the

chosen lambda for the service time at the entrance unit.

For  ≥ 0.054 the simulation did finish within the

day, thus, in the following, we compared the

empirical data with the simulation data for  = 0.054

for the entrance unit “outpatient office”, which is

equivalent to a mean service time of 19 minutes.

Especially for the “Radiology”-unit but also for

HEALTHINF2015-InternationalConferenceonHealthInformatics

the “ECG”-unit, patients’ waiting-times were

sufficiently reproducible by the given service times

with the chosen multi-agent system. Simulated and

empirical data are given in Fig. 4 and 5.

Figure 4: Empirical and simulated waiting times for the

“ECG”-unit.

Figure 5: Empirical and simulated waiting times for the

“Radiology”-unit.

However, this agreement of simulation and

empirical data did not apply for the service times in

the outpatient-unit. The empirical mean service-

duration in the outpatient-unit was approx. 29

minutes, which lead to a lambda-value of 0.034.

This value as already stated did not lead to a

converging simulation of the patient flow. It wasn't

until a lambda-value of 0.08 when the simulation did

show sufficient similarities with the empirical data.

The best agreement was found with a lambda of

0.09, which denotes a mean service time at the

outpatient office of 11 minutes.

Thus, as a result, we found that the waiting times

are not solely generated by service times. Especially

for the ambulance unit, there have to be hidden

underlying processes beyond treatment which

explain the difference of 18 minutes between the

empirical measures mean service time of 29 minutes

and the fitting simulation result of 11 minutes.

One possible process which induces virtual

waiting time is sketched: The patient enters the

ambulance. Maybe due to misplacement, his file has

to be fetched from another unit a new file has to be

created. This will take some time whilst this patient

occasionally leaves the ambulance and the next

patient is served. Soon afterwards, the patient queues

again and is followed up after the actually served

patient.

In the empirical study, the patient has included

this interruption process into his serving time, which

by definition lasts until he is directed to the next

unit. Unfortunately, this event has not been part of

the model and thus was not reproduced in the

simulation.

4 CONCLUSIONS

Originally coming from the social science, agent-

based modelling is an innovative technique currently

used to simulate patient flows and patient scheduling

in hospital environments (Kirn et al., 2000;

Lettkemann et al., 2002; Paulussen et al., 2003;

Kanagarajah et al., 2008).

We aimed to show the possibilities of a

combination of mathematical queuing theory and

agent-based-modelling for the analysis of waiting

times in the setting of an interdisciplinary hospital

ambulance with different units visited by elective

appointed patients. After solving the convergence

problem of the simulation, we were quite surprised,

that the simulation especially for the ambulance

entrance unit did show completely unexpected

results. One reason for fluctuations in the results

might be the certainty and the stability of the model,

which is questionable, as different starting values for

the random number generator led to non-negligible

variations in the simulations.

Despite these structural problems of the

simulation, we could prove that the waiting times

did not solely refer to the service times of the

modelled units. To assure an unobstructed patient

flow, we also showed that the mean service time at

the entrance unit should not exceed 25 minutes.

Although no evidence was given by the isolated

analysis of waiting times, the simulation gave hints

for a “hidden patient queue”, which after presenting

the results in the quality circle meeting was

confirmed by the ambulance staff.

For further research the implementation of a

patient satisfaction function is a reasonable

Agent-basedModellingforSimulatingPatientsFlowinaCommunityHospital

enhancement of the proposed model. Empirical

studies like those carried out by Spaite et al. (2002)

found high correlations between the decreases in

waiting time intervals and improvement in patient

satisfaction. From the mathematical point of view

this correlation can be modelled by using a logistic

regression model, where the satisfaction s is a

function of the waiting time t (Hackl and Westlund,

2000):







)(

; t = waiting time.

The parameter c models the patients reactivity on

waiting and is randomly assigned to the patient

agent which cumulates the waiting times t

of each

station. Thus, some patient will show a bad temper

after waiting only a short time at one unit whilst

others will keep their head although they have high

waiting times at all units (Pruyn and Smidts, 1993).

If then an individually assigned threshold value i.e.

s=0.7 is passed, this could prompt the patient agent

to file a complaint to the hospital. Since the

incoming complaints increase, this leads to quality

measures to lower the service times at the units by

increasing the lambda-values. This would lead to a

model with a feedback loop, which can be used to

simulate special scenarios like queuing of elderly

patients (Andersson et al., 2011).

Apart from the analysis of waiting times, such

models can also provide useful insights when being

used e.g. to simulate patients’ drug compliance and

behaviour in outcome studies. Such a system for

planning, management and decision support of

clinical trials has recently been proposed by Heine et

al. (2005).

ACKNOWLEDGEMENTS

I would like to thank Georg Johann, University of

Osnabrück, Germany for his support in

programming the model in Quicksilver.

REFERENCES

Andersson, L., Burman, M., Skär, L., 2011. Experiences

of caretime during hospitalization in a medical ward:

older patients’ perspective. Scandinavian journal of

caring sciences, 25(4), 646-652.

Becker, M., Krempels, K.H., Navarro, M., Panchenko, A.,

2003. Agent Based Scheduling of Operation Theaters.

Proceedings des EU-LAT E-Health-Workshop,

Mexiko.

Bonabeau, E., 2002. Agent-based modeling: methods and

techniques for simulating human systems. Proc Natl

Acad Sci USA; 14;99 Suppl 3:7280-7.

Brahma, P. K., 2013. Queuing theory and customer

satisfaction: a review of terminology, trends, and

applications to hospital practice. Asia Pacific Journal

of Marketing & Management 2(6): 83-89.

Burse, J., 2000. Quicksilver: A Component-Based

Environment for Agent-Based Computer Models and

Simulations. In Computational Social Organisational

Science Conference, CASOS, Pittsburgh, PA.

Della Mea, V., 2001.Agents acting and moving in

healthcare scenario: a paradigm for telemedical

collaboration. IEEE Transaction on Information

Technology in Biomedicine: 5; 10-13.

Garcia, M.L., Centeno M.A., Rivera, C., DeCario, N.,

1995. Reducing Time in an Emergency Room via a

Fast-Track. In Proceedings of the 1995 Winter

Simulation Conference, Alexopoulus, Kang, Lilegdon

& Goldman (eds.): 1048-1053.

Hackl, P., Westlund, A. H., 2000. On structural equation

modelling for customer satisfaction measurement.

Total Quality Management, 11(4-6), 820-825.

Heine, C., Herrler, R., Kirn, S., 2005. ADAPT@Agent.

Hospital: Agent-Based Optimization & Management

of Clinical Processes. International Journal of

Intelligent Information Technologies 1, 30-48.

Kadivec, S., Eržen, D., Šorli, J., 2001, Waiting times in

the University hospital. ZDRAV VESTN; 70: 391–4.

Kanagarajah, A. K., Lindsay, P., Miller, A., Parker, D.,

2008. An exploration into the uses of agent-based

modeling to improve quality of healthcare. In Unifying

Themes in Complex Systems (pp. 471-478). Springer

Berlin Heidelberg.

Kirn, S., Heine, C., Petsch, M., Puppe, F., Klügl, F.,

Herrler, R., 2000. Agentenorientierte Modellierung

vernetzter Logistikkreisläufe als Ausgangspunkt

agentenbasierter Simulation – Beispiel

Wöchnerinnenstation. In Kirn, St.; Petsch M. (Eds.):

Proceedings 2. Kolloquium des DFG-Schwerpunkt-

programms „Intelligente Software-agenten und

betriebswirtschaftliche Anwendungs-szenarien“

Ilmenau, Germany.

Lettkemann, E., Meister, M., Hanft, A., Schröter. K.,

Malitz, R., 2002. The Description of Practical Roles in

Hospital Environments. In Lindemann G, Jonker C,

Timm IJ (Eds.): Proceedings of the KI'2002 Workshop

on Modelling Artificial Societies and Hybrid

Organizations MASHO'02 2002, Aachen, Germany;

29-36.

Maglio, P. P., Mabry, P. L., 2011. Agent-based models

and systems science approaches to public health.

American journal of preventive medicine, 40(3), 392.

Moreno, A., Valls, A., Bocio, J. A., 2001. Multi-Agent

System to Schedule Organ Transplant Operations.

Inteligencia Artificial, Revista Iberoamericana de IA

2001; 13: 36- 44.

Nhlapo, M. T., 2012. The appraisal of factors affecting

waiting times and recommendations for improvement

at out-patient department (OPD) of Kopanong

HEALTHINF2015-InternationalConferenceonHealthInformatics

Hospital. Doctoral dissertation, Faculty of Health

Sciences, University of the Witwatersrand.

Olukotun, G. A., Ogbadu, E. E., 2013. The Applications

Of Waiting Line Management On The Operations Of

Public Sector Organizations: The Kogi State Health

Sector Experience (2009-2012). International Journal

of Research in Business and Technology, 3(3), 252-

265.

Ostermann, T., Priegnitz, O., Jochheim, R., 2003.

Mathematical Analysis of Patient’s Waiting Times in

Clinical Pathways of an Interdisciplinary Ambulance.

Gesundh ökon Qual manag; 8(6): 379-384.

Panayiotopoulos J.C., Vassilacopoulos, G., 1984.

Simulating hospital emergency departments queueing

systems. Eur. J. Oper. Res., 18: 250-258.

Paulussen, T.O., Jennings, N.R., Decker, K.S., Heinzl, A.,

2003. Distributed patient scheduling in hospitals.

Proceedings of 18th International Joint Conference on

Artificial Intelligence, Acapulco, Mexico; 1224-1229.

Pruyn, A., Smidts, A.,1993. Customers’ evaluations of

queues: three exploratory studies. European Advances

in Consumer Research, 1, 371-382.

Rondeau, K., 1998. Managing the clinic wait: an important

quality of care challenge. J Nurs Care Qual; 13: 11–

20.

Schwierz, C., Wübker, A., Wübker, A., & Kuchinke, B.

A., 2011. Discrimination in waiting times by insurance

type and financial soundness of German acute care

hospitals. The European Journal of Health Economics,

12(5), 405-416.

Sibbel, R., Urban, C, 2001. Agent-based modeling and

simulation for hospital management. In Theory and

Decision Library Vol. 32, pp 183-202. Springer

Netherlands.

Sonnenberg, A., 2000. Waiting lines in the endoscopy

unit. Gastrointest Endosc; 52(4):517-24.

Spaite, D.W., Bartholomeaux, F., Guisto, J., Lindberg, E.,

Hull, B., Eyherabide, A., Lanyon, S., Criss, E.A.,

Valenzuela, T.D., Conroy, C., 2002. Rapid process

redesign in a university-based emergency department:

decreasing waiting time intervals and improving

patient satisfaction. Ann Emerg Med. 39(2):168-77.

Agent-basedModellingforSimulatingPatientsFlowinaCommunityHospital