Optimal Staffing Policy
A Service System with Stochastic Travel Times
M. Al-Foraih, P. Johnson, G. Evatt and P. Duck
School of Mathematics, University of Manchester, Oxford Road, Manchester M13 9PL, U.K.
Keywords:
Workforce Staffing, Stochastic Models, Time-Varying Queues, Network Queues.
Abstract:
Private sector operators of response services such as ambulance, fire or police etc. are often regulated by
targets on the distribution of response times. This may result in inefficient overstaffing to ensure those targets
are met. In this paper, we use a network chain of M/M/K queues to model the arrival and completion of
jobs on the system so that quantities such as the expected total time waiting for all jobs can be calculated.
The Markov nature enables us to evoke the Hamilton Jacobi Bellman equation (HJB) principle to optimize the
required number of staff whilst still meeting targets.
1 INTRODUCTION
There are strong economic arguments for introducing
competition from the private sector into state emer-
gency response services, in order to improve effi-
ciency and reduce costs (Blackstone et al., 2007). The
state requires that the response service must try to
minimize the time it takes to reach an a emergency
call, as in a case of an ambulance or a fire fighters
it could save lives. Now, a private sector operator
will always look to minimise their costs so they must
be regulated effectively to maintain service to a re-
quired level. In this paper we outline a framework
to solve for the optimal staffing policy of the private
sector firm given a regulatory framework by weight-
ing the benefit of not missing targets with the cost of
staff given stochastic uncertainty in the demand for
service. This paper has been divided into 5 sections.
In section 2 we discuss the literature review and and
the previous work related to this problem. Then, in
section 3, we introduce and define the problem frame-
work and the partial differential equation (PDE) to
solve it. In section 4 we demonstrate the numerical
result and the conclusion are found in section 5.
2 LITERATURE REVIEW
There are variety of problems associated with the op-
timal management of emergency response systems,
and the location of new and existing distribution cen-
ters in particular has received a lot of attention (see
Snyder, 2006, for an overview). A notable work is
that of Singer and Donoso (2008) who combine the
problem of location with queueing theory. They used
key performance indicators such as the average re-
sponse time that improves the operation of ambulance
service providers. Some authors such as Marianov
and ReVelle (1996) have attempted to model both lo-
cation and resource/staffing levels with queues. In
their paper, they outline how to calculate the proba-
bility that a given distribution centres will have spare
capacity given current resource levels (the number of
vehicles as opposed to staff in our framework), and
then using a linear programming approach they can
optimise the position of a limited number of vehi-
cles at distribution centres to best cover the area. The
results though are time invariant and cannot be ap-
plied to something as dynamic as staffing levels where
shifts patterns change, and agency staff might be hired
to cover short term spikes in demand.
The optimal resource problem of how many staff
to employ has in general received much less atten-
tion in the literature than the related location or cov-
erage. There are some exceptions such as Fry MJ
(2006) who examined the problem of determining the
number of employees that would minimise the cost
of hiring in a fire department, subject to state staffing
requirements. He applied news-vendor type models
where the uncertainty depends on the decision vari-
ables. Nevertheless, this approach lacked the cost
of hiring and assumed a discrete time model. The
model within this paper is an extension of the stan-
110
Al-Foraih M., Johnson P., Evatt G. and Duck P..
Optimal Staffing Policy - A Service System with Stochastic Travel Times.
DOI: 10.5220/0004276902580263
In Proceedings of the 2nd International Conference on Operations Research and Enterprise Systems (ICORES-2013), pages 258-263
ISBN: 978-989-8565-40-2
Copyright
c
 2013 SCITEPRESS (Science and Technology Publications, Lda.)

dard queueing theory, utilising a Markov chain of
M/M/K queues. There have been countless other ap-
plications of these theories in problems as diverse as
highway traffic queues Evans et al. (1964) or manage-
ment of call centres (Aksin et al., 2007; Whitt, 2005).
There are some examples of queueing theory within
the problem of optimal staffing, such as Brooks et al.
(2011). They investigated the problem of care-at-
home services using a Markov decision process with
queues aiming to balance the cost of staffing against
the cost of rejection. In our model we assume that the
response service provider wishes to balance costs of
staffing against the response times, since the regulator
has incentivised that with targets. Another difference
between our approach and that of Brooks et al. (2011)
is that the inclusion of different classes of patients
resulted in the full problem becoming computation-
ally intractable. Consequently, they can only provide
heuristic bounds to guide the optimal policy. We aim
to keep our model relatively simple so that the solu-
tions can be found numerically in a reasonable time
frame.
So the unique contribution of this paper is to intro-
duce a time dynamic optimal control problem where
the control is the number of staff available, rather than
admittance or direction of calls on the system. This is
enabled by offsetting the expected total time between
receipt to arrival versus the cost of employing staff.
The result will be an optimal policy where the opti-
mal number of staff will be a function both of time
and the current state of the system. By including one
off costs at the start and the end of a shift, we can
ensure that the policy will result in realistic shift pat-
terns. The weighting of staff costs can then be varied
so as to establish the minimum staffing policy to meet
the required targets.
3 THE MODEL
Let us define a call to the service provider as a job
J which must be completed, and the staff member
that responds to that we shall denote as an engineer.
The engineer could be a policeman, fireman, ambu-
lance engineer, or a technician in the case of leakage
from a distribution network depending on the service
provider we are solving for. The different stages in
job completion are defined by the four times
• t
i
b
the time that the ith job is booked on the system.
• t
i
t
the time that the ith job is assigned to an engi-
neer.
• t
i
a
the time of arrival of the engineer at the ith job.
• t
i
d
the time that the engineer finishes the ith job.
Therefore, we define the ith job vector to be
J
i
= (t
i
b
,t
i
t
,t
i
a
,t
i
d
), (1)
and the set of jobs J = {J
1
,J
2
,..J
I
} represents all jobs
received over a fixed period of time. Furthermore, we
define η = t
a
−t
t
to be the travel time for an engineer
to reach their destination, and ξ = t
d
− t
a
to be the
total time which it takes for the engineer to finish the
job and become available again. We divide time into
K discrete periods of length ∆t, where the values of
K and ∆t may change depending on what we wish to
calculate. We can define the time at a given period to
be
t
k
= t
0
+ k∆t with k = 0, 1,..,K (2)
where t
0
is the reference time that is before the first
time/date in t
k
period of time, t
K
is after the last
time/date, and ∆t = (t
K
−t
0
)/K. Now define the num-
ber of jobs that have been received on the system
within the period t
k
to be B
k
, and the number received
but are yet to be completed at the time period t
k
as Q
k
.
Next we define the response time as the time between
the job being received on the system and the arrival
time of the engineer, or T = t
a
− t
b
. So to get an es-
timation of the response time, we must split the jobs
into three different types
• W the number of jobs waiting for the assignment
of an engineer.
• R the number of jobs where an engineer is in tran-
sit to reach a target
• A the number of jobs for which the engineer has
arrived but has not yet completed the job.
A job will count towards W if the call has been re-
ceived before (or at) the end of the current period, but
the engineer does not set off until after the end of that
period, counted in transit R if the engineer has set off
but not arrived, and counted as an arrived job A if the
engineer arrives before (or at) the end of the current
period and leaves after the start of the next period.
Obviously it follows that
Q
k
= W
k
+ R
k
+ A
k
, (3)
or Q is sum of all jobs waiting to be completed. We
impose no restriction on the number of jobs await-
ing completion but we must stipulate that the number
of engineers S available at time t
k
is greater than the
number in transit or at a job:
R
k
+ A
k
≤ S
k
. (4)
If we assume that the optimal way to manage the sys-
tem is to serve costumers on a first-come first served
basis, (although this might not always be the case if
S varies with time and we know some trips might
take a lot longer than others) then if R
k
+ A
k
< S
k
we
OptimalStaffingPolicy-AServiceSystemwithStochasticTravelTimes
111

have spare capacity and any new job that arrives will
immediately have a engineer sent out. Therefore, if
R
k
+ A
k
< S
k
then we have a sufficient number of en-
gineers to S
k
to cover all the jobs requested or in other
words there are no job waiting for an engineer to be
assigned (W
k
= 0). Conversely, we know that if jobs
are waiting to be assigned, all engineers must be out
on jobs, so that W
k
≥ 0 if R
k
+ A
k
= S
k
. So, even
though it appears as though there are three degrees
of freedom there are in fact only two, which means
that predicting the total number of jobs waiting to fin-
ish and the number of engineers travelling to reach a
target is enough information to tell us how many are
waiting and how many have arrived. Now, we can eas-
ily calculate the cumulative total time spent by each
job between it being booked onto the system and an
engineer arriving, denoted by T T , which is given by
the formula
T T =
∑
i
(t
i
a
−t
i
b
) =
∑
k
(W
k
+ R
k
)∆t + O(∆t) (5)
=
Z
t
K
t
0
W (t) + R(t)dt.
The integral form of the equation is valid as ∆t → 0
and K → ∞. This quantity gives the total time given
all jobs that are taken in the period, but we are in-
terested in the average receipt to arrival time T for a
single job. Then the average receipt to arrival time for
a job T = t
a
−t
b
over the given time period is
E[T ] =
∑
i
(t
i
a
−t
i
b
)
I
=
1
I
Z
t
K
t
0
W (t) + R(t)dt (6)
where I is again the total number of jobs. The equa-
tions (5) and (6) provides a link between the distri-
bution of T (the response times) and quantities that
are easily calculated using continuous time differen-
tial equations.
We assume that jobs arriving on the system fol-
lows a Poisson process with a time varying intensity
denoted as λ(t). Since the queues in M/M/K Marko-
vian process has a Poisson process and therefore the
inter arrival times should be exponentially distributed.
The memoryless nature of the exponential distribu-
tions ensures that we have a Markov chain even if the
mean of the process is time varying. Therefore,we
set travel times to be exponentially distributed (and
therefore memoryless) with an average length of
¯
η(t),
likewise completion times of the job are also expo-
nentially distributed with
¯
ξ(t). Now define the quan-
tity V (Q,R,t) to be some value function depending on
the stochastic processes followed by Q and R. Given
these stochastic jump processes, we can evaluate what
is the expected change in value of the system over a
small increment of time.
E[dV ] =
∂V
∂t
dt+ (7)
dt

λ(t)(V (Q + 1, R + ψ,t +dt) −V (Q,R,t))+
R
¯
η(t)
(V (Q, R − 1,t + dt) −V (Q,R,t))+
µ(Q,R,t)(V (Q − 1, R + φ,t + dt) −V (Q,R,t))

where µ(Q,R,t) =
min(Q,S)−R
¯
ξ(t)
and
ψ = 1, φ = 0 if Q < S (8)
ψ = 0, φ = 0 else if Q = S
ψ = 0, φ = 1 else if Q > S
3.1 Optimising the Number of
Engineers
In order to optimise the number of engineers, we
need to define some cash value to E[T T ], and off-
set it against the cost of employing engineers. The
value TT has the units [ time jobs] and we assign this
conversion from time jobs to cash or in other words,
the cost of delay in jobs by the variable α. The vari-
able α must have the units [£ time
−1
jobs
−1
] and can
loosely be interpreted as the negative value added to
the system by increasing the total receipt to arrival
time by one unit. If there is some fixed cost to hire
engineers that is paid continuously at the rate ε (units
[£ time
−1
jobs
−1
]) then we must set
dV = −α(R + max(Q − S,0))dt − εSdt. (9)
Equation (9) is used to calculate the total cost of hir-
ing S engineers plus the cost of T T over the time pe-
riod t ∈ [t
0
,t
K
]. Obviously we would like to find the
strategy that hires engineers in some sort of shift pat-
tern in order to cover the jobs in the optimally. In
order to do this, we must allow S to vary with time -
in fact S will be our control variable. We need to con-
sider what happens when the value of S is increased,
decreased or stays the same throughout time. This
means that our value function will now become an
implicit function of the control variable S as well as
the other variables, so that V = V (Q,R,t; S).
3.1.1 Increasing the Number of Engineers
Since changing the number of engineers happens over
an instant, we define
+
to mean just after and
−
to
mean just before. If S
+
> S
−
, then we have hired
more engineers. This means that any spare jobs wait-
ing to be assigned can be handed to the new engineers,
which results in
ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems
112

Q
+
= Q
−
and (10)
R
+
= R
−
+ max((min(Q
−
,S
+
) − S
−
),0)
and if there is some fixed cost ε
s
assigned to the engi-
neers starting the shift (this will stop engineers being
used for a short period of time) we have
V (Q
−
,R
−
,t
−
;S
−
) = V (Q
+
,R
+
,t
+
;S
+
) (11)
+ ε
s
|S
+
− S
−
|
3.1.2 Decreasing the Number of Engineers
When an engineer finishes their shift, we can assume
that if they are already on their way to a job then they
must finish it, and those leaving must be either spare
engineers or those already at jobs. This means that
the number of jobs waiting for an engineer and the
number in transit must stay constant. So if S
+
< S
−
then R
−
= R
+
.
R
+
= R
−
and (12)
Q
+
= Q
−
− min(min(Q
−
,S
−
) − R
−
,
max(min(Q
−
,S
−
) − S
+
,0))
and
V (Q
−
,R
−
,t
−
;S
−
) = V (Q
+
,R
+
,t
+
;S
+
) (13)
+ ε
f
|S
+
− S
−
|
where ε
f
is some fixed cost associated with finishing
a shift (maybe equal to ε
f
the average cost of finishing
a job if they are on their way to or yet to finish one).
Furthermore, we now have to consider what happens
to the partial differential equation (PDE) if R > S, a
situation which might arise if engineers finish shifts
whilst they are on their way to a job. In this case we
have to add an extra equation to the calculation of dV
for this case when R > S. We are assuming that any
jobs in which the engineers have arrived are lost from
the system along with any jobs that arrive at their des-
tination in the state R > S. This loss from the system
and the cost of finishing the job is captured by fixed
costs for reducing the number of engineers. Therefore
we have,
dV =
∂V
∂t
dt+
λ(t)dt(V (Q + 1, R,t +dt) −V (Q,R,t))+
R
¯
η(t)
dt(V (Q − 1,R − 1,t +dt) −V (Q,R,t)). (14)
3.1.3 Solving the PDE
Now let us define the operator L such that the combi-
nation of equations (7), (8),(13) and (14) can be cap-
tured by the single equation
∂V
∂t
− L (V ) = −α(R + max(Q − S,0)) − εS (15)
In order to solve and optimise the equations we must
solve backwards in time . The value at the end of
the period should be assigned so that it is subop-
timal to allows jobs to remain unfinished, such as
V (Q,R,t
K
;S) = PQ where P is some penalty for ev-
ery job remaining unfinished. We proceed to solve the
equations as follows. First, given that we have already
solved at t
k+1
, we must solve for all possible values
of Q, R, and S in the state space using the scheme
V
k
(Q,R; S) = V
k+1
(Q,R; S) (16)
+ ∆t
h
L(V
k+1
) + α(R + max(Q − S, 0)) + εS
i
Once all values have been found we can apply the op-
timising condition (here we must minimise the value)
by checking at each Q, R, what the optimal number of
engineers is. This can be written as
(V
∗
)
k
(Q
−
,R
−
) = min
E
h
V
k
(Q
+
,R
+
;E) +C
S,E
i
(17)
where C
S,E
is the appropriate cost function from equa-
tions (10)-(13) and V
∗
is the optimal value. Once the
conditions of (10)-(13) are applied in (15) we can then
solve backwards in time until t
0
which will be the op-
timal grid.
4 NUMERICAL RESULTS
4.1 Calculating the Optimal Number of
Engineers through the Day
Unless stated otherwise the results in this section as-
sume that we have
λ(t) = 2.5 if t ∈ [9, 16] (18)
λ(t) = 0 otherwise.
and
¯
η = 1.5,
¯
ξ = 0.5,t
k
= 24ε = 0.4, α = 3.5,ε
s
,ε
f
=
2.5 and K = 480. In figure 1, we plot the mean
path followed by both of the state variables Q and R
in the problem, the corresponding choice of optimal
S
∗
(given the mean path), alongside the intensity at
which jobs appear on the system. We can observe
that S
∗
fluctuates depending on the number of jobs
Q, keeping the total number of engineers close to the
current number of jobs on the system. As such in the
region t ∈ [0,9] we hire no engineers at all since there
is no jobs, then increase the number of engineers up
to time t ∈ [12,16] which is the peak time, and then
gradually decrease numbers until all engineers are fin-
ished their shifts at time t = 21. Given the parameters
as stated, the number of engineers S is always close
to the value of Q, keeping the cost of engineers to
OptimalStaffingPolicy-AServiceSystemwithStochasticTravelTimes
113

0
2
4
6
8
10
0 5 10 15 20 25
# of engineers
Time (Hrs)
S*
Q
lambda
R
Figure 1: The optimal hiring policy S
∗
with
¯
η = 1.5,
¯
ξ =
0.5,t
k
= 24ε = 0.4,α = 3.5ε
s
,ε
f
= 2.5 and K = 480.
0
500
1000
1500
2000
2500
3000
3500
4000
4500
0 5 10 15 20 25
Time (Hrs)
sum of value
optimal hiring policy
S=15
S=6
S=3
Figure 2: The optimal hiring policy of the engineers through
out the day in comparison with S = 3, 6 and 15.
0
2
4
6
8
10
12
14
16
0
2
4
6
8
10
12
14
0
10
20
30
40
50
60
70
value
Optimal number of engineers
# of jobs
# of engineers
value
Figure 3: The optimal hiring policy of the engineers through
out the day.
a minimum. This ensures staffing and prevents jobs
left unattended. However, we can also observe how
the cost of engineers can vary under certain param-
eter choices. In the same figure, we can see that R
fluctuates with the number of of engineers hired. The
difference between the number of engineers hired S
and the travelling engineers R is controlled by
¯
ξ and
¯
η, such that as we increase
¯
η and
¯
ξ then the difference
between S and R decreases and vice versa.
To demonstrate that optimal choices are made for
the number of engineers we compare
min
E
[
Q
max
∑
i=0
Q
max
∑
j=0
V (Q
i
,R
i
,E)] for all E (19)
for each time t and choice of engineer S. with λ = .5
for all t. We can then compare the effect of different
hiring policies with the respective optimal solutions.
In figure 2 we can see that this sum over all values
for constant hiring policies S = 3,6 and 15 is always
higher than the optimal policy S
∗
for all t. Although
this is no way rigorous, it does show the effect of hir-
ing the optimal number of engineers on V (Q, R,t;S).
In figure 3 we set Q
max
= 16. In this figure, we
have V (Q,R,S) at time t = 0 for different values of Q
and R where each represents the expected time it takes
R number of travelling engineers finish Q jobs with
S = 0. Most significantly, for Q = Q
max
and R = 0
or in other words, there is Q job demand and no en-
gineers available to travel, then V (Q, R, S) = 68.9 as
evident from the figure. It represents the expected to-
tal travelling ( or waiting time for calls). Additionally,
we can see that if R > Q then V (Q,R,S) = 0 and all
the jobs will be covered by R engineers and there is
no waiting time. To show the effect of optimizing the
number of engineers, we compare the optimal hiring
policy with S = 6 and S = 15 . As we can see in figure
4, using the same previously mentioned parameters,
V (Q,R,S) at time t = 0 for S = 15 is always higher,
in comparison with the optimal hiring policy S
∗
, for
all values of Q and R. The expected total waiting time
V (Q,R,S) = 2944 for S = 15 is significantly higher
than using the optimal hiring policy. Furthermore, in
figure 5 we compare the optimal solution with S = 6
and in this case we set Q
max
= 20. In this figure, we
can see that the expected travelling time is for S = 6
is higher than the optimal hiring policy for all val-
ues of Q and R where for Q = Q
max
and R = 0 then
V (Q,R,S) = 4002 for S = 6, and V (Q, R, S) = 120.5
for S
∗
.
0
2
4
6
8
10
12
14
16
0
2
4
6
8
10
12
14
16
0
500
1000
1500
2000
2500
3000
value
Optimal number of engineers
S=15
# of jobs
# of engineers
value
Figure 4: The hiring policy of the engineers through out the
day with S = 15.
4.2 The Effects of the Cost Parameters ε
and α
In figure 6, we set λ(t) as in equation (19) and
¯
ξ =
0.5,
¯
η = 1.5α = 3.5,K = 480, ε
s
= 2.5,ε
f
= 2.5 with
the maximum of 10 engineers. As we decrease ε then
the optimal number of engineers required to complete
ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems
114

0
5
10
15
20
0
5
10
15
20
0
500
1000
1500
2000
2500
3000
3500
4000
value
Optimal number of engineers
without optimization with =6
# of jobs
# of engineer
value
S
Figure 5: The optimal hiring policy of the engineers through
out the day with S = 6.
0
2
4
6
8
10
0 5 10 15 20
# of engineers
Time (Hrs)
S* with epsilon=0.
Q
lambda
S* with epsilon=10
S* with epsilon=5
Figure 6: The effect of the cost employing ε the optimal
hiring policy of engineers through out the day.
the job increases. This is because ε is the cost of hir-
ing and because it becomes cheaper to hire engineers
therefore we can invest in more of them. We notice
that for ε = 0 it is optimal to hire S = 10 for all t
since it is effectively free to hire them. For ε = 5 and
ε = 10 the number of engineers hired are actually less
than the number of jobs Q since it is too expensive to
hire engineers.
5 CONCLUSIONS
In this paper, we have developed a model to optimize
the number of engineers hired by the response service
operator. We modelled the arrival and completion of
jobs of a response as M/M/K queues using a time
dynamic stochastic processes and allowed the opera-
tor to the control the number of staff. We defined the
problem as a jump process PDE and categorized the
jump into three different types depending on the cur-
rent state of the system. The structure of the model
is developed from queueing theory but we put a real
options style slant on the model in which the operator
must invest fixed costs to start and end the shift pat-
terns of their staff. The main results show that the op-
timal staffing policy can effectively balance the costs
of employing staff with the total waiting time, allow-
ing an operator to find the balance of staffing levels
required to meet the regulatory targets.
REFERENCES
Aksin, Z., Armony, M., and Mehrotra, V. (2007). The mod-
ern call center: A multi-disciplinary perspective on oper-
ations management research. Production and Operations
Management, 16(6):665–688.
Blackstone, E. A., Buck, A. J., and Hakim, S. (2007).
The economics of emergency response. Policy Sciences,
40(4):pp. 313–334.
Brooks, J., Edwards, D., Sorrell, T., Srinivasan, S., and
Diehl, R. (2011). Simulating calls for service for an ur-
ban police department. In Simulation Conference (WSC),
Proceedings of the 2011 Winter, pages 1770 –1777.
Evans, D. H., Herman, R., and Weiss, G. H. (1964). The
highway merging and queuing problem. Operations Re-
search, 12(6):pp. 832–857.
Fry MJ, R. U. (2006). Firefighter staffing including tem-
porary absences and wastage. journal of operation re-
search, 54(2):pp. 351–365.
Marianov, V. and ReVelle, C. (1996). The queueing maxi-
mal availability location problem: A model for the siting
of emergency vehicles. European Journal of Operational
Research, 93(1):110 – 120.
Singer, M. and Donoso, P. (2008). Assessing an ambulance
service with queuing theory. Computers & operations
research, 35(8):2549–2560.
Whitt, W. (2005). Engineering solution of a basic call-
center model. Management Science, 51(2):pp. 221–235.
OptimalStaffingPolicy-AServiceSystemwithStochasticTravelTimes
115