Quantile Estimation When Applying Conditional Monte Carlo

Marvin K. Nakayama

Computer Science Department, New Jersey Institute of Technology, Newark, NJ 07102, U.S.A.

Keywords:

Quantile, Value-at-Risk, Variance Reduction, Conditional Monte Carlo, Conﬁdence Interval.

Abstract:

We describe how to use conditional Monte Carlo (CMC) to estimate a quantile. CMC is a variance-reduction

technique that reduces variance by analytically integrating out some of the variability. We show that the

CMC quantile estimator satisﬁes a central limit theorem and Bahadur representation. We also develop three

asymptotically valid conﬁdence intervals (CIs) for a quantile. One CI is based on a ﬁnite-difference estima-

tor, another uses batching, and the third applies sectioning. We present numerical results demonstrating the

effectiveness of CMC.

1 INTRODUCTION

For a continuous random variable X with a strictly

increasing cumulative distribution function (CDF) F

and ﬁxed 0 < p < 1, the p-quantile of X is deﬁned as

the constant ξ such that P(X ≤ ξ) = p. A well-known

example is the median, which is the 0.5-quantile. The

p-quantile ξ also can equivalently be expressed as ξ =

−1

(p).

Quantiles are often used in application areas to

measure risk. For example, in ﬁnance, a quantile is

known as a value-at-risk, and quantiles are widely em-

ployed to assess portfolio risk. For example, bank-

ing regulations specify capital requirements for a ﬁrm

in terms of 0.99-quantiles of the random loss (Jorion,

2007). In nuclear engineering, safety and uncertainty

analyses are often performed with a 0.95-quantile

(U.S. Nuclear Regulatory Commission, 1989).

Suppose that we have a simulation model that out-

puts a random variable X. When applying simple ran-

dom sampling (SRS), the typical approach to estimate

the p-quantile ξ is to run independent and identically

distributed (i.i.d.) replications of the model, and form

an estimator of the CDF from the sample outputs. In-

verting the CDF estimator yields a quantile estimator.

Because of the noise inherent in any stochastic

simulation, the quantile estimator has some error,

which should be measured. A standard way of as-

sessing the error is by forming a conﬁdence interval

for the true quantile ξ. For example, the U.S. Nu-

clear Regulatory Commission requires nuclear plant

licensees to satisfy a so-called 95/95 criterion, which

entails establishing, with 95% conﬁdence, that the

0.95-quantile lies below a mandated threshold; see

Section 24.9 of (U.S. Nuclear Regulatory Commis-

sion, 2011). Thus, we need not only a point estimate

of a quantile but also a conﬁdence interval for it.

There are several approaches to construct a CI

when applying SRS. One technique, which is some-

times called the nonparametric method, exploits a bi-

nomial property of the i.i.d. sample; see Section 2.6.1

of (Serﬂing, 1980). Another way ﬁrst shows that

the quantile estimator satisﬁes a central limit theorem

(CLT), and then unfolds the CLT to obtain a CI. The

key to applying this technique is consistently estimat-

ing the asymptotic variance constant appearing in the

CLT; approaches for accomplishing this include using

a ﬁnite difference (Bloch and Gastwirth, 1968; Boﬁn-

ger, 1975) and kernel methods (Falk, 1986). Rather

than consistently estimating the asymptotic variance,

we can instead apply batching or sectioning, the latter

of which was originally developed for SRS in Sec-

tion III.5a of (Asmussen and Glynn, 2007) and ex-

tended in (Nakayama, 2014a) to work when applying

the variance-reductiontechniques control variates and

importance sampling. Batching and sectioning divide

the i.i.d. outputs into independent batches, computing

a quantile estimator from each batch, and construct-

ing a CI from the batch quantile estimators.

In this paper, we use conditional Monte Carlo to

estimate a quantile. CMC reduces variance (com-

pared to SRS) by analytically integrating out the vari-

ability that remains after conditioning on an auxiliary

random variable Y; e.g., see Section 8.3 of (Ross,

2006) or Section V.4 of (Asmussen and Glynn, 2007).

We prove that the CMC quantile estimator satisﬁes a

280

K. Nakayama M..

Quantile Estimation When Applying Conditional Monte Carlo.

DOI: 10.5220/0005109702800285

In Proceedings of the 4th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH-2014),

pages 280-285

ISBN: 978-989-758-038-3

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

CLT and a Bahadur representation (Bahadur, 1966).

The latter shows that a quantile estimator can be ap-

proximated as the true quantile plus a linear transfor-

mation of the corresponding CDF estimator, with a

remainder term that vanishes at some rate as the sam-

ple size grows. Since the CDF estimator is typically

a sample average, it satisﬁes a CLT under appropri-

ate conditions. Thus, the Bahadur representation pro-

vides insight into why a quantile estimator, which is

not a sample average, satisﬁes a CLT. It also allows us

to construct asymptotically valid CIs for ξ by using a

ﬁnite difference or sectioning, and we develop those

CIs in this paper.

CMC has previously been employed to derive an

estimator of a sensitivity of a quantile with respect to

a model parameter (Fu et al., 2009). For example,

suppose a ﬁnancial investor has a portfolio of loans,

each of which may default. The investor may want to

estimate the sensitivity of the 0.99-quantile of the loss

of the portfolio, where the sensitivity is taken with

respect to a parameter of the loss distribution of an

individual obligor. While (Fu et al., 2009) apply CMC

to estimate quantile sensitivities, the method has not

been used (to the best of our knowledge) to estimate

the quantile itself.

The rest of the paper develops as follows. Sec-

tion 2 reviews how to apply SRS to estimate and con-

struct CIs for a quantile ξ. Section 3 develops our

CMC estimator of a quantile, shows that it satisﬁes a

CLT and Bahadur representation, and uses these re-

sults to construct CIs for ξ. Section 4 presents nu-

merical results from a simple model, and we provide

concluding remarks in Section 5. Proofs of the results

are given in (Nakayama, 2014b).

2 SIMPLE RANDOM SAMPLING

Let X be a random variable with CDF F. We ﬁrst re-

view how to estimate and construct conﬁdence inter-

vals for the p-quantile ξ = F

−1

(p) ≡ inf{x : F(x) ≥

p} of F (or equivalently of X) for a ﬁxed 0 < p < 1

when applying simple random sampling (SRS).

Let X

, i = 1, 2,.. .,n, be a sample of n i.i.d. ob-

servations from F. The SRS estimator of F(x) =

E[I(X ≤ x)] is the empirical distribution function

(x) = (1/n)

∑

i=1

I(X

≤ x). The SRS p-quantile es-

timator is ξ

= F

−1

(p). We can alternatively compute

by ﬁrst sorting the sample X

,... ,X

into the or-

der statistics X

(1)

≤X

(2)

≤··· ≤X

(n)

, and then setting

= X

(⌈np⌉)

, where ⌈·⌉ denotes the ceiling function.

Section 2.3 of (Serﬂing, 1980) provides an

overview of ξ

and its properties. For example, let

f denote the derivative (when it exists) of F. If

f(ξ) > 0, then ξ

satisﬁes the following CLT:

√

n(ξ

−ξ) ⇒ N(0, p(1−p)/ f

(ξ)) (1)

as n → ∞, where ⇒ denotes convergence in distri-

bution (e.g., see Chapter 5 of (Billingsley, 1995)),

and N(a,b

) is a normal random variable with mean

a and variance b

. Moreover, ξ

also satisﬁes a

so-called (weak) Bahadur representation (Bahadur,

1966; Ghosh, 1971):

= ξ−

(ξ) − p

f(ξ)

+ R

′

, with

√

′

⇒ 0 (2)

as n → ∞. By (2), the left side of (1) equals

−

√

n(F

(ξ) − p)/ f(ξ) +

√

′

, where the ﬁrst term

converges weakly to the right side of (1), and the sec-

ond weakly vanishes by (2). Thus, the Bahadur repre-

sentation provides insight into why ξ

, which is not a

sample average, satisﬁes a CLT, as it can be approxi-

mated in terms of the empirical distribution, which is

a sample mean.

(Ghosh, 1971) also establishes a version of (2) for

the p

-quantile with perturbed p

that converges to

p, rather than the p-quantile for ﬁxed p. This vari-

ation can be useful for constructing a consistent es-

timator of λ ≡ 1/ f(ξ), which appears in the asymp-

totic variance in (1) and can be used to construct a

conﬁdence interval for ξ. If f(ξ) > 0, then for any

= p+ O(n

−1/2

), the SRS estimator F

−1

) of the

-quantile F

−1

) satisﬁes

−1

) = ξ

′

−

(ξ) − p

f(ξ)

′

, with

√

⇒0

(3)

as n → ∞, where

′

= ξ+ (p

− p)/ f(ξ). (4)

To see how to use these results to consis-

tently estimate λ, ﬁrst note that λ =

−1

(p) =

lim

h→0

−1

(p+ h) −F

−1

(p−h)]/2h. This suggests

estimating λ with the ﬁnite difference

−1

(p+ h

) −F

−1

(p−h

)

, (5)

where h

> 0 is known as the bandwidth. The terms in

the numerator of the ﬁnite difference are precisely in

the form of (3) with p

= p±h

, which allows prov-

ing λ

⇒ λ as n → ∞ when h

= cn

−1/2

for any con-

stant c > 0; see Section 2.6.3 of (Serﬂing, 1980). Us-

ing a different proof technique, (Bloch and Gastwirth,

1968) and (Boﬁnger, 1975) also show λ

⇒ λ when

f is continuous in a neighborhood of ξ, and h

→ 0

and nh

→ ∞ as n → ∞. Thus, unfolding the CLT

in (1) leads to the following two-sided (1 −α)-level

(0 < α < 1) conﬁdence interval for ξ:

= [ξ

±z

p(1− p)λ

√

n], (6)

QuantileEstimationWhenApplyingConditionalMonteCarlo

281

where z

= Φ

−1

(1−α/2) and Φ is the CDF of a stan-

dard (mean 0 and unit variance) normal. The CI I

is asymptotically valid in the sense that P(ξ ∈ I

) →

1−α as n → ∞.

Determining an appropriate value for the band-

width h

in the ﬁnite difference can be difﬁcult in

practice. Alternatively, we can avoid trying to con-

sistently estimate λ by instead applying batching or

sectioning. In batching, we divide the n outputs

,... ,X

into b ≥ 2 equal-sized batches, where

the jth batch, j = 1,2,.. .,b, consists of the m =

n/b outputs X

( j−1)m+i

, i = 1,2,. .. ,m. A reasonable

choice for the number of batches is b = 10. For

each batch j, we deﬁne the CDF estimator F

j,m

(x) =

(1/m)

∑

i=1

I(X

( j−1)m+i

≤ x) and corresponding p-

quantile estimator ξ

j,m

= F

−1

j,m

(p). Since the n out-

puts are i.i.d., the b batches are i.i.d., so ξ

j,m

, j =

1,2, . ..,b, are i.i.d. We compute their sample aver-

age

b,m

= (1/b)

∑

j=1

j,m

and their sample variance

b,m

= (1/(b−1))

∑

j=1

(ξ

j,m

−

b,m

)

. An asymptot-

ically valid (as m →∞ with b ≥2 ﬁxed) (1−α)-level

CI for ξ using batching is then

= [

b,m

±t

b,m

√

b],

where t

= T

−1

b−1

(1 −α/2) and T

b−1

is the CDF of a

Student t distribution with b−1 degrees of freedom.

Similar to batching, sectioning was originally de-

veloped in Section III.5a of (Asmussen and Glynn,

2007) for SRS, and it replaces the batching point es-

timator

b,m

with the overall quantile estimator ξ

Speciﬁcally, let S

′2

b,m

= (1/(b −1))

∑

j=1

(ξ

j,m

−ξ

)

and the sectioning two-sided (1 −α)-level CI for ξ

when applying SRS is

′

= [ξ

±t

′

b,m

√

b].

The asymptotic validity of J

′

can be established by

exploiting the Bahadur representation in (2) for ﬁxed

= p. An advantage of sectioning over batching

arises from the fact that quantile estimators are gen-

erally biased. While the bias decreases (nonmono-

tonically) to zero as the sample size n increases, it

can be signiﬁcant for small sample sizes. The bias

of the batching point estimator

b,m

is determined by

the batch size m = n/b < n, so

b,m

can be consid-

erably more biased than the overall quantile estima-

tor ξ

, which has bias governed by the overall sample

size n. Since the sectioning CI is centered at a less-

biased point ξ

, whereas the batching CI is centered

b,m

, the sectioning CI typically has better cover-

age than the batching CI for a ﬁxed overall sample

size n = bm; see the numerical results in Section 4.

3 CONDITIONAL MONTE

CARLO

Now suppose that (X,Y) is a random vector with

joint distribution H. As before, we want to estimate

ξ = F

−1

(p), where F again denotes the (marginal)

distribution of X. Let (X

), i = 1, 2,... ,n, be a sam-

ple of n i.i.d. pairs from H.

Since F(x) = E[E[I(X ≤ x)|Y ]] = E[P(X ≤

x|Y)], a conditional Monte Carlo estimator of F(x)

(x) =

∑

i=1

E[I(X

≤ x)|Y

] =

∑

i=1

G(Y

,x), (7)

where G(Y, x) = P(X ≤ x|Y). The CMC p-quantile

estimator is

−1

(p). Applying CMC relies crit-

ically on being able to compute G and invert

Computing

−1

(p) for CMC appears to be more

involved than for SRS or the other variance-reduction

techniques examined in (Chu and Nakayama, 2012).

For example, consider the simple case when (X,Y)

has a bivariate normal distribution with zero marginal

means, unit marginal variances, and correlation ρ.

The conditional distribution of X given Y = y is

N(ρy,1−ρ

) (e.g., see pp. 167–168 of (Mood et al.,

1974)), so

G(Y,x) = P(X ≤ x|Y) = Φ

x−ρY

1−ρ

, (8)

and

(x) =

∑

i=1

x−ρY

1−ρ

Identifying

such that

(

) = p, i.e.,

−1

(p),

can be accomplished using a root-ﬁnding algorithm,

e.g., Newton’s method, the secant method or the false-

position method; e.g., see Sections 7.1 and 7.2 of

(Ortega and Rheinboldt, 1987). In contrast to the

secant and false-position methods, Newton’s method

requires computing the derivative of

. Given

,... ,Y

, note that

(x) is strictly increasing and

differentiable in x, with sample-path derivative

(x) =

∑

i=1

(1−ρ

)

−1/2

x−ρY

1−ρ

where φ is the density of a standard normal.

The following shows that the CMC p

-quantile es-

timator

−1

) satisﬁes a Bahadur representation.

Theorem 1. If f(ξ) > 0, then for any p

= p +

O(n

−1/2

), the CMC p

-quantile estimator

−1

)

satisﬁes

−1

) = ξ

′

−

(ξ) − p

f(ξ)

+ R

, with

√

⇒ 0

(9)

SIMULTECH2014-4thInternationalConferenceonSimulationandModelingMethodologies,Technologiesand

Applications

282

as n →∞, where ξ

′

is given in (4).

In particular, the results of Theorem 1 hold for

= p ﬁxed, in which case ξ

′

= ξ. It then fol-

lows from (9) that the CMC p-quantile estimator

−1

(p) satisﬁes the following CLT:

√

−ξ) = −

√

f(ξ)

(

(ξ) − p) +

√

⇒ N(0,τ

) (10)

as n → ∞, where

Var[G(Y, ξ)]

(ξ)

. (11)

The numerator in the right side of (11) arises

since

(ξ) is the sample average of the i.i.d.

G(Y

,ξ), i = 1,2,. . .,n. By the well-known variance-

decomposition formula (e.g., see Section 2.10 of

(Ross, 2006)), we have that

p(1− p) = Var[I(X ≤ ξ)]

= E[Var[I(X ≤ ξ)|Y]] + Var[E[I(X ≤ ξ)|Y]]

≥ Var[E[I(X ≤ ξ)|Y]] = Var[G(Y,ξ)],

(12)

where the inequality follows from the nonnegativity

of (conditional) variance. Thus, comparing (1) with

(10) and (11), we see that the CMC p-quantile estima-

tor

has smaller asymptotic variance than the SRS

p-quantile estimator ξ

The CLT in (10) can be unfolded to construct an

asymptotically valid conﬁdence interval for ξ if we

can consistently estimate τ

in (11). Theorem 1 can

be used to prove that the ﬁnite difference

−1

(p+ h

) −

−1

(p−h

)

(13)

satisﬁes

⇒ λ = 1/ f(ξ) as n → ∞ when the band-

width h

= c/

√

n for any constant c > 0. We can con-

sistently estimate the numerator ψ

≡ Var[G(Y, ξ)] in

(11) via

n−1

∑

i=1

G(Y

) −

where

= (1/n)

∑

i=1

G(Y

). If G(y,x) is continu-

ous in x for each y, then

= p since

(

) = p. The

proof of the consistency of

is complicated by the

fact that G(Y

), i = 1, 2,.. .,n, are not i.i.d. because

they all depend on

, which in turn is a function of all

, i = 1, 2,.. .,n. But this can be handled employing

arguments developed in (Chu and Nakayama, 2012).

Now we can consistently estimate τ using

so an asymptotically valid two-sided CI for ξ is

= [

±z

√

n]. (14)

As with SRS, choosing an appropriate value for

the bandwidth in the ﬁnite-difference

can be dif-

ﬁcult when applying CMC, and we may instead ap-

ply batching or sectioning to construct a CI for ξ

with CMC. For batching, we divide the G(Y

,·), i =

1,2, . ..,n, into b ≥ 2 nonoverlapping batches, each

of size m = n/b. As with SRS, a reasonable choice

for the number of batches is b = 10. For each j =

1,2, . ..,b, the jth batch consists of G(Y

( j−1)m+i

,·),

i = 1,2,... , m, which we use to compute a CDF es-

timator

j,m

, with

j,m

(x) =

∑

i=1

G(Y

( j−1)m+i

,x),

and p-quantile estimator

j,m

−1

j,m

(p). The b

batch quantile estimators

j,m

, j = 1,2, ..., b, are

i.i.d., and we compute their sample average

b,m

(1/b)

∑

j=1

j,m

and sample variance

b,m

= (1/(b −

1))

∑

j=1

(

j,m

−

b,m

)

. The batching CI for ξ when

applying CMC is then

b,m

= [

b,m

±t

b,m

√

b].

Because of the bias of quantile estimators, it is

often better to apply sectioning instead of batching

when n is small, where we again replace the batching

point estimator

b,m

with the overall quantile estima-

tor

. Deﬁne

b,m

= (1/(b −1))

∑

j=1

(

j,m

−

)

and the sectioning CI for ξ when applying CMC is

then

b,m

= [

±t

b,m

√

b].

As with SRS, when applying CMC, the sectioning CI

b,m

should have better coverage than the batching CI

b,m

for ﬁxed overall sample size n = bm.

The following result establishes the asymptotic

validity of the CMC CIs.

Theorem 2. Suppose f(ξ) > 0. Then the following

hold:

(i) P(ξ ∈

b,m

) → 1 −α and P(ξ ∈

b,m

) → 1 −α as

m →∞ with b ≥2 ﬁxed.

(ii) If the bandwidth h

= cn

−1/2

in (13) for any con-

stant c > 0, then P(ξ ∈

) → 1−α as n →∞.

4 NUMERICAL RESULTS

We next present numerical results from simulation

experiments on the bivariate normal discussed in

Section 3. Recall (X,Y) is bivariate normal, with

marginal means 0, unit marginal variances, and cor-

relation ρ = 0.5. Our goal is to estimate and construct

QuantileEstimationWhenApplyingConditionalMonteCarlo

283

CIs for the p-quantile ξ of X for different values of

p and different sample sizes n. Tables 1 and 2 con-

tain the results when applying simple random sam-

pling and conditional Monte Carlo, respectively, giv-

ing the estimated coverage of nominal 90% CIs for ξ

and the average half widths (AHWs) of the CIs from

independent replications, where we use different

methods to construct the CIs.

Table 1: Coverage (and average half width) of nominal 90%

conﬁdence intervals for the p-quantile of a standard nor-

mal X when applying simple random sampling (SRS) with

bandwidth h

= 0.2n

−1/2

and b = 10 batches.

p = 0.8

n Exact FD Batch Section

100 0.900 0.808 0.620 0.903

(0.235) (0.230) (0.235) (0.260)

400 0.897 0.851 0.821 0.909

(0.118) (0.117) (0.125) (0.129)

1600 0.898 0.875 0.876 0.901

(0.059) (0.059) (0.063) (0.064)

6400 0.902 0.877 0.898 0.905

(0.029) (0.028) (0.032) (0.032)

p = 0.95

n Exact FD Batch Section

100 0.902 0.799 0.825 0.861

(0.348) (0.330) (0.330) (0.340)

400 0.902 0.848 0.646 0.900

(0.174) (0.172) (0.171) (0.188)

1600 0.900 0.878 0.830 0.900

(0.087) (0.087) (0.092) (0.095)

6400 0.901 0.891 0.883 0.902

(0.043) (0.044) (0.047) (0.047)

p = 0.99

n Exact FD Batch Section

100 0.926 0.497 0.024 0.762

(0.614) (0.325) (0.330) (0.502)

400 0.905 0.913 0.696 0.841

(0.307) (0.405) (0.267) (0.284)

1600 0.907 0.891 0.907 0.907

(0.154) (0.160) (0.164) (0.168)

6400 0.902 0.893 0.887 0.902

(0.077) (0.077) (0.082) (0.083)

In each table, the column labeled “Exact” con-

tains the results for the CIs in (6) and (14) but where

we replace the ﬁnite difference estimator of λ with

its exact value. This method is typically not imple-

mentable in practice since λ is usually unknown, but

we include results for it as a benchmark to which we

compare the others. For the ﬁnite difference (FD),

we use the bandwidth h

= 0.2/

√

n in (5) and (13).

When p ≈ 1 and n is small, we can have p + h

> 1,

Table 2: Coverage (and average half width) of nominal

90% conﬁdence intervals for the p-quantile of X of a bi-

variate normal (X,Y) with ρ = 0.5 when applying condi-

tional Monte Carlo (CMC) with bandwidth h

= 0.2n

−1/2

and b = 10 batches.

p = 0.8

n Exact FD Batch Section

100 0.896 0.895 0.892 0.898

(0.087) (0.087) (0.093) (0.093)

400 0.899 0.899 0.899 0.898

(0.043) (0.043) (0.047) (0.047)

1600 0.900 0.899 0.897 0.897

(0.021) (0.021) (0.023) (0.023)

6400 0.896 0.896 0.896 0.896

(0.011) (0.011) (0.012) (0.012)

p = 0.95

n Exact FD Batch Section

100 0.889 0.898 0.873 0.895

(0.105) (0.109) (0.102) (0.103)

400 0.897 0.898 0.893 0.898

(0.049) (0.050) (0.052) (0.052)

1600 0.892 0.893 0.895 0.897

(0.024) (0.024) (0.026) (0.026)

6400 0.895 0.895 0.896 0.897

(0.012) (0.012) (0.013) (0.013)

p = 0.99

n Exact FD Batch Section

100 0.888 0.944 0.822 0.886

(0.172) (0.259) (0.114) (0.116)

400 0.893 0.967 0.882 0.894

(0.065) (0.097) (0.060) (0.061)

1600 0.889 0.913 0.893 0.896

(0.029) (0.032) (0.031) (0.031)

6400 0.893 0.898 0.897 0.897

(0.014) (0.015) (0.015) (0.015)

so the ﬁnite differences (5) and (13) become unde-

ﬁned since the inverse of the estimated CDF is eval-

uated outside of its domain. In these cases, we re-

place p+ h

and p−h

with q

≡ 1−(1− p)/10 and

≡ 2p −1 + (1 − p)/10, respectively, where q

chosen so that q

and q

are symmetric around p; the

denominator in the ﬁnite difference is then q

−q

The columns labeled “Batch” and “Section” are for

batching and sectioning, respectively, with b = 10

batches. Numerical results in (Nakayama, 2014a)

with b = 10 and b = 20 reveal that b = 20 often leads

to poorer coverage than b = 10 for small n.

In general, we see that in both tables, the cover-

ages converge to the nominal level as n gets large,

demonstrating the CIs’ asymptotic validity. When

p ≈1 and n is small, sectioning generally gives better

coverage than batching because the former centers its

SIMULTECH2014-4thInternationalConferenceonSimulationandModelingMethodologies,Technologiesand

Applications

284

CI at a less-biased point. Sectioning also outperforms

FD in terms of coverage. Comparing FD and Exact

for p = 0.99 and small n, we see that the AHW of

FD typically is quite different from AHW for Exact,

which indicates that in these cases, FD does a poor

job estimating λ, resulting in FD’s poor coverage.

Relative to SRS, CMC reduces the AHW about

60% (resp., 70% and 80%) for p = 0.8 (resp., p =

0.95 and p = 0.99). Thus, the variance reduction from

CMC improves as we consider more extreme quan-

tiles. For each of the smaller values of n, the cov-

erage for each SRS CI (except Exact) worsens as p

increases. While CMC coverage also degrades some-

what as p approaches 1, the impact is much less pro-

nounced. Also, for large n, the slightly wider AHW

for batching and sectioning compared to Exact and

FD arises because the former two methods are based

on a Student t limit, whereas the latter two rely on a

normal limit, which has lighter tails.

5 CONCLUSIONS

We developed an estimator of a quantile ξ using con-

ditional Monte Carlo, which is guaranteed to reduce

asymptotic variance compared to simple random sam-

pling. We established that the CMC quantile estima-

tor satisﬁes a weak Bahadur representation, which im-

plies a CLT holds. We used these results to produce

three asymptotically valid conﬁdence intervals for ξ

as the sample size n →∞. The CIs are based on batch-

ing, sectioning and a ﬁnite difference. Our numerical

results seem to indicate that compared to SRS, CMC

not only reduces variance, but it also leads to CIs with

better coverage. For both SRS and CMC, the sec-

tioning CI has better coverage than the batching and

ﬁnite-difference intervals for small n, especially when

p ≈ 1. Thus, of the three CIs we proposed, we recom-

mend using sectioning.

ACKNOWLEDGEMENTS

This work has been supported in part by the Na-

tional Science Foundation under Grants No. CMMI-

0926949, CMMI-1200065, and DMS-1331010. Any

opinions, ﬁndings, and conclusions or recommenda-

tions expressed in this material are those of the author

and do not necessarily reﬂect the views of the Na-

tional Science Foundation.

REFERENCES

Asmussen, S. and Glynn, P. (2007). Stochastic Simulation:

Algorithms and Analysis. Springer, New York.

Bahadur, R. R. (1966). A note on quantiles in large samples.

Annals of Mathematical Statistics, 37:577–580.

Billingsley, P. (1995). Probability and Measure. John Wiley

& Sons, New York, third edition.

Bloch, D. A. and Gastwirth, J. L. (1968). On a simple esti-

mate of the reciprocal of the density function. Annals

of Mathematical Statistics, 39:1083–1085.

Boﬁnger, E. (1975). Estimation of a density function using

order statistics. Australian Journal of Statistics, 17:1–

Chu, F. and Nakayama, M. K. (2012). Conﬁdence intervals

for quantiles when applying variance-reduction tech-

niques. ACM Transactions On Modeling and Com-

puter Simulation, 36:Article 7 (25 pages plus 12–page

online–only appendix).

Falk, M. (1986). On the estimation of the quantile density

function. Statistics & Probability Letters, 4:69–73.

Fu, M. C., Hong, L. J., and Hu, J.-Q. (2009). Conditional

Monte Carlo estimation of quantile sensitivities. Man-

agement Science, 55:2019–2027.

Ghosh, J. K. (1971). A new proof of the Bahadur repre-

sentation of quantiles and an application. Annals of

Mathematical Statistics, 42:1957–1961.

Jorion, P. (2007). Value at Risk: The New Benchmark for

Managing Financial Risk, 3rd Edition. McGraw-Hill.

Mood, A. M., Graybill, F. A., and Boes, D. C. (1974). In-

troduction to the Theory of Statistics. McGraw-Hill,

New York, 3rd edition.

Nakayama, M. K. (2014a). Conﬁdence intervals us-

ing sectioning for quantiles when applying variance-

reduction techniques. ACM Transactions on Modeling

and Computer Simulation. To appear.

Nakayama, M. K. (2014b). Efﬁcient quantile estimation

using conditional Monte Carlo. In preparation.

Ortega, J. M. and Rheinboldt, W. C. (1987). terative So-

lution of Nonlinear Equations in Several Variables.

SIAM.

Ross, S. M. (2006). Simulation. Academic Press, San

Diego, CA, fourth edition.

Serﬂing, R. J. (1980). Approximation Theorems of Mathe-

matical Statistics. John Wiley & Sons, New York.

U.S. Nuclear Regulatory Commission (1989). Best-

estimate calculations of emergency core cooling per-

formance. Nuclear Regulatory Commission Regula-

tory Guide 1.157, U.S. Nuclear Regulatory Commis-

sion, Washington, DC.

U.S. Nuclear Regulatory Commission (2011). Applying

statistics. U.S. Nuclear Regulatory Commission Re-

port NUREG-1475, Rev 1, U.S. Nuclear Regulatory

Commission, Washington, DC.

QuantileEstimationWhenApplyingConditionalMonteCarlo

285