An Empirical Comparison of DEA and SFA Method to Measure

Hospital Units’ Efficiency

George Katharakis

, Maria Katharaki

and Theofanis Katostaras

Faculty of Nursing, National and Kapodistrian University of Athens, 23 Papadiamantopoulou Str., Athens, Greece

Department of Economic Sciences, National and Kapodistrian University of Athens, 8 Pesmazoglou Str., Athens, Greece

Keywords: Efficiency, Data Envelopment Analysis (DEA), Stochastic Frontier Analysis (SFA), Bootstrap Analysis,

Translog Form, OR in Health.

Abstract: Although frontier techniques have been used to measure healthcare efficiency, their utility in decision

making process is limited by both methodological questions concerning their application. The present paper

aims to examine the data envelopment analysis (DEA) and stochastic frontier analysis (SFA) results in order

to facilitate a common understanding about the adequacy of these methods. A two-stage bootstrap DEA

method and the Translog formula of the SFA were performed. Multi-inputs and multi-outputs were used in

both of the approaches assuming two scenarios either including environmental variables or not. Thirty-two

Greek public hospital units constitute the sample. The main output of the analysis was that the efficiency

scores increased with the incorporation of environmental variables. Moreover, environmental variables

being hospital status and geographical position were found significantly correlating with inefficiency, while

patient mobility was not found strongly correlating. DEA and SFA were found to yield divergent efficiency

estimates due to the nature of the environmental variables and the measurement error. The analysis

concludes that there is a need for careful attention by stakeholders since the nature of the data and its

availability influence the measurement of the efficiency and thus it is necessary to be specific when

choosing the mathematical form.

1 INTRODUCTION

1.1 Background

Governments all over the world face the difficult

task of managing the complexities of controlling

healthcare costs while at the same time ensuring that

patients receive not only a high quality of care, but

also that this care is delivered as efficiently as

possible (Katharaki, 2008). As a result, payers and

purchasers have begun to use frontier efficiency

measurement techniques in order to measure the

performance of the healthcare sector with the aim of

supporting their decisions on healthcare units’

performance. More commonly used techniques are

data envelopment analysis (DEA) and stochastic

frontier analysis (SFA) which employ quite distinct

methodologies for frontier estimation and efficiency

measurement, each with associated strengths and

weaknesses. Specifically, “…non-statistical

approaches such as DEA have the disadvantage of

assuming no statistical noise, but have the advantage

of being non-parametric and requiring few

assumptions about the underlying technology. SFA

models on the other hand have the attraction of

allowing for statistical noise, but have the

disadvantage of requiring strong assumptions as to

the form of the frontier” (Jacobs, 2001, p.3). DEA is

favored where measurement error is unlikely to pose

much of a threat and where the assumptions of

neoclassical production theory are in question.

Conversely, SFA should have the advantage in

coping with severe measurement error and where

simple functional forms provide a close match to the

properties of the underlying production technology.

Gong and Sickles (1992) report findings along

similar lines so that “...as mis-specification of

functional form becomes more serious, DEA’s

appeal (vis-à-vis SFA) becomes more compelling”

(p.259).

Hospital units evaluations have to date been

carried out using mostly DEA-based methodologies.

During the last twenty years, non-parametric and

parametric methods have been increasingly

Katharakis G., Katharaki M. and Katostaras T..

An Empirical Comparison of DEA and SFA Method to Measure Hospital Units’ Efﬁciency.

DOI: 10.5220/0004274502420251

In Proceedings of the 2nd International Conference on Operations Research and Enterprise Systems (ICORES-2013), pages 242-251

ISBN: 978-989-8565-40-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

employed to measure and analyze the productive

performance of healthcare services. The healthcare

sector is a unique area of application, and one in

which the measurement of efficiency has burgeoned

over the past few years. Mortimer (2002)

highlighted the need for parallel application of

competing methods for frontier estimation and

efficiency measurement.

Thereby, in the efficiency analysis literature

there has been considerable interest in reconciling

SFA and DEA (Mutter et al., 2011). Both studies of

Chirikos and Sear (2000) and Jacobs (2001) that

compared SFA and DEA methods found divergent

estimations between the results. Linna (1998)

examined cost efficiency of Finish hospitals and

found that SFA and DEA generated similar results.

The last 5 years Desaia et al., (2005), Assaf and

Matawie (2008), Lee et al., (2009) share the same

prospect that neither DEA nor SFA can be regarded

as clearly dominant. Likewise, more recent studies,

Nedelea and Fannin (2012), Ippoliti and Falavigna

(2012) suggest that SFA and DEA approaches along

with other techniques are viable alternatives for

analyzing the impact of environmental variables and

dynamic effects on hospital cost efficiency,

generating similar but more consistent results in

empirical application to the efficiency analysis of

healthcare units. Moreover, the majority of the

researchers agree on the need of being aware of

using both DEA and SFA methods, along with

determining the sources of productivity factors by

regressing the efficiency scores against a set of

environmental variables.

1.2 Aim and Scope

Under this context, an empirical application of both

two-stage bootstrap DEA approach of Simar and

Wilson (2007) and SFA with the Translog functional

form (SFA

translog

) on a sample of Greek public

hospital units has been conducted in order to analyze

cost efficiency estimations comparatively. Based on

the fact that SFA is mostly used in literature under

one input and many outputs or the opposite (Bryce,

Engberg and Wholey, 2000; Chirikos and Sear,

2000; Giuffrida and Gravelle, 2001; Jacobs, 2001;

Ondrich and Ruggiero, 2001; Assaf and Matawie,

2008; Lee et al., 2009), in this paper, multi-inputs

and multi-outputs are used in both of the approaches

assuming two scenarios, either including

environmental variables or not. Thus, the analysis is

focused on discussing the results derived by the

models’ applications.

Therefore, the main purpose of our study is to

examine the “behavior” of the two-stage bootstrap

DEA approach and SFA

translog

, and how the two

methods can be used to make valid inferences about

the effects of environmental variables on estimated

cost efficiency. Nevertheless, the present study aims

to highlight the importance of the information

derived with regard to the functional forms of the

DEA and SFA method and therefore what should be

taken into account when applying them in a larger

sample of health units. Hospital managers and

policymakers can become more effective decision

makers by understanding the relationships between

efficiency and these environmental variables.

2 LITERATURE SURVEY

Many researchers have applied methods in order to

evaluate hospital efficiency, such as Data

Envelopment Analysis (DEA) and Stochastic

Frontier Analysis (SFA). These frontier methods use

an efficient frontier to identify the efficiency of

hospital units relative to a reference set of healthcare

units. DEA is a non-parametric approach that uses

mathematical programming to identify the efficient

frontier. SFA is a parametric approach that

hypothesizes a functional form and uses the data to

econometrically estimate the parameters of that

function using the entire set of DMU’s. However,

the two methods differ in some key theoretical

aspects. DEA measures efficiency relative to a

nonparametric estimate of an unobserved true

frontier, conditional on observed data (Simar and

Wilson, 2007). On the other hand, SFA requires the

specification of a functional form for the frontier,

and assumptions about the distributions of the

random error and inefficiency error terms, which

might be very restrictive (Newhouse, 1994).

In their systematic literature review regarding the

investigation of the results derived from at least one

pair-wise comparison of the DEA/SFA methods,

Katharakis and Katostaras (2012) highlighted that

both approaches of DEA and SFA were found to

yield divergent efficiency estimates due to many

factors such as statistical noise and inputs and

outputs definition, as well as data availability.

Nevertheless, different modeling approaches have

advantages and disadvantages and the choice of the

most appropriate estimation method should depend

on the type of organizations under investigation, the

perspective taken and the quality of the available

data as Hollingsworth (2008) also highlights. The

issue of testing whether an environmental variable

has a significant influence on the production process

AnEmpiricalComparisonofDEAandSFAMethodtoMeasureHospitalUnits'Efficiency

and any resulting efficiency estimates has been also

reviewed. Jacobs (2001) note that the literature

provides several different recommendations on how

to handle such variables. Katharakis & Katostaras

(2012) points out that researchers, besides the

combination of models to measure efficiency,

introduce environmental variables in the analysis,

aiming at better understanding the relationship of

these factors to efficiency and thus at better decision

making.

The most commonly used SFA method is the

Cobb-Douglas functional form and Quadratic

formula or Translog formula, using one input or one

output most of the time, along with existing

environmental factors which are analyzed

separately. DEA has the advantage that it is able to

manage complex production environments with

multiple inputs and outputs, but as a non-statistical

method it does not produce the usual diagnostic

tools with which to judge the goodness-of-fit of the

model specifications. While SFA can discriminate

between efficient units, DEA has a limited ability to

do this, although both techniques can discriminate

between inefficient hospitals (Jacobs, 2001).

Considering the above and the purpose of the study,

the choice of multi-inputs and multi-outputs was

adopted for the SFA Translog formula, indicating

the innovation suggested by this paper.

3 MATERIAL AND METHODS

3.1 Sample Synthesis and Variables

Definition

Following Katharaki (2008) research work, both

DEA and SFA model was applied on the sample of

32 Obstetrical and Gynecological (O&G) units

located in five of the ten geographical Greek NHS

regions. Below, hospital units are referred to by

number (N1–N32), for confidentiality reasons. The

evaluation is focused on DEA and SFA methods that

allow initial comparisons to be made and some early

conclusions to be reached. Based mainly on the data

fit to the model, the following are used in this study

as inputs:

 number of beds (KL);

 number of medical personnel (PR);

 total expenditure for the provision of care (SD).

Regarding the selected inputs, hospital size and

capacity were measured by the number of beds.

Most studies exclude the number of physicians

because independent contractors exist who may

admit patients. For the purpose of the current study,

it is important to include them as an input since wide

discrepancies exist between the numbers of

specialized physicians in different regions of the

country which largely determine the volume of the

O-G services that a hospital can perform (Katharaki,

2008). The input “total expenditure”, refers to the

grand total expenditure and not to the individual

resource component costs (doctors’ salaries, nurses’

salaries, etc.). The introduction of “operating

expenses” into the set of inputs aims at an estimate

of the degree of utilization of the financial resources

in relation to the "services" produced.

It should be mentioned that quantitative

expression of factors determining services provided

by healthcare units display significant difficulties

referring to both the factors’ identification and their

functional relationship to the health product.

According to this the "health product" of healthcare

units is expressed through:

 the number of female patients treated (NOS);

 the number of outpatient examinations (EIA);

 the number of lab tests (ERG).

The use of the number of O&G lab tests and

patient days as outputs of the study was selected in

order to become criteria for efficiency assessment of

units as proxy factors of the degree of resource

utilization. These criteria have been utilized in a

plethora of related studies (Chilingerian and

Sherman, 2004).

In this analysis, the environmental variables in

order to analyze the efficiency are the same for both

the stochastic frontier model and the second stage

truncated regression of DEA model. For the

specification of environmental variables, we

followed Rosko and Mutter (2011) along with

Nedelea and Fannin (2012). The primary variable of

interest is Geographical Position (GP) dummy (one

if the hospital is an urban and zero if it a rural one)

which is used to test whether rural O&Gs are more

or less cost efficient than non-rural hospitals. Two

more dummy variables that define the reputation of

the hospital, indicated by:

 Hospital Status (HS) (one with high status,

referring to tertiary and university hospitals and

zero with lower status, referring to secondary and

non-university hospitals) and

 Patient Mobility (PM) to seek healthcare services

in well-known hospitals (one referring to

hospitals that accept patients from other

geographical regions and zero otherwise)

are included to control the internal pressure

associated with efficiency estimation. The HS

variable was introduced to our analysis based on the

fact that it explains the organizations’ structure.

ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems

Following Assaf and Matawie (2008), and Chen,

Hwang and Shao (2005) the status of the hospital

depends on the position of the hospital (university

hospital or not), the experience of the doctors and

the technological infrastructure of the hospital.

Moreover, the PM is a variable which gives the

patients’ mobility to well-known tertiary hospitals

for their better treatment. PM variable has also been

used by Ippoliti and Falavigna (2012) who argue

that patient mobility may be due to a previous

personal experience or to remarks by someone closer

to the patient and that the perception mechanism is

linked to reputation. In this study the classification

of each hospital of the sample regarding PM was

based on Katharaki (2008) research work who

describes the mobility of patients through regions in

order to seek healthcare services.

3.2 Data Analysis

The empirical research was conducted using two

scenarios: the application of DEA and SFA model

either with environmental variables or without them.

The programming language R environment version

2.15 was used for the data analysis, along with the

software package FEAR 1.15 of Wilson (2010) and

the software package FRONTIER 4.1 of Coelli

(2007).

The SFA Translog and the DEA CRS approaches

were used to conduct the empirical analysis of the

first scenario which was not investigated the

environmental effect on inefficiency. With regard to

the method used, DEA, a method originating from

Farrell’s 1957) seminal work, popularized by

Charnes, Cooper and Rhodes (1978) and Banker,

Charnes and Cooper (1984), provides a non-

parametric alternative to parametric frontier

production function analysis. This non-parametric

method allows the calculation of technical efficiency

measured that can be either input or output oriented

(Charnes, Cooper and Rhodes, 1978; Charnes and

Cooper, 1985; Cooper, et al., 2004; Katharaki,

2008). To estimate the efficiency of the Greek public

units used in the sample, the CCR (Charnes, Cooper

and Rhodes) input oriented model was used (1978).

With the help of input and output variables, the cost-

minimizing input vector for each hospital is

calculated using linear programming (Nedelea and

Fannin, 2012). Next, cost efficiency is measured as

the ratio of minimum cost to observed cost and takes

a value between 0 and 1, where a value of 1

indicates a cost efficient hospital (Coelli

et al., 2005).

Regarding SFA, this parametric method was

based on the quantitative economy theory.

According to Farrell (1957) theory of efficiency

measurement, Aigner et al., (1997) and Meeusen and

van den Broeck (1977) independently constructed an

error structure of stochastic frontier analysis to

measure productive efficiency of firm.

SFA is a

parametric approach, and is suited to measure

efficiencies of stochastic industry for input/output

information. To complete the model estimation, it is

important to specify and use the suitable functional

form. Translog and Cobb-Douglas cost functions are

the most well known formulas for research,

especially in evaluating the efficiency of units.

Translog function is very commonly used. It is a

generalization of the Cobb-Douglas function and it’s

a flexible functional form providing a second order

approximation. Cobb-Douglas and Translog

functions are linear in parameters and can be

estimated using least squares methods. For the

purpose of our empirical research the Translog

function has been used since we had multi-inputs

and multi-outputs of the O&G units. In this paper,

we use the Translog form (formula embedded in the

package frontier 4.1 for multi-inputs and outputs)

with three inputs and three outputs provided in the

following equation (equation 1), respectively:











































































































































∗

























∗











































(1)

where





= the variables of the outputs (NOS= Patients

hospitalized, EIA=Patients examined in outpatient

clinics, ERG=Lab tests) for the i

healthcare unit at

time t





= Beds for the i

healthcare unit at time t





= Total expenditures (€) for the i

healthcare

unit at time t





= Medical personnel for the i

healthcare

unit at time t



= Random error





= Non-negative random variable (or technical

inefficiency)

Data analysis of the second scenario was based

on the two-stage bootstrap DEA method and the

Translog formula of the SFA including the

explanatory variables that have been defined (see

section 3.1). The differences between the SFA and

the DEA approaches are that the SFA requires

AnEmpiricalComparisonofDEAandSFAMethodtoMeasureHospitalUnits'Efficiency

functional forms on the production frontier, and

assumes that firms may deviate from the production

frontier not only due to technical inefficiency but

also from measurement errors, statistical noise or

other non-systematic influences (Admassie,

Matambalya, 2002). For this purpose, in the formula

of the SFA Translog frontier, the second non-

negative random variables 



which are assumed to

be independently and identically distributed normal

random variables as truncations at zero with 



means and variances 





(



~0, 





) are

known as the technical inefficiency effects and in

our model was formed under the environmental

factors, which were earlier defined. Thus equation 2

represents the inefficiency effects model and is the

following for the second scenario:





= σ

+ σ





+ σ





+ σ





+ 



(2)

where





= dummy variable of geographical position

(0, 1) for the i

healthcare unit at time t





= dummy variable of hospital status (0, 1)

for the i

healthcare unit at time t





= dummy variable of moving patient (0, 1)

for the i

healthcare unit at time t





=Random error (



~0, 





)

This research focus on how the environmental

variables influence hospital cost efficiency. From the

first stage of the DEA approach the efficiency scores

are estimated, then regressed in the second stage by

the three environmental variables in order to

investigate if the hospital inefficiency is changed by

these explanatory variables.The second stage of the

two-stage DEA model is conducted by regressing

environmental variables on the healthcare units’

CRS technical inefficiency scores which are

predicted from the first step of the two-stage DEA

model. The units’ technical inefficiency scores are

used as the dependent variable. The set of

environmental variables are used as independent

variables for the two-stage DEA model. The

estimated inefficiency scores are normally bounded

between zero and one. Applying the method of

truncated regression with such a dependent variable

that its values are bounded between zero and one

will lead to biased and inconsistent estimators, since

the truncated method is likely to predict inefficiency

scores which are greater than one (Coelli et al.,

2005). A disadvantage of DEA is that it has no

statistical properties. Simar and Wilson (2007) have

recently addressed this problem and showed that it is

possible to obtain statistical properties for DEA via

the use of the ‘‘bootstrap’’ approach. The bootstrap

approach can also be extended to account for the

impact of environmental variables on efficiency.

These variables are viewed as possibly affecting the

production process but not under the control of

managers. Determining how these variables

influence efficiency is thus essential for deriving

performance improvement strategies. The procedure

used in this study follows that of Simar and Wilson

(2007). A comprehensive discussion of the bootstrap

procedure and its advantages are also provided in

Simar and Wilson (2007).

Both of these approaches have been popular the

last years among researchers in order to explain

valid inferences about the impact of environmental

variables on hospital cost efficiency. What is clear

from the existing literature is that none of the

existing papers (to our knowledge) have adopted the

bootstrapped DEA procedure comparing it with an

SFA model for multi-inputs and multi-outputs.

4 RESULTS

Table 1 shows summary of efficiency scores (per

unit and per geographical area) estimated by both

DEA CRS, DEA bootstrap and SFA Translog model

under the two scenarios of the analysis, with and

without determinants. Efficiency intervals in two

methods DEA and SFA are respectively between

zero and one. Moreover with regard to the first

scenario, the DEA CRS mean efficiency score was

81.56% while the mean efficiency estimated using

SFA

Eq1

was 85.07%. Note the increased score of

approximately 120% for the SFA

Eq1

estimation of

units N9, N16, and N29, something that can be

explained by the structure of the hospital

organization since they have a small amount of

O&G beds and of which the two are regional

hospitals.

From table 1 it is derived that the efficiency

scores obtained from DEA CRS and SFA

Eq1

without

determinants differ, which is consistent with

Chirikos and Sear (2000). The Spearman's Rank

Correlation between the mean of efficiencies

calculated in different methods was then estimated.

The results are listed in Table 2; as it is seen there is

no significant correlation in the different methods.

Subsequently, results of the Maximum

Likelihood Estimation of equation 1 of the SFA

Translog model are provided in Table 3. All

variables of the stochastic frontier regression proved

significant.

ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems

Table 1: The efficiency score of the units of the sample using DEA and SFA model under the two scenarios.

Hospital

units

DEA CRS

efficiency score

SFA

efficiency

score (Eq

)

% change

DEA bootstrap

efficiency score

(bias corrected)

SFA efficiency

score (Eq

1+2

)

change

N1 0.9512 0.76038 -20.06% 0.7716 0.8917 15.57%

N2 1.0000 0.96840 -3.16% 0.8096 0.9762 20.58%

N3 1.0000 0.93015 -6.99% 0.8108 0.9621 18.66%

N4 0.7196 0.88657 23.20% 0.6199 0.9452 52.48%

N5 0.7794 0.72841 -6.54% 0.6922 0.8027 15.96%

N6 0.5038 0.52935 5.07% 0.4327 0.9456 118.53%

N7 0.5916 0.75046 26.85% 0.5093 0.6842 34.34%

N8 1.0000 0.90595 -9.41% 0.8145 0.9596 17.81%

N9 0.4162 0.91890 120.78% 0.3701 0.8966 142.26%

N10 0.8463 0.66225 -21.75% 0.7516 0.7293 -2.97%

N11 1.0000 0.98596 -1.40% 0.8482 0.9810 15.66%

N12 0.8891 0.82467 -7.25% 0.8034 0.8810 9.66%

N13 1.0000 0.71870 -28.13% 0.8078 0.8244 2.05%

N14 0.4777 0.87582 83.34% 0.4167 0.9336 124.05%

N15 1.0000 0.79965 -20.04% 0.8691 0.8669 -0.25%

N16 0.4337 0.96563 122.65% 0.3552 0.9322 162.44%

N17 0.6852 0.86720 26.56% 0.5947 0.9693 62.99%

N18 0.6603 0.84691 28.26% 0.5840 0.8745 49.74%

N19 0.7634 0.83798 9.77% 0.6597 0.8867 34.41%

N20 0.8084 0.67411 -16.61% 0.7135 0.7988 11.96%

N21 1.0000 0.83900 -16.10% 0.8458 0.9071 7.25%

N22 0.6024 0.73069 21.30% 0.4965 0.8009 61.31%

N23 1.0000 0.97582 -2.42% 0.8572 0.9843 14.83%

N24 1.0000 0.95069 -4.93% 0.8223 0.8961 8.97%

N25 0.9818 0.87170 -11.21% 0.8705 0.9164 5.27%

N26 1.0000 0.96252 -3.75% 0.8669 0.9730 12.24%

N27 0.9096 0.97093 6.74% 0.8232 0.9854 19.70%

N28 1.0000 0.92440 -7.56% 0.8318 0.8966 7.79%

N29 0.4348 0.89203 105.16% 0.3572 0.9201 157.59%

N30 0.8590 0.87164 1.47% 0.7764 0.8572 10.41%

N31 1.0000 0.87509 -12.49% 0.8369 0.8857 5.83%

N32 0.6343 0.97526 53.75% 0.5392 0.9669 79.32%

Mean 0.8156 0.8507 4.30% 0.6945 0.8924 28.49%

Table 2: DEA CRS vs. SFAEq1 Spearman's Rank

Correlations rho.

coefficient = 0.241759

p-value = 0.1825 >0.05

Note: coefficient was insignificant at 0.05% level.

Table 3: SFA Results without determinants.

Coefficient Std. Error t value

Intercept -4.5136e+03 9.8655e-01 -4575.185***

LogKL -3.5771e+02 9.5342e-01 -375.191 ***

LogSD 3.4379e+03 9.0588e-01 3795.058 ***

LogPR -1.6328e+02 9.4125e-01 -173.477 ***

4.5061e-02 3.7098e-03 12.146 ***

γ 9.9025e-01 4.5601e-02 21.715 ***

Log Likelihood function 23.2629

Note: ***denotes significance at 1% level, **significance at 5%

level, *significance at 10% level

Provided that DEA models incorporate only

discretionary inputs and the fact that environmental

factors that may influence efficiency are not taken

into consideration in the analysis, scenario 2 was

introduced and under the null hypothesis of positive

effect of the environmental variables GP, HS and

PM on inefficiency, SFA Translog under equation 1

and 2 was performed. In other words, GP, HS and

PM dummies were included into the SFA Translog

model as shifted variables or else variables that

explain the inefficiency level. Table 1 summarizes

the estimated efficiency scores that are likely to

substantially increase, while Table 4 outlines the

significance of the introduced variables of the MSE

estimation of SFA

Eq(1+2)

. In particular, the two

environmental variables GP and HS found to explain

inefficiency as significant. This is also derived from

AnEmpiricalComparisonofDEAandSFAMethodtoMeasureHospitalUnits'Efficiency

Ippoliti and Falavigna (2012) and Chen, Hwang and

Shao (2005).

Moreover, the coefficients of the KL and PR

variables found to be negatively correlating with

inefficiency before and after the explanatory

variables introduction (Table 3 & 4). According to

Chen, Hwang and Shao (2005) hospitals with a large

bed size, experience a lower inefficiency score. On

the other hand, the variable SD was found to be

significantly correlating with inefficiency in both

scenarios. This finding is in line with Katharaki

(2008) results, indicating the need for more rational

utilization of economic resources.

Table 4: SFA Results with determinants.

Coefficient Std. Error t value

Intercept -4.7798e+03 9.8891e-01 -4833.3560***

LogKL -3.7874e+02 9.6128e-01 -393.9903***

LogSD 3.4567e+03 9.2055e-01 3755.0573***

LogPR -1.5535e+02 9.4804e-01 -163.8625***

GP -7.9177e-01 1.8268e-01 -4.3341***

HS 6.9518e-01 1.4319e-01 4.8551***

MP 2.2101e-01 1.1332e-01 1.9504

1.9973e-02 1.0085e-02 1.9805*

γ 9.6286e-01 3.4351e-01 2.8030**

Log Likelihood function 32.87449

Note: ***denotes significance at 1% level, **significance at 5%

level, *significance at 10% level

Following Simar and Wilson (2007), a DEA

bootstrap was conducted. The results are also

presented in Table 1. The Spearman's Rank

Correlation between the mean of efficiencies scores

of DEA bootstrap and SFA

Eq1+2

were also calculated.

The results are listed in Table 5; as it is seen, there is

still no significant correlation in the different

methods.

Table 5: Boootsrap DEA CRS vs. SFAEq

1+2

Spearman's

Rank Correlations rho.

coefficient = 0.21004

p-value = 0.2475>0.05

Note: coefficient was insignificant at 0.05% level.

Considering that the most common approach in

testing the impact of environmental variables on

efficiency involves the use of two-stage analysis,

where according to McDonald (2009) ‘‘Stage 1 is

used to use nonparametric DEA to calculate the

efficiency with which output is produced from

physical inputs. Stage 2, on the other hand, uses

regression to relate efficiency scores to factors seen

to influence’’ (p. 792), and that Simar and Wilson

(2007) have recently criticized this approach, and

suggested instead of a bootstrap approach in which it

is possible to improve the accuracy of the regression

estimates, we regressed the derived bias corrected

bootstrap efficiency scores on the environmental

variables GP, HS and PM (following the

methodology presented in section 3.2). Note that

2000 bootstrap replications (B=2000) was used,

following Simar and Wilson (1999) who highlighted

the adequate coverage of the confidence intervals by

choosing the appropriate number of replications.

At the last step of our analysis, the effect of

determinants on inefficiency was estimated through

the model of the truncated regression. Results of the

Maximum Likelihood Estimation for the parameters

on DEA CRS initial scores and on the bias corrected

bootstrap scores are provided in Table 6. Comparing

the results with those from the SFA method (Table 3

and 4) all variables proved significant and likely

similar. The estimated coefficients and standard

errors for the models are also presented in Table 6.

Table 6: Estimated effects of environmental variables in

both approaches.

DEA results without the effect of determinants

Coefficient Std. Error t value

Intercept 0.6604 0.05347 12.352 ***

KL -0.0001643 0.002434 -0.007

SD 2.467e-07 8.526e-08 2.893**

PR -0.01773 0.006159 -2.880**

Log Likelihood function 13.694348

Two- stage DEA results with determinants after

bootstrap

Coefficient Std. Error t value

Intercept 0.5664788 .0458635 12.35***

KL -0.000144 .0018592 -0.08

SD 2.23e-07 6.72e-08 3.31**

PR -0.0170342 .0052943 -3.22**

GP -0.3272004 .1759237 -3.86***

HS 0.2450065 .1442114 3.70***

MP 0.1418569 .1156827 1.23

Log Likelihood function 18.57163

Note: ***denotes significance at 1% level, **significance at 5%

level, *significance at 10% level

Furthermore, Table 6 outlines the positive and

highly significant coefficient of GP, and negative

and highly significant coefficient of HS dummy,

suggesting that the geographical position of a

healthcare unit, as well as the hospital status of the

unit influence their performance.

5 DISCUSSION

This paper has proposed a framework to measure the

ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems

100

efficiency of hospital units, aiming to examine the

adequacy of two different methods that are

commonly used in literature. Both DEA and SFA

approaches are efficiency frontier analysis, and

provide a suitable way of approaching the

measurement of hospital efficiency. Hospitals are

aimed to minimize inputs and operating efficiently.

Under this context, this paper applies the two

methods to evaluate the efficiency of 32 hospital

units. In particular, a two-stage bootstrap DEA

method and the Translog formula of the SFA were

performed. Multi-inputs and multi-outputs were

used in both of the approaches assuming two

scenarios either including environmental variables or

not.

The main output of the analysis was that the

efficiency scores increased with the incorporation of

environmental variables in the SFA model and

decreased when bootstrap is applied. Specifically,

the analysis shows that the average efficiency scores

of SFA

Eq1

model is the highest (0.85), followed by

DEA

CCR

model (0.81), while the SFA

Eq(1+2)

model

increased (0.89) when environmental variables were

taken into consideration. This result is in line with

Prochazkova (2011), and Nedelea and Fannin

(2012). In addition, when applying the bootstrap

approach and regressing the bias corrected

estimations on the same environmental variables, the

average score decreases to 0.69. Considering the

bootstrapped results, none of the healthcare units

appear to be close to full efficiency and even the

rankings are not preserved. This confirms previous

results from Simar and Wilson (1998; 1999) who

argued that traditional DEA models tend sometimes

to present firms as efficient, when they are actually

not. Consistent to Cordero, Pedraja and Santin

(2009) who outlines that one stage approach

overestimates efficiency especially in the small

sample due to the loss of discrimination power in

DEA after including additional variables (non-

discretionary inputs), the above finding could be

further justified from the small sample of our

analysis.

Moreover, the significant correlation of

environmental variables GP and HS with the

inefficiency are in line with the findings of Ippoliti

and Falavigna (2012) and Chen et al., (2005),

indicating that future research would include a more

detailed study of organizational factors (Minvielle et

al., 2005; Minvielle et al., 2008). In addition, the

present study provides valuable information

regarding deployment of medical staff and beds and

the utilization of financial resources. SFA results

indicate the need for measures taken regarding the

more rational utilization of economic resources.

With regard to the methodology used, a large

number of efficiency analysis studies use SFA with

cross-sectional data. However, the cross-sectional

stochastic frontier model has been shown to have

some limitations. First, in cross-sectional stochastic

frontier models, firm-specific efficiency is

unidentified and researchers typically estimate

expectations of efficiency conditional on a

composite residual. Second, cross-sectional

stochastic frontier models require specific

distributional assumptions for each error component

in order to estimate efficiency.

Alternatively, one can use the two-stage

approach along the line of Simar and Wilson (2007)

with cross-sectional data. From the results, it is

clearly that the DEA and SFA approach have many

advantages and disadvantages as well. Both

techniques constitute two alternatives solutions for

analyzing the effects of the environmental variables

on hospital efficiency. It is shown that similar and

consistent results have been obtained in our

empirical application from the two methods

considering the efficiency analysis of O&G units.

6 CONCLUSIONS

Different methods have been utilized for adjusting

efficiency scores to control the environmental

factors (Cooper, Seiford and Zhu, 2004). The

purpose of our work was to reach a wide variety of

stakeholders, each of which faces different pressures

and values in the selection and application of

efficiency measures. Moreover, this paper is

intended to create a common understanding among

these stakeholders about the adequacy of tools to

measure healthcare efficiency. Given the limitations

of frontier techniques, it may be that they are best

employed in tandem, when possible, and if different

methods suggest similar directions for results then

the validity of such findings is enhanced. Since the

healthcare industry is one area where efficiency

measurement may have a direct policy impact, a

cautious approach is necessary. The use of models

with restrictions placed upon the weight given to

variables, in order to reflect underlying production

models or policy values, is also an interesting area

requiring further research to justify the use of such

restrictions. The quality of data available for use

may also be a problem to be addressed.

Notwithstanding the caveats mentioned earlier

regarding making comparisons across studies, and

AnEmpiricalComparisonofDEAandSFAMethodtoMeasureHospitalUnits'Efficiency

101

that perhaps work needs to be undertaken to think of

ways of making efficiency studies comparable, these

findings may have important policy implications for

the organizational structure of healthcare delivery.

Besides that, the paper has a number of other

limitations. The panel has been restricted to one year

of observations in an unbalanced form with a small

sample of the healthcare units. According to Coelli

et al. (2005) SFA models should be applied in much

bigger samples. Furthermore, focusing solely on the

improvement of the overall inefficiency, a policy

maker or a manager may opt to alter a specific

decision variable. It is thus necessary to carefully

address issues regarding improving the managerial

decision-making process through quantitative

analysis.

To sum up, careful attention should be paid to

the purpose of the analysis and to how results are to

be used. In particular, if they are to be used to

influence economic behavior - for example in the

form of setting targets, or identifying candidates for

inspection - then the potential costs of making

incorrect inferences should be recognized. The

results of this analysis should not serve as a

background for immediate policy responses. It rather

points out special circumstances and provides

motivation for further research. At the same time, it

is fully acknowledged that economic analysis of

Greek hospitals is not telling the whole story. It

should be supplemented by surveys of satisfaction

with the quality of care or surveys of patient criteria

for choosing the hospital unit, and thus include

quality of care, other managerial factors and even

clinical research and political change, as exogenous

variable factors, in order for the analysis to provide

an overall picture.

ACKNOWLEDGEMENTS

The current paper constitutes part of the thesis titled

‘‘Evaluating Efficiency of Health Care Units with

Stochastic Frontier Analysis” that has been co-

financed by the European Union (European Social

Fund – ESF) and Greek national funds through the

Operational Program "Education and Lifelong

Learning" of the National Strategic Reference

Framework (NSRF) - Research Funding Program:

“Heracleitus II. Investing in knowledge society

through the European Social Fund”.

REFERENCES

Aigner, D., Lovell, C., Schmidt, P., Formulation and

estimation of stochastic frontier production functions.

Journal of Econometrics, 6, pp. 21–37.

Admassie, A., Matambalya, F., 2002. Technical Efficiency

in Small and Medium Scale Enterprises: Evidences

from a Survey of Enterprises in Tanzania - Easter

Africa. Eastern Africa Social Science Research

Review, 17(2), pp. 1-29.

Assaf, A., Matawie, K., 2008. Cost efficiency modeling in

healthcare food service operations. International

Journal of Hospitality Management, 27(4), pp. 604–

613.

Banker, R., Charnes, A., Cooper, W.W., 1984. Some

Models for Estimating Technical and Scale

Inefficiencies in Data Envelopment Analysis.

Management Science, 30, pp. 1078-1092.

Bryce, C., Engberg, J., Wholey, D., 2000. Comparing the

Agreement among Alternative Models in Evaluating

Hmo Efficiency. Health Service Journal, 35(2), pp.

509-528.

Charnes, A., Cooper, W.W., 1985. Preface to topics in

Data Envelopment Analysis. Annals of Operations

Research, 2, pp. 59-94.

Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring

the efficiency of decision-making units. European

Journal of Operational Research, 2, pp. 429–44.

Chen, A., Hwang, Y., Shao, B., 2005. Measurement and

sources of overall and input inefficiencies: Evidences

and implications in hospital services. European

Journal of Operational Research, 161, pp. 447-468.

Chilingerian, J. A., Sherman, H. D., 2004. Health care

applications. From Hospitals to Physicians, from

productive efficiency to quality frontiers. In: Cooper

WW, Seiford LM, Zhu J, editors. Handbook on data

envelopment analysis. Boston/London: Kluwern

Academic Publisher.

Chirikos, T., Sear, A., 2000. Measuring hospital

efficiency: a comparison of two approaches. Health

Services Research, 34(6), pp. 1389-1408.

Coelli, T., 2007. A guide to FRONTIER version 4.1: A

computer program for stochastic frontier production

and cost function estimation. CEPA Working Paper

96/07, Department of Econometrics, University of

New England, Armidale, Australia.

Coelli, T., Rao, D., O’Donnell, C., Battese, G., 2005. An

introduction to efficiency and productivity analysis,

New York: Springer 2nd edition.

Cooper, W. W., Seiford, M., Zhu, J., 2004. Handbook on

Data Envelopment Analysis, Boston: Kluwer

Academic Publisher.

Cordero, J., Pedraja, F., Santin, D., 2009. Alternative

approaches to include exogenous variables in DEA

measures: A comparison using Monte Carlo.

Computers and Operations Research, 36, pp. 2699-

2706.

Desaia, A., Ratick, S., Schinnar, A., 2005. Data

envelopment analysis with stochastic variations in

ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems

102

data. Socio-Economic Planning Science, 39, pp. 147–

164.

Farrell, M., 1957. The measurement of productive

efficiency. Journal of the Royal Statistical Society,

120(3), pp. 253-281.

Giuffrida, A., Gravelle, H., 2001. Measuring performance

in primary care: Econometric analysis and DEA.

Applied Economics, 33(2), pp. 163-175.

Gong, B., Sickles, R., 1992. Finite sample evidence on the

performance of stochastic frontiers and data

envelopment analysis using panel data. Journal of

Econometrics, 51, pp. 259-284.

Hollingsworth, B., 2008. The measurement of efficiency

and productivity of health care delivery. Health

Economics, 17(10), pp. 1107–1128.

Ippoliti, R., Falavigna, G., 2012. Efficiency of the medical

care industry: Evidence from the Italian regional

system. European Journal of Operational Research,

217, pp. 643-652.

Jacobs, R., 2001. Alternative methods to examine hospital

efficiency: Data Envelopment Analysis and Stochastic

Frontier Analysis. Health Care Management Science,

4, pp. 103-115.

Katharaki, M., 2008. Approaching the management of

hospital units with an operation research technique:

The Case of thirty two Greek Obstetric and

Gynaecology Public Units. Health Policy, 85(1),

pp.19-31.

Katharakis, G., Katostaras, T., 2012. SFA vs. DEA for

measuring healthcare efficiency: A systematic review.

Health Policy, (under review).

Lee, R., Bott, M., Gajewski, B., Taunton, R., 2009.

Modelling Efficiency at the Process Level: An

Examination of the Care Planning Process in Nursing

Homes. Health Services Research, 44(1), pp. 15–32.

Linna, M., 1998. Measuring hospital cost efficiency with

panel data models. Health Economics, 7, pp. 415-427.

McDonald, J., 2009. Using least squares and tobit in

second stage DEA analyses. European Journal of

Operational Research, 197, pp. 792–8.

Meeusen, W., Van den Broeck, J., 1977. Efficiency

estimation from Cobb-Douglas production function

with composed error. International Economic Review,

18, pp. 435-444.

Minvielle, E., Dervaux, B., Retbi, A., Aegerter, Ph.,

Boumendil, A., Guincestre, M., Tenaillon, A., Guidet,

B., 2005. Culture, Organizations, and Management in

Intensive Care: Construction and Validation of

Multidimensional Questionnaire. Journal of Critical

Care, 20(2), pp. 126-38.

Minvielle, E., Phillipe, A., Dervaux, B., 2008. Assessing

Organizational Performance in Intensive Care Units: A

French Experience. Journal of Critical Care, 23, pp.

236-44.

Mortimer, D., 2002. A Systematic Review of Direct DEA

vs SFA/DFA Comparisons, Working Paper 136,

Centre for Health and Evaluation, Australia.

Mutter, L., Rosko, D., Greene, H., Wilson, W., 2011.

Translating frontiers into practice: taking the next

steps toward improving hospital efficiency. Medical

Care Research and Review, 68(1), pp. 35-195.

Nedelea, C., Fannin, J., 2012. Efficiency Analysis of Rural

Hospitals: Parametric and Semi-parametric

Approaches. In Southern Agricultural Economics

Association Annual Meeting, 4-7 February 2005,

Birmingham, Alabama.

Newhouse, J., 1994. Frontier estimation: how useful a tool

for health economics? Journal of Health Economics,

13, pp. 317-322.

Ondrich, J., Ruggiero, J., 2001. Efficiency measurement in

the stochastic frontier model. European Journal of

Operational Research, 129, pp. 434-442.

Prochazkova, J., 2011. Efficiency of Hospitals in the Czech

Republic: DEA & SFA Applications. Ph.D. Faculty of

Social Sciences, Institute of Economic Studies,

Charles University in Prague.

Rosko, D., Mutter, L., 2011. What have we learned from

the application of stochastic frontier analysis to U.S.

hospitals? Medical Care Research and Review, 68(1),

pp. 75-100.

Simar, L., Wilson, P., 1998. Sensitivity analysis of

efficiency scores: How to bootstrap in nonparametric

frontier models. Management Science. 44, pp. 49–61.

Simar, L., Wilson, P., 1999. Estimating and bootstrapping

Malmquist indices. European Journal of Operational

Research, 115, pp. 459–471.

Simar, L., Wilson, P., 2000a. A General methodology for

bootstrapping in non-parametric frontier models.

Journal of Applied Statistics, 27, pp. 779–802.

Simar, L., Wilson, P., 2000b. Statistical inference in

nonparametric frontier models: The state of the art.

Journal of Productivity Analysis, 13, pp. 49–78.

Simar, L., Wilson, P., 2007. Estimation and inference in

two-stage, semi-parametric models of production

processes. Journal of Econometrics, 136, pp. 31-64.

Wilson, P., 2010. FEAR 1.15 User’s Guide. Available at:

<http://www.clemson.edu/economics/faculty/wilson/S

oftware/FEAR/FEAR-1.15/fear-user-guide.pdf>

[Accessed 25 July 2012].

AnEmpiricalComparisonofDEAandSFAMethodtoMeasureHospitalUnits'Efficiency

103