A MODEL USING DATA ENVELOPMENT ANALYSIS FOR THE

CROSS EVALUATION OF SUPPLIERS UNDER UNCERTAINTY

Nicola Costantino

, Mariagrazia Dotoli

, Marco Falagario

and Fabio Sciancalepore

Dipartimento di Ingegneria Meccanica e Gestionale, Politecnico di Bari, Via Re David 200, 70126, Bari, Italy

Dipartimento di Elettrotecnica ed Elettronica, Politecnico di Bari, Via Re David 200, 70125, Bari, Italy

Keywords: Business Intelligence, Supplier Evaluation, Data Envelopment Analysis, Uncertainty, Monte Carlo Method.

Abstract: The paper addresses one of the key objectives of the purchasing function of a supply chain, i.e., the optimal

selection of suppliers. We present a novel methodology that integrates the well-known cross-efficiency

evaluation called Data Envelopment Analysis (DEA) and the Monte Carlo approach, to manage supplier

selection considering uncertainty in the supply process, e.g. evaluating potential suppliers. The model

allows to distinguish among several suppliers, overcoming the limitation of the traditional DEA method of

not distinguishing among efficient suppliers. Moreover, the technique is able to classify suppliers with

uncertain performance. The method is applied to the selection of suppliers of a Southern Italy SME.

1 INTRODUCTION

Within the purchasing management area, the process

of supplier selection is currently widely investigated

in the scientific literature, particularly as regards the

private sector, due to its strategic role in the success

of a Supply Chain (SC) (Costantino et al., 2011).

Supplier evaluation techniques periodically identify

and verify the best potential candidates able to

provide the expected performance level to the SC.

Typically, supplier selection is a multi-objective

decision problem with conflicting objectives, such

as, besides the obvious goal of (low) price, also

quality, quantity, delivery, performance, capacity,

communication, service, geographical location, etc.

The wide literature of the area is a synonym of the

importance of supplier choice and the interested

reader may find a detailed discussion of the

appeared contributions in Ho et al. (2010).

Roughly, the multi-objective approaches

proposed in the related literature for the solution of

the supplier selection problem may be classified into

individual model and integrated techniques. One of

the best-known individual methods is the so-called

Data Envelopment Analysis (DEA) technique, due

to its robustness and simplicity of application (Ho et

al., 2010). The DEA technique is based on linear

programming to determine the efficiency of several

units subject to the decision (Charnes et al., 1978).

However, a limitation of the classical DEA is that it

distinguishes only between inefficient and efficient

suppliers, without enabling a discrimination among

the elements of the latter set. Such a characteristic

makes it difficult to apply DEA for supplier

selection, particularly in the case of a single sourcing

purchasing, i.e., with one supplier only. With the

aim of improving the method discriminating power,

Sexton et al. (1986) proposed the so-called cross-

efficiency DEA method that evaluates the decision

units efficiency in a crossed way. However, the

resulting technique is quite complex, since it

requires the set-up and solution of a two-level

optimization problem, and deterministic, so that it

cannot manage uncertain data on suppliers.

Nevertheless, uncertainty is one of the most relevant

issues in SC management and this is particularly

apparent in the supplier selection process. Indeed,

such a problem is often concerned with the

evaluation of potential candidates, with which the

buyer has not had previous commercial

relationships, so that the corresponding key

performance indicators are inevitably vague.

This paper proposes an integrated model for

supplier selection based on the cross-efficiency DEA

and Monte Carlo simulation. The technique main

advantages are two, as follows: on the one hand it

enables the purchasing manager to distinguish

among multiple suppliers that according to the

classical DEA method are considered as equally

efficient, on the other hand it is able to classify

152

Costantino N., Dotoli M., Falagario M. and Sciancalepore F..

A MODEL USING DATA ENVELOPMENT ANALYSIS FOR THE CROSS EVALUATION OF SUPPLIERS UNDER UNCERTAINTY.

DOI: 10.5220/0003638901520157

In Proceedings of the International Conference on Knowledge Management and Information Sharing (KMIS-2011), pages 152-157

ISBN: 978-989-8425-81-2

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

suppliers with uncertain performance against some

defined criteria for evaluation of the supply.

2 THE DEA METHOD FOR THE

EVALUATION OF SUPPLIERS

2.1 The Traditional DEA Technique

The DEA method (Charnes et al., 1978) is a

technique for classifying multiple Decision Making

Units (DMU) in a compared way while measuring

the maximum unit efficiency with respect to the

performance of all the analyzed DMU. In particular,

the units are characterized by the sharing of a set of

resources used to produce goods or services. In the

DEA method the efficient DMU are the vertices of a

Pareto face: based on these, the other DMU

efficiencies are evaluated. As regards the supplier

evaluation and selection area, DEA allows

determining, among a set of partners, a subset

composed by the suppliers using the given (input)

resources to produce the required (output)

goods/services in the most efficient way. To this

aim, several inputs express the contribution required

to the supplier (e.g., the purchasing price, the lead

time, etc.) and several outputs identify its

performance in the procurement process (e.g.,

delivery timeliness, finite product quality).

In general, a supplier selection problem may be

defined by a set of suppliers offering the requested

product

{}

, ,.....,

Sss s= and by a set of

conflicting objectives

{}

, ,.....,

Ccc c= , against

which the suppliers are to be classified. We assume

that the set of criteria is partitioned as

CC C=∪,

with

{}

, ,.....,

Ccc c= and

{}

, ,.....,

OHH HK

Ccc c

++ +

= , representing the set of

input and output criteria respectively, with H+K=n

being the overall number of criteria.

The generic supplier

S∈

has efficiency:

kkf

hhf

⋅

∑

(1)

that is the ratio between the weighted sum of the

outputs and the weighted sum of the inputs, where:

is the k-th type output (k=1, 2,…K) referred to

supplier

S∈ ; x

is the h-th type input (h=1,

2,…,H) referred to supplier

S∈ ; u

is the weight

assigned to the

k-th type output; v

is the weight

associated with the h-th type input.

The aim of the DEA method is associating to the

outputs and inputs of supplier

S∈ , given their

values,

a set of weights leading to maximize

efficiency, while taking into account that this cannot

by definition be higher than 1. In such a way the

efficiency of the

f-th supplier (f=1,2,…,F, F being

the total number of analyzed suppliers) is evaluated

solving the following mathematical programming

problem, determining the sets of weights

and v

that maximize

max

kkf

hhf

⋅

∑

with f=1,2,…,F,

subject to

kkf

hhf

⋅

≤

⋅

∑

with f=1,2,…,F,

uv≥ for k=1,2,…,K; h=1,2,…,H.

(2)

The first constraints of (2) represent an upper

bound (equal to 1) in terms of absolute efficiency for

all suppliers using the optimal weights for the

f-th

vendor, while the second constraints of (2) impose

that all weights are non negative. The programming

problem (2) is non-linear with its unknowns: hence,

determining the solution with numerous potential

suppliers and evaluation criteria is computationally

intensive. However, this problem may be simplified

by linearizing it, using the so-called

output-oriented

procedure

, as follows (Charnes et al., 1978):

max max

kkf

Euy

⋅

∑

with f=1,2,…,F

subject to

kkf hhf

uy vx

⋅

−⋅≤

∑∑

with f=1,2,…,F,

hhf

⋅

∑

uv≥ for k=1,2,…,K; h=1,2,…,H.

(3)

Solving (3) we compute the maximum efficiency of

each vendor: a supplier

S∈ is efficient if it

exhibits a unitary efficiency value

. Suppliers may

accordingly be classified in a descending order of

efficiency, thus leading to a ranking.

A MODEL USING DATA ENVELOPMENT ANALYSIS FOR THE CROSS EVALUATION OF SUPPLIERS UNDER

UNCERTAINTY

153

2.2 The Cross-efficiency DEA Method

The described DEA method can only distinguish

between efficient and inefficient suppliers, typically

providing a set of maximally efficient vendors,

without discriminating among these. Such a

characteristic makes it difficult its application to the

supplier selection problem, particularly in the case

of single sourcing purchasing. The so-called cross-

efficiency DEA method (Sexton et al. 1986)

improves the discriminating power of the traditional

DEA technique, evaluating the efficiency of each

supplier not only with respect to the set of weights

that is optimal supplier itself, but also with respect to

the sets of weights that are optimal for the other

vendors, i.e., that maximize their efficiencies. In

such a way, the

f-th supplier efficiency is evaluated

as the cross-efficiency CE

given by the average of

all the efficiency values that the supplier obtains by

varying the considered weights: hence the evaluation

becomes a cross-evaluation rather than a self-

evaluation. The resulting cross-efficiency matrix

={CE

}

∈ \ is determined by evaluating,

with respect to the i-th supplier, considered each

time as a pivot, the related efficiency

of all the

other suppliers with index f. Hence, the generic

diagonal element of CE, indicated by

CE for

i=1,…,F, is determined as the solution of problem

(3) solved with a pivot

i, i.e., with f=i. Hence we set

ii i

CE E= , where E

is the optimal value of the

objective function of the linear programming

problem (3) for the

i-th supplier. As summarized

earlier, for each supplier

S∈ the values of the

optimal input and output weights of its efficiency

with respect to the

i-th pivot supplier

u and

v are

successively employed to determine

as follows:

kkf

hhf

⋅

∑

with f,i=1,2,…,F.

(4)

However, (4) is not univocally applicable, since

there exist multiple weight combinations that

maximize the

i-th supplier efficiency. With them,

also the efficiencies of the other suppliers

with

respect to s

would vary. Hence, it is necessary to

univocally choose, for each supplier, a set of weights

among the combinations that maximize the

efficiency. To solve the issue, Green and Doyle

(1995) propose a second-level optimization

procedure that is to be executed for each

i-th

supplier after the solution of the described linear

programming problem, as follows:

11,

min

kkf

kffi

==≠

⎛⎞

⎜⎟

⎝⎠

∑∑

subject to:

11,

hhf

hffi

==≠

⎛⎞

⎜⎟

⎝⎠

∑∑

kki i hhi

uy E vx

−

∑∑

kkf hhf

uy vx

⋅

−⋅≤

∑∑

with f=1,2,…,F,

uv≥ per k=1,2,…,K; h=1,2,…,H.

(5)

In (5), it is imposed that each pivot supplier

∈

chooses, among the sets of weights

maximizing its efficiency

, the set that minimizes

the overall efficiency of other vendors. The set of

weights

(

)

uv with k=1,2,...,K and h=1,2,…,H

solving this problem is employed to determine all

elements

with f=1,2,…,i-1,i+1,…,F according

to (4). Therefore, all suppliers

S∈ with f=1,…,F

can be classified according to their overall cross-

efficiency value that can be determined by averaging

the elements of the

f-th row of the cross-efficiency

matrix CE as follows:

CE CE

∑

with f=1,…,F.

(6)

3 ADDRESSING UNCERTAINTY

IN SUPPLIER SELECTION BY

THE DEA TECHNIQUE

The DEA technique, both in its traditional version

and in its cross-efficiency extension, is

deterministic. In other words, the inputs and outputs

of each supplier are assumed as certain and the

model does not consider any uncertainty element.

On the contrary, such aspects usually characterize

the supply process and should, even more

importantly, be taken into account when the

evaluation is referred to potential commercial

partners, with whom no previous relations exist. The

issue of uncertainty on DEA data was already

addressed in the related literature. Dyson and Shale

(2010), in particular, distinguish among four

different approaches: Imprecise DEA,

Bootstrapping, Chance-Constrained DEA, and

Monte Carlo simulation. The Imprecise DEA model

allows employing performance values that are

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

154

imprecise, i.e., expressed either as values in a range

or as ordinal ranked values (that are defined by a

ranking of the alternatives for the single attribute)

rather than cardinal values. Bootstrapping, instead, is

a technique consisting in re-sampling a sample of

real observations of the uncertain variables: for each

new extraction, the values of the corresponding

objective function are computed, so as to obtain a

sample distribution of the variable. The Chance-

Constrained DEA allows employing stochastic

performance values both in input and output: the

probability constraints assure that the probability

that the observed outputs (inputs) are higher (lower)

than the best possible values overcomes a given

threshold.

This work focuses on the use of Monte Carlo

simulation for the application of DEA to the supplier

selection problem based on stochastic data. We

propose a novel method relying on the idea that the

uncertain input and output values may be modelled

by suitable probability distributions, based on real

observations or on a simple estimate of such data.

The chosen distributions (and their opportunely

estimated characteristic parameters) may be

employed for a series of casual extractions useful to

determine the efficiency of each DMU. Such a

procedure was already adopted in the SC supplier

choice with the traditional DEA method by Wong et

al. (2008). The work by Kao and Liu (2009) is also

significant, evaluating the efficiency of Taiwan

banks with the DEA technique using stochastic input

and output methods that are evaluated by a Beta

distribution: they demonstrate that 2000 iterations

are sufficient to obtain convergent results.

This paper integrates the DEA stochastic

methodology proposed in Kao and Liu (2009) with

the cross-efficiency DEA for application to the

supplier evaluation and selection problem. In

particular, let

and Y

be the h-th stochastic input

value (with

h=1,…,H) and the k-th stochastic output

value (with

k=1,…,K), respectively, for supplier

S∈ . These variables are modelled by a specific

Beta probability distribution, called Beta-PERT

(Vose, 2008). Such a distribution suits very well the

cases in which no information is available on the

values assumed in the past by the stochastic

variables and it is thus impossible to define a

frequency-based probability density function (Law

and Kelton, 2000). Indeed, the characteristic

parameters of such a distribution may be determined

using a three-estimates approach that is inspired

from the stochastic PERT technique for project

management, while obtaining a lower standard

deviation (and thus a more faithful reproduction of

the expert estimate) than the well-known triangular

distribution (Vose, 2008). In particular, calling

min

mod

, and

max

respectively the lowest

possible, most probable, and highest possible

estimated values, respectively, that the generic input

variable

may assume, the corresponding Beta (or

Beta-PERT) distribution is defined by the following

characteristic parameters (Vose, 2008):

min mod min max

mod max min

()(2 )

()()

Xhf hf hf hf

hf X hf hf

xxxx

xxx

−−−

−−

max

min

()

hf hf

Xhf X

Xhf

αμ

−

(7)

with

min mod max

hf hf hf

(8)

The resulting Beta-PERT distribution is defined

as follows (Vose, 2008):

min mod max

max min min

(, , )

( , )( )

hf hf hf

BetaPERT x x x

Beta x x x

αα

=−+

(9)

Analogously, considering the stochastic variable

, given

min

mod

, and

max

respectively the

lowest possible, most probable, and highest possible

estimated values of the variable we set:

min mod min max

mod max min

()(2 )

()( )

Ykf kf kf kf

kf Y kf kf

yyyy

yyy

−−−

−−

max

min

()

kf kf

Ykf Y

Ykf

αμ

−

(10)

with

min mod max

kf kf kf

yyy

(11)

so that

min mod max

max min min

(, , )

( , )( )

kf kf kf

BetaPERT y y y

eta y y y

αα

=−+

(12)

As a consequence, all the input and output

stochastic indicators

and y

may be modelled as

stochastic variables that are distributed according to

(9) and (12), respectively. Applying the Monte Carlo

method allows assigning at each iteration a casual

value to all the stochastic variables according to the

cited probability density functions. Such values are

employed to solve at each iteration the cross-

efficiency problem (3) and (5). The final cross-

A MODEL USING DATA ENVELOPMENT ANALYSIS FOR THE CROSS EVALUATION OF SUPPLIERS UNDER

UNCERTAINTY

155

efficiency values associated to the f-th supplier for

f=1,2,…,F are evaluated by averaging the overall

cross-efficiency index

computed for all the

iterations, so that a ranking of the suppliers is

established by the descending order of such values.

Summing up, the proposed method integrates

both the advantages of traditional DEA (evaluating

the supply efficiency and avoiding data

normalization) and cross-efficiency DEA

(optimizing the weight set so as to shift from self-

evaluation to peer-evaluation) with respect to

alternative supplier selection techniques. In addition,

the technique is able to deal with uncertain data in

the supply, typical of the real context.

4 USING THE STOCHASTIC DEA

MODEL FOR THE SUPPLIER

SELECTION OF AN SME

We apply the described model for the cross-

efficiency evaluation of suppliers to the supplier

selection process of a small enterprise of southern

Italy, operating in the areas of sale, set-up, and

maintenance of hydraulic plants. Before the

subsequently reported investigation, the case study

SME used the classical lowest price approach to

choose suppliers. On the contrary, thanks to the

proposed supplier selection tool, it was able to

compare the performance of suppliers that it

employed in the past with that of other potential

partners, based not only on price but also

considering additional uncertain data. The

considered supply refers to a specific component, a

cast iron mainspring check valve. While for the

current suppliers the available data are deterministic,

the uncertainty referring to some data of the

performance of potential partners was modelled

using Monte Carlo simulation and the previously

introduced Beta-PERT distribution. In particular, we

consider two currently existing suppliers (denoted

by E1 and E2) for the product supply. Their

performance is compared to that of six potential

additional suppliers (P1, P2, P3, P4, P5, and P6).

The considered efficiency indicators are five. In

particular, three are input indices, i.e., refer to

resources that are required to the SME in the supply

process: 1) component purchasing price (in

€); 2) lead time for the order execution, i.e., time

elapsing between order emission and product

delivery (in days, d); and 3) geographical distance

between supplier and SME (in Km). In addition, we

consider two output indices: 1) quality of the

provided component, defined as a percentage ratio

between working components and overall number of

delivered components (%); and 2) delivery

reliability, expressed as a percentage ratio between

orders that are dealt with on time and overall number

of orders (%).

Table 1: Input and output data for current and potential

suppliers.

Suppl.

Input criteria Output criteria

Price

[€]

Lead Time

[d]

Distance

[Km]

Quality [%]

Reliability

[%]

110 40 1000 99.3 50

91 50 900 99 37.5

95 (30,35,45) 970 (90,95,99) (30,50,75)

78 (42,45,50) 20 (90,92,95) (30,50,75)

132 (32,40,50) 833 (96,99,100) (40,60,85)

130.67 (32,40,50) 897 (90,95,99) (40,60,75)

114.50 (35,45,50) 813 (90,95,99) (35,55,77)

133.10 (32,40,50) 898 (90,95,99) (50,60,80)

Table 2: Cross-efficiency matrix and average cross-

efficiency index of suppliers.

Suppl.

- Efficiency with respect to

Cross

Eff.

E1 E2 P1 P2 P3 P4 P5 P6

0.95 0.87 0.84 0.02 0.83 0.87 0.89 0.85 0.77

0.82 0.94 0.70 0.02 0.62 0.66 0.74 0.63 0.64

0.99 0.95 1.00 0.02 0.90 0.93 0.96 0.92 0.83

0.97 1.00 0.82 1.00 0.85 0.90 0.99 0.87 0.92

0.94 0.75 0.84 0.03 0.97 0.92 0.92 0.91 0.79

0.90 0.73 0.82 0.02 0.88 0.94 0.89 0.89 0.76

0.87 0.79 0.78 0.02 0.80 0.84 0.91 0.82 0.73

0.92 0.74 0.84 0.02 0.91 0.93 0.92 0.96 0.78

With reference to the data, we remark that

purchasing prices and geographical distances are

deterministic values. Indeed, prices may be

determined using the price list of the suppliers,

neglecting price variations during the year. On the

contrary, the remaining indices, i.e., lead time,

quality, and reliability, are deterministically defined

for the current suppliers E1 and E2, using historical

data of the SME, while they can be only

stochastically defined for potential suppliers using

the previously described the Beta-PERT approach.

Table 1 summarizes the problem data. For each of

the eight considered suppliers, we evaluate its cross-

efficiency value according to the presented method.

Due to the presence of stochastic elements among

the problem data, we solve the problem using the

described Monte Carlo simulation, thanks to 1000

iterations that are executed in the well-known

MATLAB software environment. The results of the

procedure are in Table 2. The last column of the

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

156

table show that the most efficient supplier is

potential supplier P2 (that is favoured by the

geographical proximity to the SME, see Table 1),

followed by potential supplier P1. A key factor to

this result is the optimal set of coefficients chosen

for P2: it is presumable that the higher incidence

falls back on the coefficients related to price and

(especially) to geographical distance, two factors

thanks to which supplier P2 is predominant over the

remaining vendors. As a consequence, the cross-

efficiency values associated to the other suppliers

are two orders of magnitude lower than those

characterizing P2 (ranging between 0.02 and 0.03, as

the fifth column of Table 2 shows). Moreover, the

last column of Table 2 remarks that the four most

efficient suppliers in terms of cross-efficiency are all

in the set of potential suppliers, i.e., they are P2, P1,

P3, and P6, in a descending order of efficiency.

The SME purchasing manager evaluated the

obtained results, underlying as major advantages of

the method its ability to take into account multiple

criteria, its capability to distinguish between the

required resources and the overall performance, and

ultimately its skill in assessing both the supply

process effectiveness and efficiency.

5 CONCLUSIONS

We propose a novel method for the optimal selection

of suppliers based on the well-known Data

Envelopment Analysis (DEA) technique. In

particular, we extend a DEA method for the cross-

evaluation of efficiency, previously presented in the

literature to discern among maximally efficient

suppliers, using the Monte Carlo simulation method,

so as to enable the treatment of uncertain data. The

technique application to the supplier selection

process of an Italian SME, shows its

straightforwardness and its ability to discerning

among suppliers, also in case of uncertain data.

Future developments include further validation of

the method and detailed comparison with alternative

approaches presented in the scientific literature.

ACKNOWLEDGEMENTS

This work was supported by the TRASFORMA

“Reti di Laboratori” network funded by the Italian

Apulia Region.

REFERENCES

Charnes, A., Cooper, W. W., Rhodes, E., 1978. Measuring

the efficiency of decision making units. Eur. J. of

Operational Research, Vol. 2, pp. 429–444.

Costantino, N., Dotoli, M., Falagario, M., Fanti, M.P.,

Mangini, A.M., Sciancalepore, F., 2011. Supplier

selection in the public procurement sector via a data

envelopment analysis approach, 6 pp., Proc. 19th

IEEE Mediterranean Conf. on Control and

Automation (MED 2011), Corfu, Greece, June 23-25.

Dyson, R. G., Shale, E.A., 2010. Data envelopment

analysis, operational research and uncertainty, J Oper

Res Soc, Vol. 61, Issue 1, pp. 25 – 34

Green, R., Doyle, J., 1994. Efficiency and cross-efficiency

in DEA: Derivation, Meanings and Uses. J Oper Res

Soc Vol. 45, pp. 567-578

Ho, W., Xu, X., Dey, P. K., 2010. Multi-criteria decision

making approaches for supplier evaluation and

selection: A literature review. Eur. J. of Operational

Research, Vol. 202, pp. 16-24.

Kao, C., Liu, T.-S., 2009. Stochastic data envelopment

analysis in measuring the efficiency of Taiwan

commercial banks. Eur. J. of Operational Research,

Vol. 196, pp. 312–322.

Law, A. M., Kelton, W. D., 2000. Simulation Modeling

and Analysis. Quarta ed., McGraw-Hill, New York.

Sexton, T. R., Silkman, R.H., Hogan, A.J., 1986. Data

envelopment analysis: Critique and extensions, in R.

H. Silkman (Ed.), Measuring efficiency: An

assessment of data envelopment analysis. San

Francisco, CA: Jossey-Bass.

Vose, D., 2008. Risk analysis: a quantitative guide. John

Wiley and Sons, Second Edition.

Wong, W. P., Jaruphongsa, W., Lee, L. H., 2008. Supply

chain performance measurement system: a Monte

Carlo DEA-based approach. Int. J. of Industrial and

Systems Engineering ,Vol. 3, pp. 162 – 188.

A MODEL USING DATA ENVELOPMENT ANALYSIS FOR THE CROSS EVALUATION OF SUPPLIERS UNDER

UNCERTAINTY

157