Assessment and Choice of Software Solution with the Analytic Network

Process Method

Jerzy Michnik

and Krzysztof Kania

Operations Research Department, University of Economics in Katowice, ul. 1 Maja 50, Katowice, Poland

Knowledge Engineering Department, University of Economics in Katowice, ul. 1 Maja 50, Katowice, Poland

Keywords:

Analytic Network Process (ANP), Software Assessment.

Abstract:

Assessment and choice of software solution belong of the most difﬁcult tasks facing business and IT experts.

Multiple criteria decision methods can help in making that kind of a decision. However most of the methods is

based on the assumption of criteria independence which is rarely fulﬁlled in practice. We applied the Analytic

Network Process (ANP) as an aid for a choice of software solution and compare its results with the Analytic

Hierarchy Process (AHP) method which is more often used in such a task.

1 INTRODUCTION

Under the pressure of rapid development of infor-

mation and communication technologies (ICT) and

the growing importance of information systems in

business, the organizations often face the problem

of matching the available ICT to business needs.

A choice and successful implementation of the ver-

satile ICT system is a serious challenge for an or-

ganization. Such a project lasts a long time, needs

substantial investment and re-organization of most of

the business procedures. A complete ICT system is

a compound structure with many specialized com-

ponents and an exchange of information between its

components.

The aim of our work was to develop the effective

decision aid for an assessment and choice of ICT solu-

tion characterized by a wide range of attributes by ap-

plying the Analytic Network Process (ANP) method

for ranking the decision alternatives. The ANP seems

to be more suitable then the other methods as it has

the ability to handle the complicated decision model

with many criteria and dependencies among them.

The article is structured as follows. Sec. 2 deﬁnes

a research problem, Sec. 3 presents the ANP method.

Sec. 4 contains the model of Business Intelligence

(BI) systems assessment. Sec. 5 concludes the article.

2 THE PROBLEM OF CHOOSING

THE ICT SYSTEM

Selecting a speciﬁc software is a stage of the whole

decision process, where requirements formulated in

the sphere of business and ICT meet together. Man-

agers want the software to give them the greatest pos-

sible business opportunities and focus mainly on the

software functionalities, while the ICT professionals

have to take into account many technological limita-

tions, existing and legacy systems, the possibility of

performing additional tasks (administration, support

and safety) and many others.

Selecting a software for the large-scale systems is

a strategic decision because it determines the operat-

ing environment for a long time and bounds an orga-

nization to a particular vendor. It can be implemented

in different ways (Woitsch et al., 2009). The most

common approaches to that task can be described as

heuristic approach as they are based on knowledge

of experts involved in the process of software assess-

ment. Hence, a quality of the decisions depends pri-

marily on a quality of knowledge and an experience

of experts. Yet, the serious difﬁculty encumber the

heuristic procedure in practice. It is evoked by the

enormous quantity of information that have to be pro-

cessed on the way to ﬁnal decision. ICT is a highly

compound system of several components with many

various sub-elements that are characterized by a large

number of qualitative and quantitative features.

Risk of making the wrong choice is high because

326

Michnik J. and Kania K..

Assessment and Choice of Software Solution with the Analytic Network Process Method.

DOI: 10.5220/0003991603260329

In Proceedings of the 14th International Conference on Enterprise Information Systems (ICEIS-2012), pages 326-329

ISBN: 978-989-8565-10-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

of the variety of offers, the high cost of software

implementation that excludes rapid changes of en-

vironment, very long time needed for software de-

ployments and because of technological and hardware

linkages established during the project. All these risks

can be reduced by a precise projection of business

needs and technological constraints during the proce-

dure of software selecting. Unfortunately, the desired

properties of the system are often dependent on each

other and they create a net rather then a simple hierar-

chy. So, it is necessary to use a method that gives the

possibility of more than just a simple imposition of a

set of weights.

Some number of formal methods that support

software selection has been reported. The authors

of Computer Science Technical Report mentioned,

among others, 4 papers that use the linear weighted at-

tribute method (simple additive weighting) (Fritz and

Carter, 1994). The other similar multiple criteria de-

cision methods, like SMART (Valiris et al., 2005) or

ELECTRE II (Stamelos et al., 2000), also have been

tested. Lai et al. report the results of a case study

where the AHP method was employed to support the

selection of multimedia authoring system (Lai et al.,

2002). Selecting the best software product among the

alternatives for each module in the development of

modular software systems has also been done with the

aid of AHP (Jung and Choi, 1999).

All methods mentioned above base on an assump-

tion that the criteria, considered in the evaluation of

alternatives, are independent. Yet, the ICT system

is compounded from the interfering modules and it

leads to some dependencies between criteria. Use of

a method that can involve dependencies in the ana-

lyzed system may substantially improve the results.

Wu proposed a hybrid model that combines the

Decision Making Trial and Evaluation Laboratory

(DEMATEL) with the ANP and the zero-one goal

programming (ZOGP) to get an effective solution

that considers both ﬁnancial and non-ﬁnancial factors

(Wu, 2008). Recently, ANP has been used to select

most suitable simulation software (Aya

g, Zeki, 2011)

and ERP system (Wieszała et al., 2011).

Almost all of the publications cited above con-

cern the assessment of single, specialized software or

consider (Wu, 2008) the series of the mutually non-

excluded IT projects. Our evaluation deals with more

complex implementation of a whole, multi-modular

ITC system in an enterprise when only one alterna-

tive is to be selected.

3 THE ANALYTIC NETWORK

PROCESS

The Analytic Network Process (ANP) (Saaty, 2005)

is deﬁned as a multiple criteria method that derives

priority scales of absolute numbers from individual

judgments. The numbers come out from the pair-

wise comparisons of elements of the studied system.

One provides the judgment by answering two kinds of

questions: ’Which of the two elements is more dom-

inant with respect to a criterion?’ or ’Which of the

two elements inﬂuences the third element more with

respect to a criterion?’

The ANP procedure can be summarized in the fol-

lowing steps:

1. Set up: a) the control criterion representing the

decision problem, b) the main groups of crite-

ria (named components or clusters) characterizing

the decision problem, c) the criteria that belong

to each cluster, d) the decision alternatives, e) the

relations between elements of the decision model

(criteria and alternatives).

2. Make all pairwise comparisons for relations in the

model using the two kinds of questions mentioned

above.

3. Perform the following operations: a) calculate pri-

ority vectors for supermatrix and cluster matrix,

b) build the unweighted supermatrix, c) weight

the unweighted supermatrix with the cluster ma-

trix, d) calculate the limit supermatrix.

4. Read out the overall priorities for alternatives

from the limit supermatrix. Discuss the results.

If needed, make the suitable modiﬁcations of the

model and repeat the procedure.

All steps besides Step 3, are the tasks that need to

be made by people engaged in the decision process

(decision maker(s) and/or analyst). Step 3 has a com-

putational character and can be automatized with a

suitable software (in this work, like in many others,

the specialized software ”Superdecisions” has been

used). The short description of the operations of Step

3 is presented below.

A priority vector is derived from paired compar-

isons matrix by normalizing its columns and taking

the geometric mean form rows (in the same way as

in the AHP). Let’s assume that we need to compare

p elements of the model with respect to some control

criterion. So, the pairwise comparison matrix C will

be the square matrix of size p× p. Saaty (Saaty, 2005)

suggests to use the following scale to translate the ver-

bal comparisons (easier to obtain from decision mak-

ers) into numbers: equal importance = 1; moderate

AssessmentandChoiceofSoftwareSolutionwiththeAnalyticNetworkProcessMethod

327

importance = 3, strong importance = 5, very strong

importance = 7, extreme importance = 9. The even

numbers 2, 4, 6, 8 are used for an assessment lying

between the above main points of scale.

Each priority vector becomes a column of matrix

i j

= [w

]

×n

, where n

) is a number of elements

in cluster i ( j). Let’s assume that there is a system of

N clusters. Then the supermatrix will be constructed

from N ×N blocks, i.e. W = [W

i j

]

N×N

. W

i j

represents

the inﬂuence of the elements from cluster i on the el-

ements from cluster j. The supermatrix W represents

the inﬂuence priority of the element on the left of the

matrix on the element at the top of the matrix.

In the next step, the supermatrix is transformed

into a weighted supermatrix, i.e. to the matrix, whose

columns sums to unity. Initially the supermatrix

columns are made up of several eigenvectors which,

in normalized form, sum to one and hence that col-

umn sums to the number of nonzero eigenvectors.

The weighted supermatrix can be obtained by weight-

ing the initial supermatrix with the cluster matrix. The

cluster matrix contains eigenvectors representing the

priorities of clusters with respect to the general con-

trol criterion (in most cases it will be a main objec-

tive).

In the end the limit matrix is derived by raising

the weighted matrix to an arbitrarily high power. This

procedure sums up the inﬂuences along paths of dif-

ferent length in the underlying network and deter-

mines the overall priorities.

4 THE ANP MODEL FOR BI

SYSTEM SELECTION

The need for analysis and evaluation of BI en-

vironments results from the fact that none of the

largest vendors of integrated analytical platforms of-

fers full functionality required in management prac-

tice of business or public organizations. Moreover, an

open source software often has functionality similar

to commercial tools or even enhances speciﬁc busi-

ness analytic modules. Flexibility and ease of adap-

tation to the particular needs are the advantages of an

open source software.

Our example follows and supplements BI environ-

ments evaluation presented in (Dudała et al., 2010).

The evaluation was conducted in ﬁve modules (ac-

cording to classical architecture of BI environments):

Database/Data Warehouse server, ETL tools, OLAP,

Data Mining and Reporting tools. In each module a

set of criteria was proposed. This modules take a role

of criteria clusters in the ANP model. They are com-

plemented by two other clusters: Main objective and

Alternatives. The structure of the model, generated

by Superdecisions software, is presented in ﬁg. 1.

Figure 1: Clusters and their interrelations in the ANP

model.

The cluster numbering and priorities of criteria

clusters are as follows: 1. Alternatives; 2. Busi-

ness Intelligence Vendor (main objective); 3. Data

Base/Data warehouse server (0.042); 4. Data Mining

(0.193); 5. ETL (0.229); 6. OLAP (0.418); 7. Report-

ing server (0.116).

The limited space prevent us to present the com-

plete input data. Below, there are given only some

parts of the unweighted supermatrix. Table 1 contains

the example of criteria and their initial priorities.

Table 1: Criteria in cluster 6 OLAP.

No. Criterion Priority

61 Diff. data sources OLAP 0.139376

62 Graph. interf. OLAP 0.163747

63 Lic/ﬁn cond. OLAP 0.162196

64 MDX language 0.139376

65 MS Ofﬁce int. OLAP 0.139376

66 Security OLAP 0.127965

67 User supp. OLAP 0.127965

The priorities of alternatives with respect to the

selected criteria are presented in Tab. 2 (sample for

the cluster OLAP).

Table 2: Selected alternatives’ priorities with respect to cri-

teria from cluster 6 OLAP.

Alternative

Diff. data

sources OLAP

User supp.

OLAP

V1 0.084863 0.187670

V2 0.459105 0.187670

V3 0.154541 0.363056

V4 0.154541 0.199955

V5 0.146949 0.061648

A number of dependencies between criteria have

to be considered. They regard criteria belonging to

the same cluster and are represented by ’inner depen-

dence loops’ in ﬁg. 1. The dependence of ’Lic/ﬁn

ICEIS2012-14thInternationalConferenceonEnterpriseInformationSystems

328

cond.’ on other criteria in cluster ’Performance’

on ’Scalability’, ’Paralell computing’ and in cluster

’OLAP’ is a good example (see Tab. 3).

Table 3: The dependence of ’Lic/ﬁn cond.’ in cluster 6

OLAP.

No. Criterion Priority

62 Graph. interf. OLAP 0.113512

64 MDX language 0.539254

65 MS Ofﬁce int. OLAP 0.244404

67 User supp. OLAP 0.102830

Calculations for the ANP and AHP models have

been done with Superdecisions software (ANP Team,

2012). Fig. 2 shows the ﬁnal priorities of alternatives

given by ANP. For the sake of comparison the results

of AHP are also presented.

Figure 2: Final priorities of alternatives given by ANP and

AHP models.

As it can be seen at Fig 2, the assessment with

the ANP not only has changed the overall ranking but

also has differentiated vendors much more then AHP.

Hence, the ANP gives a better base for the ﬁnal deci-

sion.

5 CONCLUSIONS

A software selection is a task that have to take into

account multiple, often interdependent factors. This

article shows how this task can be done with the ANP

method. In comparison with the other methods, ANP

allows better modeling of the needs of users as it al-

lows for the relationships between elements of the

modeled system. In fact, our example has demon-

strated that an inclusion of the interrelations among

factors may lead to different results in comparison

to methods with independence principle (represented

here by AHP).

We have built and solved the ANP model for an

extended problem of BI system selection, in which all

main modules has been considered: Data Base/Data

Warehouse Server, Data Mining, ETL, OLAP and Re-

porting Server. Each of these modules, in turn, con-

tained 7-14 criteria. Altogether the problem embod-

ied 49 criteria, and additionally there were some de-

pendencies between them. Inevitably, requiring the

hundreds of comparisons, the procedure became re-

ally labor intensive and high demanding. The only ex-

cuse of this inconvenience is that several hours spent

on pair-wise comparisons may be assumed as not so

high cost in comparison with the overall time and

money expense of such a big and important project.

This also suggests a potential direction of the future

study towards the methods with similar capabilities

but less laborious and less demanding.

REFERENCES

ANP Team (2012). www.superdecisions.com.

Aya

g, Zeki (2011). Evaluating simulation software alterna-

tives through ANP.

Dudała, J., Gołuchowski, J., Kajfosz, K., Kania, K., and

Sta

s, T. (2010). Business intelligence environment

comparison (in polish). Technical report, University

of Economics in Katowice, Katowice.

Fritz, C. and Carter, B. (1994). A classiﬁcation and sum-

mary of software evaluation and selection methodolo-

gies. Technical Report 940823, Mississippi State Uni-

versity.

Jung, H. and Choi, B. (1999). Optimization models for

quality and cost of modular software systems. Euro-

pean Journal of Operational Research, 112(3):613–

619.

Lai, V., Wong, B., and Cheung, W. (2002). Group decision

making in a multiple criteria environment: A case us-

ing the AHP in software selection. European Journal

of Operational Research, 137(1):134–144.

Saaty, T. L. (2005). Theory and Applications of the Analytic

Network Process. Decision Making with Beneﬁts, Op-

portunities, Costs and Risks. RWS Publications, Pitts-

burgh.

Stamelos, I., Vlahavas, I., Refanidis, I., and Tsouki

as, A.

(2000). Knowledge based evaluation of software sys-

tems: a case study. Information and Software Tech-

nology, 42(5):333–345.

Valiris, G., Chytas, P., and Glykas, M. (2005). Making de-

cisions using the balanced scorecard and the simple

multi-attribute rating technique. Performance Mea-

surement and Metrics, 6(3):159–171.

Wieszała, P., Trzaskalik, T., and Targiel, K. (2011). Ana-

lytic network process in ERP selection. In Multicrite-

ria Decision Making’10-11, pages 261–286. Univer-

sity of Economics in Katowice, Katowice.

Wu, W. (2008). A hybrid approach to IT project selec-

tion. WSEAS Transactions on business and Eco-

nomics, 5(6):361–371.

AssessmentandChoiceofSoftwareSolutionwiththeAnalyticNetworkProcessMethod

329