MEASURING THE IMPACT OF ENTERPRISE SYSTEMS ON

BUSINESS OBJECTIVES

Donald Chand, George Hachey, James Hunton, Vincent Owhoso, Sri Vasudevan

Bentley College, Waltham, MA 02452, USA

Keywords: Enterprise Resource Planning

Systems, Balanced Scorecard

Abstract: Our team is engaged in a study seeking to dev

elop a holistic framework for measuring the success of ERP

systems The rationale for our project is based on (1) the belief that “nobody invests in ERP systems solely

for the sake of improving business processes,” but to achieve better business results and (2) our finding that

there is a lack of consensus and clarity in the literature about how to define the success of ERP systems. As

part of this study, we analyzed an SAP implementation by a major international aircraft engine

manufacturing and service organization, and it revealed that ERP systems impact the business objectives of

the firm and the balanced scorecard approach can be adapted to build a holistic framework for identifying

success measures that link to company’s goals and strategies. This research-in-progress paper reports our

findings on balanced scorecard applicability for assessing the business contributions of ERP systems.

1 INTRODUCTION

ERP stands for Enterprise Resource Planning, and

the term is used for any software system designed to

support and automate the business processes of

medium and large businesses. The complexity of

implementing ERP systems is well noted and

analyzed in the literature [(Ahituv, Neumann &

Zviran, 2002), (Bhattacherjee, 2000), (Brown &

Vessey, 2001), (Markus & Tanis, 2000), (Markus,

Yanis & Fenema, 2000), (Scheer & Habermann,

2000), (Willcocks & Sykes, 2000)]. However, the

problem of assessing the benefits of ERP systems is

less well understood. Markus and Tanis (2000)

suggest that ERP success should consider the firm’s

motivations for implementing ERP systems and the

time period when the benefits are assessed. Shang

and Seddon (2000) suggest that there are different

types of ERP benefits that can be classified as IT

infrastructure benefits, operational benefits,

managerial benefits, strategic benefits and

organizational benefits. Rosemann & Wiese (1999)

and Rosemann (2001) suggest that the balanced

scorecard approach can be used to measure the

performance of ERP systems. They view the ERP

life cycle in terms of two phases, namely, ERP

implementation and ERP use, and they suggest

adapting the balanced scorecard dimensions to each

phase as an IT project. In summary, the key

learning from the ERP success literature is that ERP

benefits should be conducted at different points in

time and each assessment should reflect the benefits

to the timeframe under consideration.

The Balanced Scorecard, a technique developed

Kaplan & Norton (1996), is a powerful

framework for translating qualitative business

strategies into quantitative goals, and linking the

strategies and goals to internal organizational actions

via a robust set of key performance indicators.

Several researchers [Martisons et. al. (1999), Van

Grembergen & Van Bruggen (1997), Van

Grembergen, Saull & Haes (2003)] have suggested

that the balanced scorecard may help to evaluate the

performance of information systems (IS) and to

evaluate IT investments in a holistic manner.

Rosemann (2001) approach of defining an ERP

balanced scorecard does not directly connect with

the business goals and strategy of the organization

because the customers of the ERP system in their

approach are primarily the internal users of the

organization, the business processes are the IT

processes that support the implementation and

operation of the ERP system, and the business

objectives are the IT objectives for the ERP system.

Rosemann and Wiese assume that the cascading

balanced scorecards from the ERP systems level, to

609

Chand D., Hachey G., Hunton J., Owhoso V. and Vasudevan S. (2004).

MEASURING THE IMPACT OF ENTERPRISE SYSTEMS ON BUSINESS OBJECTIVES.

In Proceedings of the Sixth International Conference on Enterprise Information Systems, pages 609-612

DOI: 10.5220/0002656206090612

 SciTePress

the IT department level, to the organization level

will link the ERP balanced scorecard to the

organizational business objectives. We did not find

this to be a satisfactory approach to measure the

success of ERP systems. Instead, we decided to use

a case study approach to analyze successful ERP

implementations to study the impact of ERP systems

on the customers, business processes, financial and

the learning and innovation dimensions of the

organizational balanced scorecard. Thus, the

objective of this project is to develop frameworks

for aligning ERP implementations and operations

with strategic business objectives in a way that will

maximize the value-added contribution of ERP

systems to organizations.

We are just completing the pilot case study of an

engine manufacturing and service organization that

claims to be very successful in implementing the

first six phases of its SAP implementation in the last

three years. Our interviews with the IT managers

and business managers brought out sample measures

in all four categories of balanced scorecard that have

been positively impacted by their ERP

implementation thus far. The rest of this paper will

describe the SAP implementation and sample

organizational success measures that are positively

impacted by the SAP systems.

2 SAP IMPLEMENTATION

SAP is the dominant ERP vendor. SAP is both a

company, founded in 1972 and head quartered in

Walldorf, Germany, and software that automates

collaborative business processes for all types of

industries and for every major market. SAP is being

implemented in our case study organization using a

phased approach, with a four year planned roll-out

for each phase.

The goal of phase I was to demonstrate a quick

success, and develop a trusting working relationship

among the different SAP implementation partners. A

newly acquired engine service depot was selected as

the initial site for implementing SAP’s materials

requisition modules. This phase I implementation

ran smoothly using the standard implementation

methodology suggested by the business process

consulting partner.

In phase II the company made a decision to

implement the core financial models next. To avoid

customization of SAP code, the company adopted an

out-of-box implementation strategy with standard

configuration. The financial models were

implemented globally.

In phase III, the warehousing and parts tracking

modules in SAP were implemented. The inventory

tracking system is based on standard bar

codes/scanner technology.

In phase IV, the front-to-back supply chain was

automated at the initial site. In this phase, full order

processing, engine and parts management, labor and

material accounting, logistics, parts disposition and

invoicing modules were implemented.

In phase V, full project management of engine

production assembly was implemented in SAP. In

this phase, the process of receiving an engine, giving

it an overall inspection, tearing it down into

modules, inspecting the modules, repairing parts or

acquiring and replacing parts, and assembling,

testing and shipping the engine back to the customer

became paperless. This implementation also allowed

links with external suppliers via an Internet portal.

The goal of phase VI was to deploy the full

materials-requisition solution that was successfully

implemented at the initial site to engine center sites

2 and 3. The lessons learned in phase IV enabled a

very smooth deployment at the site 3 Engine Center.

The organization is now involved in deploying

the component repair modules of SAP in all other

globally located repair facilities.

3 SUCCESS MEASURES

The organization claims to be very happy with its

ERP implementation efforts and outcomes. When

we asked for specific measures that could illustrate

the success of ERP in our case organization,

company managers offered the following examples:

• On time delivery of overhauled engines

improved from less than 50% to 95%;

• On time delivery of spare parts improved from

30% to 90%;

• The timeliness and availability of information

improved dramatically;

• The reliability and integrity of data improved

noticeably;

• Internal controls were tightened;

• Work stoppages, caused by lack of parts on the

floor, dropped from several per month to an

average of one-half per month;

• Enterprise-wide standardization of core finance

modules now provides the same financial

information to all the different stakeholders;

• The inventory availability rates increased from

60-70% to 95%;

• The cost of engine assembly process has been

significantly reduced;

ICEIS 2004 - DATABASES AND INFORMATION SYSTEMS INTEGRATION

610

• The later phases of the ERP implementation

were executed better than the earlier phases;

• The lessons learned from each phase are now

captured, documented and managed across the

enterprise;

• Refined training methods are enabling users to

adapt to the system quickly and more

effectively, mostly due to the innovative

involvement of power users in the creation and

delivery of training;

• Transfer of implementation process knowledge

from external consultants to company managers

was very effective.

It is interesting to note that these success

measures fall in one or more of the four categories

of balanced scorecard, as shown below:

Customer-Oriented measures

• On time delivery of overhauled engines

• On time delivery of spare parts

Business Process measures

• Timeliness and availability of information

• Reduction in work stoppages (Caused by lack of

parts on the floor)

• Little or no business disruption when next ERP

phase goes live

• Enterprise-wide standardization of the core

finance modules

Finance

• Cost of inventory reduction

• Cost of engine assembly process

Learning and Innovation

• Later phases are better executed than earlier

phases

• Capturing, documenting and managing the

lessons learned from each phase

• Effectiveness of training in enabling users to

adapt to the system quickly

• Early involvement of power users in the

creation and delivery of training

• Knowledge transfer from external consultants

into the company

4 FINDINGS

Although these examples of success measures in the

four BSC categories suggest the applicability of

BSC to the ERP domain, they are quite incomplete

in the sense there are many more measures in each

of the four dimensions. In addition, the granularity

of these measures is uneven, and because a balanced

scorecard is not just a compilation of all the

measures it is not clear whether each of these

measures should be included in a balanced

scorecard. However, these measures demonstrate the

impact of the ERP system on the business objectives

of the firm as discussed below.

4.1 Impact on business objectives via

the business process category

In the engine service centers, the on-time delivery of

engines improved from less than 50% to between 95

to 100%. Similarly, in the engine assembly process,

work stoppages due to unavailability of parts

dropped from several stoppages per month to an

average of ½ per month. Also, the inventory is better

managed now with the on-hand-and-available rate

improving from 60-70% to 95%. In short, the ERP

system has drastically improved productivity and

has driven efficiency. Furthermore, the ability to

mine corporate data has led to improved human

resource planning, better investment spending and

higher quality decision-making

4.2 Impact on business objectives via

customer category

A common requirement in the engine manufacturing

and service business is ad hoc requests for a spare

part. This happens every time an airplane has engine

problem and local diagnostics reveal a part failure

that needs replacement. Prior to the deployment of

the ERP system, there was no way to directly track

and locate the exact warehouse in which the required

spare part is located, hence, customers had to call

each warehouse individually until the part was

located. This process took a significant amount of

time and the delay costs were considerable. Today

however, the ERP system shows every warehouse

where each spare part is located, and this allows the

customer to select and order delivery of the part

from the nearest warehouse, thus improving the

delivery time on spare parts. Similarly, the

implementation of the ERP system in the engine

service centers has streamlined the engine overhaul

process leading to improved engine turn-around time

to the customer. Thus the impact of process

efficiency in this case is improved customer

satisfaction.

4.3 Impact on business objectives via

finance category

The company also indicated that the cost of doing

business has been dramatically reduced. The

MEASURING THE IMPACT OF ENTERPRISE SYSTEMS ON BUSINESS OBJECTIVES

611

reduction of work stoppages, the timeliness of data

availability and better controls have improved

corporate performance and promoted labor

efficiencies. Also, better inventory and supply chain

management has resulted in decreased costs.

Simultaneously, the ability to make accurate

commitments to trading partners and improve turn-

around time has increased the after market business,

thereby resulting in increased revenues.

4.4 Impact on business objectives via

learning and innovation category

The end-user training process developed by the SAP

business process consultants relied on the standard

SAP business processes. The training materials

were narrative, documented in very thick books and

cumbersome. By phase III of the implementation,

the organization realized that the training process

was not effective because many interventions were

required to get the users become reasonably

proficient in using the new system. The organization

developed a new innovative training process that

required power users from each user groups to work

with the new system for 4 - 5 weeks and help

develop training materials for peers. A work-step

instruction book with 10 pages of PowerPoint slides

was developed for each module. Users were

allocated to appropriate training classes where they

sat in front of computer terminals in a classroom

facilitated by the power user on selected production

data to learn how to use the systems and its features.

This new training process has been very effective

and has replaced the old training process.

An example of learning and innovation in the

user domain involves the design and modification of

ERP input and output screen for the hourly workers.

Most of the hourly workers are not computer literate

and the multiple screen user interfaces of SAP were

challenging for these workers. The implementation

team developed simplified task-driven screens for

this audience.

5 CONCLUSION

This research-in-progress paper has illustrated that

ERP systems impact the business objectives of the

firm and, therefore, the balanced scorecard approach

can be used to assess the value and success of an

ERP system. We have now developed a holistic

business scorecard framework for identifying the

success measures for ERP systems. The details of

this framework will be reported in our next paper.

REFERENCES

Ahituv, N., S. Neumann and M. Zviran., 2002. “A Systems

Development Methodology for ERP Systems, Journal of

CIS, Spring, 42, 3, pp. 56-67.

Bhattacherjee, A., 2000. Beginning SAP R/3 Implementation

at Geneva Pharmaceuticals, CAIS, Vol. 4, Article 2.

Brown, C. and I. Vessey, 2001. NIBCO’s “BIG BANG”,

CAIS, Vol. 5, Article 1.

Kaplan, R. & D. Norton, 1996.

The Balanced Scorecard:

Translating Strategy into Action, HBS Press.

Markus, L and C. Tanis, 2000. “The Enterprise Systems

Experience—From Adoption to success.” In R.W.

Zmud (Ed.) Framing the domains of IT research:

Glimpsing the Future Through the Past, Cincinnati, OH,

Pinnafles Educational resources, Inc.

Markus, L., C. Yanis, and P. C. can Fenema, 2000.

“Multisite ERP Implementations,” CACM, 43, 3, 42-46.

Martisons, M., R. Davison & D. Tse, 1999. ‘The Balanced

Scorecard: A foundation for Strategic Management

Information Systems” Decision Support Systems, Vol.

25, 1, pp.71-81.

Rosemann, M. & J. Wiese, 1999. “Measuring the

performance of ERP software: A Balanced Scorecard

Approach,” Proceedings of the 10

Australasian

Conference on Information Systems, Wellington,

Rosemann, M., 2001. “Evaluating the Management of

Enterprise Systems with the Balanced Scorecard.” In: IT

Evaluation Methods and Management. Ed. W. van

Grembergen. Idea Group Publishing, pp. 171-184.

Scheer, A. & F. Habermann, 2000. “Making ERP a

Success,” CACM, 43, 4, 57-61.

Van Grembergen, W. and R. Van Bruggen, 1997.

“Measuring and improving corporate technology through

the balanced scorecard technique,” Fourth European

Conference on the Evaluation of Information Systems,

Deflt , 163-171.

Van Grembergen, W. R. Saull, and S. De Haes, 2003.

“Linking the IT balanced scorecard to business

objectives at a major Canadian Financial group,” JITCA,

5, 1, 23-45.

Willcocks, L. & R. Sykes, 2000. “Multisite ERP

Implementations,” CACM, 43, 4, 32-37.

ICEIS 2004 - DATABASES AND INFORMATION SYSTEMS INTEGRATION

612