A first step towards the creation of an implementation support tool
Johan Magnusson, Andreas Nilsson
Department of Informatics, University of Gothenburg, Viktoriagatan 13, Gothenburg, Sweden
Fredrik Carlsson
Siemens Business Services AB, Stockholm, Sweden
Keywords: ERP, Implementation, Forecasting
Abstract: The continuing soar in popularity when it comes to standardized information sy
stems sold en masse under
the labelling of Enterprise Resource Planning (ERP) Systems is somewhat kept under control by the ever
flowing stream of reports from the industry of implementations gone bad. According to some researchers it
is possible to assume that as many as 90% of all initiated ERP implementation projects can be regarded as
failures as a result of changes in scope, prolongation of the project time or simply budget overruns. With the
implementation of an ERP system being a very costly and risky endeavour, organizations considering
“getting on the bandwagon” stand much to gain from pre-emptively forecasting the probability of success
for an ERP implementation in their enterprise. Given this, the purpose of this paper is to investigate a
possible conceptual framework for forecasting ERP implementation success and discuss the role of such a
framework in a software based tool. This was achieved through an initial in-depth literary review aimed at
finding factors affecting the outcome of the ERP implementation projects. These results were then
communicated to an industrial support group comprised of possible ERP implementation stakeholders. After
lengthy discussions concerning the usability, validity and reliability of the proposed list of factors, a
conceptual framework was agreed upon for forecasting ERP implementation success. The framework was
then tested against a number of possible stakeholders outside the industrial support group. As the results
show we have been able to create a conceptual framework for forecasting ERP implementation success that
is currently in the second wave of testing. The usability, validity and reliability of the framework is
discussed and elaborated upon, and this paper concludes that the perceived usability and hence also value of
the conceptual framework is substantial, whereas the validity and reliability remain to be tested.
With the overall soar in popularity for enterprise
wide systems such as MRP (during the 1970’s)
MRPII (during the 1980’s) and ERP (during the
1990’s) (See for instance Al-Mashari (2001) and
Schtub (1999) for a historical overview of the
evolution of enterprise wide systems), any possible
business benefit that these systems bring to the
adopting enterprise is directly dependant upon a
successful implementation.
Parr & Shanks (2000) take a further look upon
hy ther
e seems to be such an abnormal failure rate
for the implementation of ERP and go as far as
quoting Martin (1998) who stated that as many as
90% of all ERP implementations are either late or
over budget. If the success of a project (such as an
ERP implementation) is supposed to be measured as
for instance Whyte & Fortune (2003) stipulate (with
the variables time, budget, specifications and
consequences of project on organization), this would
lead to a failure rate of 90% for all ERP
These figures might at first seem dismal, but
the process o
f ERP implementation
encompassing both the actual implementation of a
Magnusson J., Nilsson A. and Carlsson F. (2004).
A CONCEPTUAL FRAMEWORK FOR FORECASTING ERP IMPLEMENTATION SUCCESS - A first step towards the creation of an implementation
support tool.
In Proceedings of the Sixth International Conference on Enterprise Information Systems, pages 447-453
DOI: 10.5220/0002602404470453
standard system and a more or less extensive change
in the fundamental process-structure of the
enterprise to fit the processes supported by the
standard system, they are not as extreme as might be
expected. In fact, Procaccino et al (2002) state that
85% of all IT-related projects fail and with this
relative high failure rate for projects spanning the
entire spectra of complexity, a success rate of 10%
for complex IT-related projects such as ERP
implementation projects might even be considered
With the current status of the IT-market being
somewhat in turmoil, any estimation of the global
ERP market is indicative at most. However,
according to Yen, Chou & Chang (2003), over 70%
of the Fortune 1000 companies have implemented
core ERP systems and the license fees for ERP
systems in Europe comprise of over half of the total
software license fees in Europe. When it comes to
the future size of the worldwide ERP market,
estimates vary from 11,90 Billion $US in the year
2007 (ARC Advisory Group, 2002) to 66.6 Billion
$US in the year 2003 (AMR Research, 1999).
As many researchers previously have pointed
out, the risks involved with implementing an ERP
are substantial (see for instance Davenport (1998);
Scott & Vessey (2000) and Sarker & Lee (2003) for
an overview of failed ERP implementations).
However, as the boom in the ERP market has shown
during the recent years, this does not intimidate the
Given the complex nature of the implementation
of enterprise wide and enterprise critical systems,
and the often painful and arduous experience that the
ERP adoption process leads to, the purpose of this
paper is to present a conceptual framework for
forecasting the probability of ERP implementation
success and discuss the role of such a framework in
a software based tool.
The process of ERP implementation is in this
paper regarded as any alteration in the current
system architecture of the enterprise related to some
kind of enterprise wide information system. With
this broad definition of ERP implementation, we
encompass such alterations to the system
architecture as upgrades and continued roll-outs. The
notion of “ERP implementation success” is defined
as the success of the implementation project, and
“probability of ERP implementation success” is
measured by to what extent an organization fulfils a
number of factors.
A schematic graph of the research-process is
presented in Figure 1 below.
As can be seen in Figure 1, the first step was to
identify a number of factors (15) through an
extensive literary review (encompassing a total of
155 articles or books) and present these to the
Industrial Support group. The factors were then
discussed and one factor (Competence) was added
along with a division of the now 16 factors into 4
categories. In addition to this a lengthy discussion
concerning the usability of the resulting conceptual
framework and the scientific validity and reliability
was held, creating further input for the academic
representatives and their further work with the
conceptual framework.
Figure 1: Methodological design
After designing the framework taking all input
into consideration the academics decided to
distribute the results through a software-based tool
with a web interface. This decision was based on
previous experience from the researchers stating that
the spread out usage of web-based technology would
in this way work in our favour, but several other
possibilities like workshop-methodology and expert
interviews were taken into consideration.
The software based tool ERP Scorecard was
designed as simply an electronically distributed
version of the questions comprising the conceptual
framework. Along with some additional
functionality regarding the management and
distribution of results, the tool was distributed free-
of-charge to 10 organizations currently undergoing
some sort of ERP implementation. As the tool
underwent initial testing during the summer of 2003
and was redesigned in accordance to the test-results,
the end results were a tool ready for extensive
dissemination during the late fall of 2003. As part of
ongoing research at the University of Gothenburg,
the tool pools all data centrally, creating a large
database for future quantitative analysis. The results
from the testing towards the Industrial Testers (see
Figure 1) was information regarding the perceived
usability of the software based tool and the
conceptual framework, along with data to be used as
a means of measuring the validity and reliability of
the framework.
The conceptual framework is as previously stated a
framework comprised of a number of un-weighed
factors with the ERP implementation project as a
focus. This highlights the link between fulfilment of
the factors and a positive outcome of the project, and
for the framework to as usable as possible we have
based the total level of factor fulfilment on how
many of the final 16 factors were fulfilled. For
instance; if the responding organization fulfils 12 out
of the 16 factors (simply yes or no based on 5
questions per factor), this will result in the
forecasting of a 75% probability of success, and in
the tool a text describing what the organizations
strengths and weaknesses are related to the different
factors will be presented along with a quick-list of
possible future managerial actions to strengthen the
identified weaknesses.
A description of the 16 factors with the
corresponding literary support can be found in Table
1 below. As shown under the heading of “Factor
description”, the object of analysis is the
During the industrial feedback sessions
concerning the first draft of the conceptual
framework, a need for the user to see some sort of
structure in the 16 factors was identified (see Figure
1 for further information regarding the research
methodology). This resulted in the reorganizing of
factors into four overlying categories or that would
enhance the usability of the framework. The
reliability and validity of such a categorization was
considered to be irrelevant, with the need of the
future user in sharp focus.
The four categories were identified as Top
Management, Project, Organization and System; and
they are presented together with the underlying
factors in Figure 1 below. As previously described
the fulfilment of the factors is in the basic outline of
the framework measured by five questions each (Q1-
5 in Figure 2), resulting in a total of 80 questions.
These questions have been left out of this paper as
an affect of them constantly being under revision
and testing.
Table 1: Factor name, description and corresponding literary references
Factor Name Factor Description Literary support
Strategy The organization should have a clear, communicated business
strategy and an aligned IS/IT strategy.
Aladwani, 2001;Al-Mashari et al, 2003
Al-Mashari, 2001; Cooke & Peterson, 1998; Davenport,
1998; Donovan, 1999 ; Holland 6 Light, 1999; Pinto &
Slevin, 1987; Schneider, 1999 ; Stevens, 1998; Umble
et al, 2003; Whyte & Fortune, 2002
Leadership The organization should have a strong and committed
leadership that has the ability to motivate the employees to
Al-Mashari & Zairi, 2000; Al-Mashari et al, 2003;
Mandal & Gunasekaran, 2003; Sarker & Lee, 2003
Schneider, 1999; Skok & Legge, 2002
Whyte & Fortune, 2002
Support The organization should have a top management and steering
committee of the ERP Implementation project that is highly
committed to the implementation and is comprised of
individuals with differentiated views of the implementation.
Aladwani, 2001; Kerzner, 1987; Mabert et al, 2001;
Mandal & Gunasekaran, 2003; Parr & Shanks, 2000;
Pinto & SLevin, 1987; Procaccino et al, 2002; Skog &
Legge, 2002 ; Umble et al, 2003
Whyte & Fortune, 2002
Competence The organization should have individuals with a broad
competence of ERP, BPR or other IT-related projects involved
in both the steering committee and the entire project.
No clear support found
Team The organization should have an implementation project team
that is comprised of individuals representing different views
and perceptions of the enterprise and the enterprise system.
Mabert et al, 2001; Sarker & Lee, 2003; Schneider,
1999; Skog & Legge, 2002; Umble et al, 2003; Whyte
& Fortune, 2002
Management The organization should have an excellent project
management for the implementation project and ensure that
the management does not present only a business- or technical
perspective of the implementation.
Cooke & Davis, 2002; Kerzner, 1987
Kirby, 1996; Mandal & Gunasekaran, 2003; Parr &
Shanks, 2000; Procaccino et al, 2002; Skog & Legge,
2002; Umble et al, 2003; Whyte & Fortune, 2002
Plan The organization should have a previously defined and well
communicated project methodology that envelops both
documentation procedures and clear performance
measurements with routines for monitoring progress.
Al-Mashari et al, 2003; Cooke-Davis, 2002; Mabert et
al, 2001 ; Mandal & Gunasekaran, 2003; McDonough
III, 2000; Parr & Shanks, 2000; Pinto & Slevin, 1987;
Procaccino et al, 2002; Schneider, 1999; Skog & Legge,
2002; Umble et al, 2003; Whyte & Fortune, 2002
External The organization should have an ability to manage the
influence of external consultants in the implementation project
and also be able to optimally transfer the knowledge from the
consultants into the organization.
Skog & Legge, 2002; Whyte & Fortune, 2002
Culture The organization should have a business culture that highlights
the importance of learning, knowledge, past experience and
change, as well as a strategy for knowledge management.
Al-Mashari, 2001; Ash & Burn, 2003; Chan, 1999;
Cooke-Davis, 2002 ; Davenport, 1998 ; Gable et al,
1998 ; Holland & Light, 1999; Krumbholz & Maiden,
2001; Schneider, 1999; Scott & Vessey, 2000; Soffer,
Golany & Dori, 2003; Stevens, 1997; Sumner, 1999;
Whyte & Fortune, 2002
Change The organization should have a fundamental willingness and
readiness for change as well as an explicit change
management strategy.
Aladwani, 2001; Al-Mashari & Zairi, 2000; Al-Mashari
et al, 2003; Ash & Burn, 2003; Hall, 2002; Hammer &
Stanton, 1999; Hong & Kim, 2002; Jiang & Muhanna,
2000: Kerzner, 1987; Laughlin, 1999; Mabert et al,
2001; Mandal & Gunasekaran, 2003;
Markus & Tanis, 2000; Parr & Shanks, 2000;
Schneider, 1999; Skog & Legge, 2002 ; Umble et al,
2003 ;
Whyte & Fortune, 2002
Process The organization should have a high level of process-maturity
and explicit guidelines for process management.
Al-Mashari et al, 2003; Al-Mashari, 2001; Bingi et al,
1999; Cooke-Davis, 2002; Edwards, 1999; Hong &
Kim, 2002; Hong & Kim, 2002; Koch et al, 1999;
Mandal & Gunasekaran, 2003; Marius & Ashok, 1996;
Palaniswamy & Frank, 2000; Skok & Legge, 2002;
Soh et al, 2000;Weil & Olson, 1989
Communication The organization should have a detailed communication plan
and strategy that ensures the successful communication of
project plan and progress to all relevant stakeholders.
Aladwani, 2001; Al-Mashari & Zairi, 2000; Al-mashari
et al, 2003; Mabert et al, 2001; Mandal & Gunasekaran,
2003; Pinto & Slevin, 1987; Schneider, 1999; Skog &
Legge, 2002; Swan et al, 1999; Whyte & Fortune, 2002
Technology The organization should have a clear understanding of the
existing legacy environment and the technological aspects
involved in the implementation of the ERP system.
Al, Mashari et al, 2003; Al-Mashari, 2001; Bancroft et
al, 1998; Barnes, 1999; Bingi, 1999; Harrell et al, 2001;
Holland & Light, 1999; Hong & Kim, 2002; Keller &
Teufel, 1998; Koch et al, 1999 ; Mabert et al, 2001 ;
Mandal 6 Gunasekaram, 2003; Parr & Shanks, 2000;
Schneider, 1999; Soffer, Golany & Dori, 2003; Swan et
al, 1999; Umble et al, 2003; Xu, Nord, Brown & Nord,
Training The organization should have a clear educational strategy
concerning the ERP implementation that involves routines for
early hands on training for the employees.
Aladwani, 2001; Al-Mashari et al, 2003 ; Mabert et al,
2001; Mandal & Gunasekaran, 2003; Skok & Legge,
2002; Umble et al, 2003; Whyte & Fortune, 2002
User The organization should have an implementation process that
strives for a high level of user acceptance early on through the
use of constant presumptive end-user consultations.
Mandal & Gunasekaran, 2003; Pinto & Slevin, 1987;
Procaccino et al, 2002; Skog & Legge, 2002; Whyte &
Fortune, 2002
Empowerment The organization should have a high level of implementation
process transparency and a staff policy that empowers team
members, end-users and management.
Aladwani, 2001; Grifith et al, 1999 ; Hong & Kim,
2002 ; Mabert et al, 2001 ; Markus & Robey, 1988 ;
McDonough III, 2000; Parr & Shanks, 2000; Sarker &
Lee, 2003; Schneider, 1999
Figure 2: The resulting conceptual framework
With the methodology applied to exploring the
possible conceptual framework for forecasting
ERP implementation success involving a literary
review followed by a re-evaluation of the
identified factors towards industrial
representatives, there are some implications
concerning the usability, validity and reliability of
the framework that need to be addressed.
When it comes to the usability and the
pragmatic value for the users of the conceptual
framework it has through our first meetings with
the industrial representatives and the first wave of
testing been rated as very high. The outcome of the
assessment is according to our findings not
primarily the forecasting of the probability of
success, but rather the pedagogic value of, on a
conceptual level, identifying relevant factors of
importance for the outcome of the ERP
implementation project. By disseminating the
questionnaire (and perhaps also the results)
throughout the organization to relevant
stakeholders, the organization will act to heighten
the level of consciousness concerning possible
pitfalls during a potential ERP implementation and
hence be better prepared for the project. After the
initial wave of testing, this (along with the creation
of a material for discussion in the Steering
Committee or equivalent) is regarded as the chief
added value of the forecast.
Concerning the validity of the 16 factors as
representing the basis for forecasting the total
probability of success, it is fairly weak. Very little
research has been conducted with data that
provides the opportunity to see the relationship
between the fulfilment of one factor and a positive
outcome regarding ERP, and this clearly limits the
possibility of creating a conceptual framework
with high validity. There is also the issue
concerning the relationship between the different
factors and their interlacing. It is natural to assume
that they both overlap and miss several necessary
factors that in the end will be necessary to
correctly forecast the outcome of the project, IF
this is at all possible.
When it comes to the reliability of the approach
of using the fulfilment of a number of un-weighed
factors as the basis for assessing and forecasting
the outcome of the project in focus, it can be
regarded as weak. We can not be sure that we have
the correct instruments to measure what we set out
to measure, but this is a question that more or less
becomes an issue for the future design of possible
methods for collecting the necessary data.
As we have described in this paper, the current
design of the forecasting is through the use of a
distributed web-based questionnaire encompassing
a total of 80 questions, five per factor; but this is in
no way the only design possible for conducting the
assessment. Alternative designs such as workshop-
based meetings with future stakeholders within an
organization and semi-structured interviews with
organizational representatives are also quite
possible, and also most likely to result in a higher
degree of reliability when it comes to the final
forecast. This can be seen as a trade-off between
quantity and quality in the resulting data, and with
the current design we have chosen to aspire a high
quantity, perhaps at the cost of quality.
Concerning the role of the conceptual
framework in the software-based tool, we can
conclude that according to our initial findings we
regard the pragmatic value as substantial.
This work was made possible through the Paper
Mill and EKIT-project at Gothenburg School of
Economics and Commercial Law/University of
Gothenburg, and the Enterprise Systems Research
Aladwani, A.M. 2001. Change management strategies
for successful ERP implementation. Business Process
management journal, 7(3):266-275
Al-Mashari, M. & Zairi, M. 2000. Information and
business process equality: the case of SAP R/3
implementation. Electronic journal on information
systems in developing countries, 2.
Al-Mashari, M. 2001. Process orientation through
Enterprise Resource Planning (ERP): A review of
critical issues. Knowledge and Process Management,
Al-Mashari, M., Al-Mudimigh, A. & Zairi, M. 2003.
Enterprise resource planning: a taxonomy of critical
factors. European journal of operational research
146: 352-364
Ash, C.G. & Burn, J.M. 2003. A strategic framework for
the management of ERP enabled e-business change.
European journal of operational research 146: 374-
Bancroft, N., Seip, H., Sprengel, A. 1998. Implementing
SAP R/3: How to introduce a large system into a
large organization. Manning publications CO:
Greenwich, CT.
Bancroft, N.H., Seip, H. & Sprengel, A. 1998.
Implementing SAP R/3, 2nd edition, Manning
Publications, Greenwich, CT
Barnes, M. 1999. Customization of ERP requires
development skills. Information week 22 feb
Bingi, P., Sharma, M.K. & Godla, J.K. 1999. Critical
Issues affecting an ERP implementation. Information
systems management 16(3):7-14
Cooke-Davies, T. 2002. The “real” success factors on
projects. International journal of project
management, 20:185-190
Davenport, T.H. 1998. Putting the enterprise into the
enterprise system. Harvard Business Review,
Donovan, M. 1999. Successful ERP implementation the
first time. Midrange ERP August.
Gattiker, T.F. & Goodhue, D.L. 2000. Understanding the
plant level cost and benefits of ERP: will the ugly
duckling always turn into a swan? In: Proceedings of
the 33rd Annual Hawaii International Conference on
System Sciences
Griffith, T.L., Zammuto, L. & Aiman-Smith, L. 1999.
Why new technologies fail? Industrial
Hall, R. 2002. Enterprise Resource Planning systems
and organizational change: transforming work or
organization? Strategic Change, 11:263-270
Hammer, M & Stanton, S. 1999. How process
enterprises really work. Harvard Business
Harell, H.W., Higgins, L. & Ludwig, S.E. 2001.
Expanding ERP application software: Buy, Lease,
Outsource or write your own? The Journal of
Corporate Accounting and Finance, Jul/Aug:37-43
Holland, C: & Light, B. 1999. A critical success factors
model for ERP implementation. IEEE Software
May/June 30-35
Hong, K.K. & Kim, Y-G. 2002. The Critical Success
factors for ERP implementation: an organizational fit
perspective. Information & Management 40:25-40
Jiang, J.J, Muhanna, W.A. & Klein, G. 2000. User
resistance and strategies for promoiting acceptance
across system types. Information and Management
Keller, G. & Teufel, T. 1998. SAP R/3 process oriented
implementation. Addison-Wesley: Wokingham
Kerzner, H. 1987. in search of excellence in project
management. Journal of systems management: 30-39
Koch, C. Slater, D. & Baatz, E. 1999. The ABC’s of
ERP. CIO: London
Krumbholz, M. & Maiden, N. 2001. The implementation
of enterprise resource planning packages in different
organizational and national cultures. Information
systems 26:185-204
Laughlin, S.P. 1999. An ERP game plan. Journal of
Business Strategy: 32-37
Mabert, V.A., Soni, A. & Venkataramanan, M.A. 2001.
Enterprise Resource Planning: Common myths
versus evolving reality. Business Horizon, May-June.
Mandal, P. & Gunasekaran, A. 2003. Issues in
implementing ERP: A case study. European Journal
of Operational Research 146:274-283
Marcus, M.L. & Tanis, C. The Enterprise System
Experience – From adoption to success In…
Marius, J. & Ashok, S. 1999. Package software:
selection and implementation policies. INFOR: 133-
Markus, M.L. & Robey, D. 1983. The organizational
validity of management information systems. Human
relations 36(3):203-226
Markus, M.L. & Robey, D. 1988. Information
technology and organizational change: causal
structure in theory and research. Management
Science 34(5):583-598
McDonough III, E.F. 2000. Investigation of factors
contribution to the success of cross-functional teams.
Journal of product innovation management, 17:221-
Palaniswamy, R. & Frank, T. 2000. Enhancing
manufacturing performance with ERP systems.
Information systems management 17(3): 43-55
Parr, A. & Shanks, G. 2000. A model for ERP project
implementation. Journal of Information Technology,
Pinto, J.K. & Slevin, D.P. 1987. Balancing strategy and
tactics in project implementation. Sloan Management
Procaccino, J.D., Verner, J.M., Overmeyer, S.P. &
Darter, M.E. 2002. Case study: factors for early
prediction of software development success.
Information and software technology, 44:53-62
Sarker, S. & Lee, A.S. 2003. Using a case study to test
the role of three key social enablers in ERP
implementation, Information and management. In
Schneider, P. 1999. Wanted: ERPeople Skills. CIO
Scott, J.E. & Vessey, I. 2000. Implementing Enterprise
Resource Planning systems: the role of learning from
failure. Information systems frontiers, 2(2):213-232
Skog, W. & Legge, M. 2002. Evaluating Enterprise
Resource Planning (ERP) Systems using an
interpretive approach. Knowledge and Process
Management, 9(2):72-82
Soffer, P., Golany, B. & Dori, D. 2003. ERP Modeling:
a comprehensice approach. Information systems
Soh, C. Kien, S.S. & Tay-Yap, J. 2000. Cultural fits and
misfits: is ERP a universal solution?
Communications of the ACM 43(3):47-51
Sumner, M. 1999. Critical Success factors in enterprise
wide information management systems projects.
Proceedings of the ACM-SIGCPR Conference:297-
Swan, J., Newell, S. & Robertson, M. 1999.
The illusion
of “best practice” in information systems for
operations management. European Journal of
Information Systems 8:284-293
Weill, P. & Olson, M.H. 1989. An assessment of the
contingency theory of management information
systems. Journal of Management Information
Systems 6(1):59-85
Whyte, D. & Fortune, J. 2002. Current practice in
project management – an empirical study.
International journal of project management, 20:1-11
Xu, H., Nord, J.H., Brown, N. & Nord, G.D. 2002.
Industrial management and data systems