Risk Analysis Techniques for ERP Projects
based on Seasonal Uncertainty Events
Paulo Mannini
1a
, Edmir P. V. Prado
1b
, Alexandre Grotta
1,2 c
and Leandro Z. Rezende
1d
1
IS Post-graduation Program (PPgSI), University of São Paulo (USP), São Paulo, Brazil
2
IS Graduation Program, Federal Institute of Sao Paulo (IFSP), São Paulo, Brazil
Keywords: ERP, Risk Analysis, Seasonal Uncertainty Events.
Abstract: Risk management is fundamental in order to increase Enterprise Resource Planning (ERP) project success
rate in order to plan, prevent and react to risks and uncertainties. But based on the literature review, we
identified a few studies relating to both seasonal uncertainty events (SUE) and ERP projects. Given this
context, this research objective is to analyse the most appropriate risk assessment techniques for ERP projects
based on SUE. In order to achieve this goal, we performed and Systematic Review of Literature and we
applied the Delphi technique with Project Management Professionals and Enterprise Directors. According to
the SLR result, we identified 16 techniques that are more suitable to deal with SUE on ERP projects. After
the Delphi panels perspective, six techniques pointed out as the most suitable for these projects. In addition,
we identified that not all techniques described by the literature converged with the researched context reality.
These findings are very relevant for both the Academia and the Industry to scaffolding SUE on ERP projects.
1 INTRODUCTION
Enterprise Resource Planning Systems (ERP)
systems are computer information systems designed
to process organizational transactions and enable
real-time planning, production and response to
consumers (Amid et al., 2012). Many organizations
have been implementing ERP systems via ERP
projects since the 1990s. ERP systems aim to achieve
enterprise uniformity between information systems
and the real business towards making the
organizations more competitive (Rajagopal, 2002).
Anyhow, ERP projects are a major concern to
organizations (Amid et al., 2012). For instance, that
are indications that some ERP projects might have a
bad reputation of being very costly and ineffective for
organizations (Motwani et al., 2005) including the
underdevelopment countries context. Part of these
claims is related to poor risk planning and control
over processes in ERP projects, resulting in negative
effects on the project outcomes (Tsai et al., 2009).
a
https://orcid.org/0000-0003-3563-4356
b
https://orcid.org/0000-0002-3505-6122
c
https://orcid.org/0000-0003-2549-138X
d
https://orcid.org/0000-0002-5543-9155
An ERP project is often complex and risky: it
requires a large investment across, it takes a long time
to the concluded. An ERP project also carries a high
risk to the organization (Qi & Zhu 2012; Aloini et al.,
2012): one of the top reasons is that managers do not
take into account proper manners to analyze all the
risks involved into an ERP project. Even further,
seasonal uncertainty events (SUE) are misconceived
(Schmidt et al., 2001) on ERP projects. One SUE
example on ERP projects is the freezing period, a
period of time when no software updates are allowed
except for the emergency ones (Neubarth et al. 2016).
However, despite the importance of SUE in ERP
projects risk analysis, there are few studies in the
literature relating to seasonality and project risk
management. Given this context, this research aims to
answer the following research question: What are the
most appropriate techniques to analyze risks in ERP
projects influenced by SUE? Thus the research main
objective is to analyze the most appropriate risk
assessment techniques for ERP projects based on
SUE. We defined two goals to achieve this objective:
Mannini, P., Prado, E., Grotta, A. and Rezende, L.
Risk Analysis Techniques for ERP Projects based on Seasonal Uncertainty Events.
DOI: 10.5220/0009420601730180
In Proceedings of the 22nd International Conference on Enterprise Information Systems (ICEIS 2020) - Volume 1, pages 173-180
ISBN: 978-989-758-423-7
Copyright
c
2020 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
173
(1) identify the techniques used for risk analysis in
ERP projects via a Systematic Literature Review
(SLR); (2) analyze the importance of identified
techniques to SUE via the Delphi technique.
This research has the other five sections. In the
next section, we present the theoretical bases used by
this research. In Section 3, we present the research
methodology that provides a research overview. In
Section 4, we present the data collection procedures.
In Section 5, we present and discuss the research
results, followed by the last section: conclusion.
2 THEORETICAL BASES
This section defines the main concepts used by this
research: ERP, project risk management, seasonality,
and risk techniques and methods.
(1) ERP. ERP systems were originally deployed to
facilitate manufacturing and business processes. Over
time, they evolved to include all organization
processes, such as sales, marketing, and human
resources. Companies are now using ERP via web and
mobile solutions in order to connect the entire value
chain, including their suppliers (Rainer and Cegielski,
2012). Additionally, there is a lack of employees’
knowledge regarding what an ERP system is and how
to operate it (Motwani et al., 2005).
(2) Project Risk Management. There are several
definitions of risk in the literature. One of the most
accepted definition is given by the Project
Management Institute (PMI): considers negative risks
as threats and positive risks as opportunities. For
PMI, project risk management as an area of expertise
that encompasses seven processes: plan risk
management, identify risks, perform a qualitative risk
analysis, perform a quantitative risk analysis, plan
risk responses, implement risk response, and monitor
risks (PMI, 2017). Another acceptable definition of
risks is the deviations from expectations, caused by
uncertainties that impact objectives positively or
negatively (ISO 2016). Anyhow, other organizations
consider the term risk as something negative only
(OGC, 2009; IPMA, 2015). Project managers often
lack knowledge about formal methods for project risk
management planning (Globerson and Zwikael
2002).
(3) Seasonality and SUE. Seasonality is a
periodic variation that presents a constant long-term
pattern. These variations are repeated, such as
annually, semi-annually or quarterly. An example of
seasonality may be a sales increase during the
Christmas season (Passari, 2003). Thus, SUE are
those uncertainties that might have a higher
probability to occur during certain periods know as
seasonally (Acebes et al. 2014). However, a project
risk analysis method must consider several other
aspects. Thus, we consider the seasonality effects on
projects according to three different types.
i. External Environment. For instance, the
winter might be considering an external
environment factor given that, for instance,
the snow might affect a construction project
or even the in-person meetings of an ERP
project (Acebes et al. , 2014).
ii. Products and Services. Seasonality might
affect the way people sell and acquire
products or services, in which the ERP
system might be prepared for those needs
(Mattsson, 2010).
iii. Processes and Operations. ERP system
freezing as an example of the most frequent
IT seasonality, and it affects the process and
company operations (Prado et al., 2017).
(4) Risk Techniques and Methods. The
technique might be defined as the manner that
technical details are addressed in order to achieve the
desired result. The method might be defined as a
systematic procedure, and an inquiry mode applied to
a particular discipline. Thus, techniques are applied
by humans and might utilize one or more methods
towards producing the desired result (PMI, 2017).
Regarding SUE and risk analysis techniques, we
can classify existing methods and their tools to fit into
four technique categories. The qualitative risk
analysis process techniques were based on (PMI,
2017) for RC. The categories DS, DP, and IP were
defined based on (Stair & Reynods, 2017), as follows:
o Risk Identification: groups risk identification
techniques. They are split into two categories:
Risk Categorization (RC) and [Risk] Data
Source (
DS);
o Risk Calculation: groups how risks are
calculated. It is identified by one single category,
the [Risk] Data Processing (DP);
o Risk
Presentation: groups how risk data is
presented and shared among the project team It
is identified by one single category, the [Risk]
Information Presentation (IP).
3 RESEARCH METHODOLOGY
This section defines the research methodology and
phases. In the first phase, we conducted an SLR. At the
second phase, we conduct a Delphi technique, two
rounds were needed. The conduction and the results of
the research are described in the following sections.
ICEIS 2020 - 22nd International Conference on Enterprise Information Systems
174
3.1 Research Phases
This is an exploratory study, based on qualitative data
(Creswell, 2019) applied to ERP system
investigation. It aims to contribute with new findings
regarding the gap in ERP projects regarding SUE.
This research had two phases as outlined in Figure 1.
Figure 1: Research phases.
The first phase (SLR) has the objective to identify
and classify the techniques used to analyze project
ERP project risks according to the literature. In the
second phase, were ranked these techniques
according to the prism of a Delphi technique. The
methodology used by these two phases is detailed at
the next two subsections as follows.
3.2 Systematic Literature Review
The SLR was performed based on (Kitchenham,
2009). The data were acquired from four different
search engines: Scopus, Science Direct, IEEE, and
ACM, covering a state-of-art period from 2012 to
2019. The specific query strings for each search
engine were created according to the following
keyword: ("Project risk analysis" OR "Project Risk
Assessment") AND "Project Risk Management";
(quali* OR quanti*) AND ("Risk Assessment" OR
"Risk Analysis") AND project AND "Risk
Management".
The protocol for quality criteria was based on
(Kitchenham, 2009) and is described as follow:
(1) Exclusion Criteria: i) duplicated articles; ii)
articles whose access were not free; iii) articles that
did not deal with project risk management; iv) articles
that did not address qualitative and/or quantitative
aspects of risk analysis; v) articles that did not provide
details techniques details or characteristics;
(2) Inclusion Criteria: studies that had
techniques for assessment and/or application in risk
analysis projects. We then read the articles to collect
the information according to the details given in the
next subsection.
3.3 Delphi Technique
According to Skulmoski et al. (2007) and Skinner et
al. (2015) Delphi technique is an appropriate
technique for acquiring expert recommendations
when addressing a research problem in the IT field. It
is suitable for ranking technology issues of new IT
product development projects, like the one proposed
in this research.
According to Dalkey and Helmer (1963), the
Delphi technique uses a group of experts, is based on
pre-established criteria, uses multiple rounds of
questioning with these experts, through questionnaire
or interview, and is applied individually to avoid
direct confrontation between them. Skinner et al.
(2015) state that there is no limitation on the number
of experts but should include people with knowledge
and experience in the subject being evaluated.
The Delphi technique has been used for risk
management in IT projects, mainly to prioritize the
risk factors involved in these projects (Huang et al.,
2004; Schmidt et al., 2001; Nakatsu & Iacovou,
2009). This technique is especially suitable for
studies in which the objective is to improve
understanding of problems, opportunities or solutions
(Skulmoski et al., 2007).
The need or not for a new Delphi panel round is
assessed by the Kendall coefficient of concordance (W)
and the statistical significance of this coefficient and
defined by the following formula (Siegel et al., 2006):
𝑊
12𝑆
𝑚²𝑛
𝑛² 1
𝑚
𝑇𝑗

Variable S represents the sum of the standard
deviations of all elements. Variable m represents the
number of panel members. Variable n represents the
number of elements evaluated in the panel. Thus,
variable Tj =
∑
𝑡𝑖³ 𝑡𝑖


, where ti is the number of
ranks in the ith grouping and gj is the number of draw
groups in the jth ordering set.
4 DATA COLLECTION
This section describes how both SLR (phase 1) and
Delphi (phase 2) data were collected and aggregated.
The results of this section are presented in the next
section (Results and Discussion).
4.1 SLR Data Collection
The SLR data was collected from the select articles
according to the four steps highlighted in dark-gray in
Risk Analysis Techniques for ERP Projects based on Seasonal Uncertainty Events
175
Figure 1. In summary, the four steps are detailed as
follows: First: we read the titles and abstracts to
verify if they met the defined criteria; Second: we
read the full paper text in order to verify the adequacy
of the article to the research objectives; Third: we
applied the quality criteria; Fourth: we collect all the
relevant information from the remaining 42 articles.
Figure 2: SLR Process.
4.2 Delphi Data Collection
This subsection first describes the participants'
profile, followed by the Delphi technique details. This
section also details the Delphi first and second
rounds.
4.2.1 Panelists Profile
The selection of Project Management Professionals
and Enterprise Directors to compose the Delphi panel
was based on two criteria: professional knowledge of
the topic of research at the academic or the
professional level. All participants were from the
same country. Below are the minimum criteria
considered.
Degree in the areas of Engineering or
Information Systems. In addition to undergraduate
Information Systems, all undergraduate engineering
was accepted, as the complexity of IT techniques and
methods exhibits typical characteristics of
engineering activities.
Experience in project management. Selected
professionals must have at least five years of
experience in public or private organizations; or at
least three years of experience in public or private
organizations and with postgraduate in project
management.
The reason for adopting these criteria is because
professionals with technical background and project
management experience have the appropriate profile
to address project risk management issues. After
selecting the professionals who met the minimum
criteria of knowledge and experience, two groups of
experts were defined:
System analysts, project managers, and project
leaders, hereinafter referred to as Project
Professionals (PP).
• Project professionals who work or have worked
on projects, but held an executive position,
hereinafter referred to as the board of directors (BD).
The selected experts were generically referred to
as “panelists” in subsequent steps of the Delphi
technique.
4.2.2 Delphi Technique Details
The questions sent to the experts were grouped into
the four categories identified in the SLR (RC, DS,
DP, and IP). We use online surveys through the
eSurveysPro platform. Some parts of the
questionnaires were customized for each participant
to facilitate the collection of personal data and to
provide participants with their choices in previous
rounds.
Panelists' opinions were collected through
questions about the level of importance of risk
analysis techniques in SUE. The questions used a
five-point Likert scale: 1-very low; 2-low; 3-medium;
4-high; and 5-very high. The following criteria were
used to finalize the panel rounds:
The value of the coefficient of concordance W
equal to or greater than 0.5, determining a high
convergence between opinions (Schmidt, 1997).
The chi-square value (χ2) greater than 43.82.
According to Siegel et al. (2006), chi-square values
for samples with 19 degrees of freedom, have p-value
of less than 0.001. In Fisher's significance scale
(Morettin & Bussab, 2017) p-value less than 0.001
represents a very strong statistical significance of W.
In summary, there were two rounds until the
criteria for completing the research rounds were met.
The first round of the Delphi panel allowed
participants to interact and suggest adjustments to
instrument questions.
The suggestions were analyzed by the researchers
before making any changes. In the first round,
researchers were also allowed to clarify questions for
participants about the research or the applied
questionnaire.
The second round used the outcome of the first
one, providing participants with the first choices and
allowing them to review their previous responses to
achieve convergence between the group. In this way,
participants received feedback from the first round
keeping their anonymity.
Start
Query the
databases
Read titles
and
abstracts
Read the
papers
Review
quality
criteria
385 articles
extracted
122 papers
selected
72 papers
selected
42 papers
selected
End
ICEIS 2020 - 22nd International Conference on Enterprise Information Systems
176
4.2.3 Delphi First Round
The panelists were contacted via email or Linkedin
platform. We explained the purpose of the survey and
provided the access link to the questionnaire.
The questionnaire was custom made, changing
only the names of the participants. It allowed
panelists' contributions to refining the questions of
the questionnaire. All contributions received were
reviewed and necessary adjustments made.
The first round was applied in March 2019 to 34
professionals. After a period of one month, it was
decided to close the first round of the panel with the
participation of 18 panelists, 14 from the PP group
and four panelists from the BD group.
The opinions collected resulted in values of W =
0.29 and chi-square = 99.18, which indicates a low
degree of convergence of opinions among the
panelists. Thus, it was necessary to carry out a new
round with the objective of increasing the degree of
convergence.
4.2.4 Delphi Second Round
The second round of the Delphi panel consisted of the
same questions evaluated in the first round and sent
to the 18 participants in the first round, allowing them
to review their answers. The responses were collected
in May 2019 and were attended by 12 panelists from
the PP group and four from the BD group. Opinions
collected in the second round resulted in values of W
= 0.52, which indicates a high degree of convergence
of opinions among the participating panelists in
(Schmidt, 1997).
The calculated value for chi-square was 158.08,
which indicates that W has high significance. Based
on these values, the results of the second round met
the panel's completion criteria, with W value having
high convergence and high significance.
5 RESULTS AND DISCUSSION
This section presents the SLR results, the Experts
Group Analysis, and the Delphi results. It also
presents the consolidated result. Then we discuss the
results and their limitations.
5.1 SLR Results
As SLR result, we identified 16 different risk
management techniques in project management,
according to the categories defined at the research
method section: Risk Category (RC), Data Source
(DS), Data Processing (DP) and Information
Presentation (IP). The median rank considers the
most/least cited risk analysis techniques in the
literature as detailed in Table 1 as follows.
Table 1: SLR risk analysis techniques.
Technique
Category
X Technique short name Technique Description
Risk Identification: risks are identified by…
f.
RC
Risk category
T01 - Risk source The sources where risks might come from (their sources) 13
T02 - Project area The project area where risks might come from 5
T03 - Project phase Each project phase where risks might come from 3
DS
Data Source
T04 - Experts opinion Taking into account subject matter (experts) opinion or vision 31
T05 - Historical database Investigating and utilizing risk historical data of past projects
or company knowledge
11
T06 - Stakeholders opinion Taking into account stokeholds opinion or vision 1
Risk Calculation: risks are calculated via…
DP
Data
Processing
T07 - Risk probability and impact
analysis
The probability of a certain risk might occur times the impact
it might cause if it happens
21
T08 - Modeling and Simulation Computer to model and simulate risk and their impacts 15
T09 - Multicriteria decision making
analysis
Merging several criteria to analyze and decide 12
T10 - Fuzzy Logic Using the fuzzy logic to predict risks and their consequences 10
T11 - Risk interdependence analysis Analyzing the interdependence among risks 7
Risk Presentation: risks are presented as…
IP
Information
Presentation
T12 - Prioritized list of risks A simple list of risks group by their prioritization 18
T13 - Tables A table showing the risks 13
T14 - Charts A chart, like pareto or pie charts 10
T15 - Project relationship managemen
t
A visual representation of risks and the project management 8
T16 - Impact and probability matrix A matrix that relates each impact with their probability 6
Legend: X= Comparing with the median; = Above; = below; f. = Citation frequency;
Risk Analysis Techniques for ERP Projects based on Seasonal Uncertainty Events
177
Table 2: Risk analysis techniques ranking after Delphi panels.
5.2 Experts Group Analysis
The most suitable techniques for risk analysis in SUE
are those classified as very high or high importance
and they are presented in table 2 (T07, T12, T16, T03,
T05, T11 and T02). There was a very strong similarity
between the results of the PP and BD groups, and with
the total number of panelists. Techniques ranked with
importance high and very high are the same for all
groups with only one exception in the BD group. This
group assigns more importance to T01 than to T02
Only five classifications, out of 16 made by the PP
and BD groups, had a gap greater than two. Most of
these techniques (T01, T02 and T03) belong to the
RC category. This shows that the biggest difference
in the assessment made by these two groups is related
to risk categorization techniques.
Professionals that held an executive position (BD)
ranked the techniques that categorize risk by source
and project phase more importantly than techniques
that categorize risk by project area. It is plausible to
conclude that the major importance assigned by these
professionals to phase risk categorization techniques
is that they do not perform activities directly related
to risk mitigation. They manage the project outcomes
and therefore have a different view of the activities.
5.3 Delphi Results
The outcome of the second round of the Delphi panel
consisted of 16 panelists, resulted in the scores shown
in table 2. The scores are presented by panelists group
(PP and BD) in order to compare the answers of these
two groups. The techniques ranking was based on the
five-point Likert scale used by panelists to answer the
questions.
5.4 Consolidated Results
Table 3 presents the techniques classified by citation
frequency in the literature and by the importance
given by the experts. The groups' analysis is presented
below. Risk Categorization (RC).
Table 3: Consolidate literature review vs. experts’ review.
TC Techniques LR* ER*
RC T01 - Risk categorization by source #6 #12
T02 - Risk categorization by project
area
#14 #7
T03 - Risk categorization by project
phase
#15 #4
DS T04 - Experts opinion #1 #10
T05 - Historical database #8 #5
T06 - Stakeholders opinion #16 #11
DP T07 - Risk probability and impact
analysis
#2 #1
T08 - Modeling and Simulation #4 #14
T09 - Multicriteria decision making
analysis
#7 #15
T10 - Fuzzy Logic #10 #16
T11 - Risk interdependence analysis #12 #6
IP T12 - Prioritized list of risks #3 #2
T13 - Tables #5 #9
T14 - Charts #9 #8
T15 - Project relationship
management
#11 #13
T16 - Impact and probability matrix #13 #3
Legend: * = underlined scores are above the median; TC = Technique
category; LR = Literature Ranking; ER = Experts Ranking
Technique T03 was rated by experts as the most
important among risk categorization techniques,
although it is the least mentioned in the literature.
Techniques Level of Ranking Ranking differences
importance All groups PP BD between groups
(A) (B) (C) (AxB) (AxC) (BxC)
T07 - Risk probability and impact analysis Very high 1 1 3 0 2 2
T12 - Prioritized list of risks 2 2 2 0 0 0
T16 - Impact and probability matrix 3350 2 2
T03 - Risk categorization by project phase High 4 4 1 0 3 3
T05 - Historical database 5742 1 3
T11 - Risk interdependence analysis 6660 0 0
T02 - Risk categorization by project area 7592 2 4
T14 - Charts Medium 8 8 8 0 0 0
T13 - Tables 910101 1 0
T04 - Experts opinion 10 9 11 1 1 2
T06 - Stakeholders opinion Low 11 11 12 0 1 1
T01 - Risk categorization by source 12 14 7 2 5 7
T15 - Project relationship managemen
t
13 12 15 1 2 3
T08 - Modeling and Simulation Very low 14 13 13 1 1 0
T09 - Multicriteria decision making analysis 15 16 14 1 1 2
T10 - Fuzzy Logic 16 15 16 1 0 1
ICEIS 2020 - 22nd International Conference on Enterprise Information Systems
178
However, considering risk management in SUE,
categorization by phase is important, probably due to
it allows better management of seasonal project
uncertainties.
Data source (DS). Technique T05 (historical
database) was rated above the median both in the
frequency of literature citation and in the degree of
importance given by the experts. This result is in line
with the work of Acebes et al. (2014) who used this
technique for risk analysis in SUE. Another finding
was a divergence in classification between techniques
T04 (Expert Opinion) and T06 (Stakeholder
Opinion). While the literature cites more the former,
the experts assigned more importance to the latter. In
addition, for Delphi panel experts the two techniques
have very close importance rankings. Therefore, it is
advisable to evaluate the cost-benefit ratio of
technique T04, because it involves the cost of an
expert opinion whereas technique T06 has a similar
level of importance, with a lower cost.
Data process (DP). Technique T07 (Risk
probability and impact analysis) was the most cited
data processing technique in the literature and
considered the most important by experts. This shows
that the risk exposure measure by probability and
impact is widely accepted by experts and the
literature. On the other hand, the T10 technique
(Fuzzy Logic) is not among the most important
according to experts. We identified that this result
was due to the lack of knowledge of six panelists
about the T10 technique applied to project risk
analysis.
Information presented (IP). Reports showing a list
of prioritized risks were highlighted by experts as the
most important and were the most cited in the
literature. This is evidenced by the T12 technique
(Prioritized list of risks) that was above the median in
experts’ opinion and literature citations. On the other
hand, there is a difference between experts and the
literature: the former prefers information in charts and
matrices and the latter on tables.
In summary, the consolidate results point out the
most relevant techniques to deal with SUE among
those techniques found in the literature and also
among those ones pointed out by experts.
5.5 Research Limitations
The most relevant limitations of this research are
related to the data collection and analysis process.
Qualitative data collection might be subject to human
bias, given they involve human judgment on
collecting and classifying them (Kitchenham, 2009).
In order to avoid bias, we carefully planned and
followed the research methodology according to the
scientific procedures as much as possible.
Additionally, the results might be limited to the
Delphi participants’ vision and contexts only.
6 CONCLUSIONS
ERP systems are a fundamental part of present-day
Enterprise technological infrastructure. Anyhow,
implementing and maintaining these information
systems is not an easy deal, which, most of the time,
is delivered by ERP projects.
Seasonality brings several additional risks that are
misconceived in ERP such as the seasonal uncertainty
events (SUE) (Schmidt et al., 2001). Given this gap,
the main objective of this research was to identify and
analyse the most appropriate techniques to analyze
risks in ERP projects influenced by SUE. We defined
two goals to achieve this objective: (1) identify the
techniques used for risk analysis in ERP projects via
a Systematic Literature Review; and (2) analyze the
importance of identified techniques to SUE via the
Delphi technique.
We identified that, according to the literature,
from 2012 to 2019, there were cited 16 techniques
that have been used to address SUE on ERP projects.
The most relevant ones were, for risk identification:
risk identification by source (T01) and Experts
opinion (T04). For risk calculation: risk probability
and impact analysis (T07) and modeling and
simulation (T08). And finally, in order to present the
risks to the team and stakeholders, prioritized list of
risks (T12) and Tables (T13).
We then presented these finds to participants into
two sessions of Delphi. After their perspective, the
results were for instance that the most used by ERP
project techniques are: Risk probability and impact
analysis (T07), Prioritized list of risks (T12), and
impact and probability matrix (T16). On the other
side, the rare techniques were: Modeling and
Simulation (T08), Multicriteria decision making
analysis (T09) and Fuzzy Logic (T10).
The main research question was: what are the
most appropriate techniques to analyze risks in ERP
projects influenced by SUE? Based on these two
finds, we might state that at least 16 techniques are
actual reported regarding ERP projects. On the other
hand, there were a few techniques such as T07 and
T12 in which both literature review and industry
converged. Given these finds, we can conclude that
there is space in order to match the scientific
researches and academic world with the enterprises’
reality regarding SUE and ERP projects.
Risk Analysis Techniques for ERP Projects based on Seasonal Uncertainty Events
179
The contribution of this work to identify and
summarize all the techniques that related SUE and
ERP projects, thus helping both Industry and
Academic fields to identify, apply and training their
staff and stakeholders on these techniques. As future
researches, to expand to order contexts and countries.
Other identified gap is related to the low usage of
certain techniques, whenever we consider the
Academic or the Industry areas.
REFERENCES
Acebes, F., Pajares, J., Galán, J. M., and López-Paredes, A. 2014.
Exploring the influence of seasonal uncertainty in project
risk management. Procedia - Social and Behavioral
Sciences, 119, 329-338.
Aloini, D., Dulmin, R., and Mininno, V. 2007. Risk management
in ERP project introduction: Review of the literature.
Information & Management., 44(6), 547-567.
Aloini, D., Dulmin, R., and Mininno, V. 2012. Risk assessment
in ERP projects. Information Systems, 37(3), 183-199.
Amid, A., Moalagh, M., and Ravasan, A. Z. 2012. Identification
and classification of ERP critical failure factors in Iranian
industries. Information Systems, 37(3), 227-237.
Creswell, J. W. 2009. Qualitative Methods. In Research Design:
Qualitative, Quantitative, and Mixed Methods Approaches.
Thousand Oaks: SAGE Publications, 183–214.
Dalkey, N., & Helmer, O. 1963. An experimental application of
the Delphi method to the use of experts. Management
science, 9(3), 458-467.
Globerson, S., Zwikael, O. 2002. The impact of the project
manager on project management planning processes.
Project management journal, 33(3), 58-64.
Huang, S.M., Chang, I.C., Li, S.H., & Lin, M.T. 2004. Assessing
risk in ERP projects: identify and prioritize the factors.
Industrial management & data systems, 104(8), 681-688.
IPMA. 2015. Individual competence baseline 4th version.
International Project Management Association.
ISO. 2016. ISO Guide 73:2009. Risk management-Vocabulary.
Retrieved January 15, 2019 from <
https://www.iso.org/standard/44651.html>
Kitchenham, B. et al. 2009. Systematic literature reviews in
software engineering-a systematic literature review.
Information and software technology, 51(1), 7-15.
Mattsson, Stig-Arne. 2010. Inventory control in environments
with seasonal demand in Operations Management Research
3 (3): 138–45. https://doi.org/10.1007/s12063-010-0035-1.
Motwani, J., Subramanian, R., & Gopalakrishna, P. 2005.
Critical factors for successful ERP implementation:
Exploratory findings from four case studies. Computers in
Industry, 56(6), 529-544.
Morettin, P. A., and Bussab, W. O. 2017. Estatística Básica,
Saraiva Educação. Sao Paulo: 8
th
edition.
Nakatsu, R. T. & Iacovou, C. L. 2009. A comparative study of
important risk factors involved in offshore and domestic
outsourcing of software development projects: A two-panel
delphi study. Information & Management, 46(1), 57-68.
Neubarth, R., Guedes, E., de Araújo, E., & Rosini, A. 2016. IT
Governance and Change Management: Impacts on
Increasing Availability on a Financial Institution's Business
Platforms. In: 13th CONTECSI International Conference
on Information Systems and Technology Management.
OGC. 2009. Managing successful projects with PRINCE2. The
Stationery Office.
Passari, A. F. L. 2003. Exploration of atomized data for retail
sales forecasting using neural networks. Dissertation
Thesis, School of Economics, Business and Accounting,
University of São Paulo, São Paulo, Brazil.
PMI. 2017. A guide to the project management body of
knowledge (PMBOK Guide). Project Management Institute,
Newtown Square, Pennsylvania, 6th edition.
Prado, E., Mannini, P., and Barata, A. 2017. Risk Management
Analysis carried out by Third Parties in Enterprise Resource
Planning Systems. In: 14th CONTECSI International
Conference on Information Systems and Technology
Management, p. 2242-2266.
Qi, D., & Zhu, Z. 2012. The knowledge-transfer risk evaluation
model of the execution of ERP project based on AHP. In:
Fifth International Conference on Business Intelligence and
Financial Engineering, p. 636-640.
Rainer, R. K., & Cegielski, C. G. 2012. Introduction to
Information Systems: Enabling and Transforming Business,
4th edition. USA: John Wiley & Sons.
Rajagopal, P. (2002). An innovation - diffusion view of
implementation of enterprise resource planning (ERP)
systems and development of a research model. Information
and Management, 40(2), 87-114.
Schmidt, R C., Lyytinen, K., Keil, M., & Cule, P. E. 2001.
Identifying software project risks: An international Delphi
study. Journal of management information systems, 17(4),
5-36.
Schmidt, R. C. (1997). Managing Delphi surveys using
nonparametric statistical techniques. Decision Sciences,
28(3), 763-774.
Siegel, S., Castellan, N. J., Carmona, S. I. C., Cláudio, D., Silva,
D., and Accorsi, A. (2006). Estatística Não-Paramétrica para
Ciências do Comportamento. Artmed, Porto Alegre, 3th
edition.
Skinner, R., Nelson, R. R., Chin, W. W., & Land, L. 2015. The
Delphi method research strategy in studies of information
systems. Communications of the Association for Information
Systems, 37(1), 31-63.
Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi
method for graduate research. Journal of information
technology education, 6:1.
Stair, R., & G. Reynolds. 2017. Fundamentals of Information
Systems. Boston: Cengage Learning.
Tsai, W. H., Lin, S. J., Lin, W. R., and Liu, J. Y. 2009. The
relationship between planning & control risk and ERP
project success. In: IEEE
International Conference on
Industrial Engineering and Engineering Management,
1835-1839.
ICEIS 2020 - 22nd International Conference on Enterprise Information Systems
180