ERP POST-IMPLEMENTATION TRAINING PROGRAM

ASSESSMENT

Identifying Key Factors that Improve Cognitive Outcomes

Biswadip Ghosh

and Jason J. Woodfork

Department of Computer Information Systems, Metropolitan State College of Denver, Denver, U.S.A.

Keywords: Enterprise Resource Planning, Post-implementation Training, Cognitive Outcomes, Collaborative Training,

ERPSim, Fuzzy Composite Programming.

Abstract: Enterprise Resource Planning (ERP) systems, such as SAP, feature a rich set of integrated business

applications. To maximize the long term benefits from ERP implementations, organizations need to support

end users with effective training during the post-implementation phase. Training programs that build the

end-user’s cognitive skills and business procedural knowledge are particularly important as they allow the

users to understand the broader scope of the ERP system implementation and the strong integration of

multiple business processes and functions. Given the high cost and variety of ERP training programs, there

is a need to create validated models to assess content and benefits of such training programs. Using a field

study of a collaborative, team-based training program with the ERPSim simulation tool, this paper develops

and validates a fuzzy logic model to assess cognitive outcomes. The study finds that training team

characteristics, particularly heterogeneity and cooperation, are most important in achieving higher levels of

cognitive outcome. The results of the study imply that for ERP implementation success, the end-users must

be given suitable training programs that allow them to share and integrate cross functional knowledge.

Moreover, the success of such training programs needs to be periodically measured to assess cognitive

outcomes.

1 INTRODUCTION

Enterprise Resource Planning (ERP) systems consist

of an integrated set of co-operating business

applications that enable an organization to configure,

deploy, use and manage its business resources (e.g.

materials, plants, human resources, capital). These

systems allow an organization to operate cross

functional business processes with shared “master

data” and integrate information across the whole

organization (Davenport, 2004). ERP systems

establish a much tighter level of integration within

an organization, such that the information flow

between functional departments happens in real

time. Prior to ERP system implementation,

departments worked in silos and had longer lag

times to fix transactional data issues. However, after

ERP implementation, they can no longer take

additional time to update functional data entries

before those records start affecting other

departments (Bingi et al., 1999). Due to the rich

integrated functionality and complexity of ERP

systems such as SAP, organizations typically use a

“no customizations” implementation strategy,

whereby the system is implemented off the shelf

with limited changes. The organization then attempts

to adapt their own business processes to match the

embedded logic and “best practices” in the ERP

system (Hirt and Swanson, 2001). Such adaptations

require changes in established work practices that

frustrate the end users (Soh and Sia, 2005); (Willis

and Willis-Brown, 2002). These end-users typically

have very limited visibility to the full scope of the

ERP system implementation (Carr, 2003). Moreover

the training provided to these users is mostly skill-

based, highly procedural and narrow in scope, which

does not allow the typical user to grasp the tight

integration among business departments outside of

their work area and job function (Macris, 2011).

ERP system implementation failures are widely

reported in the IS (Information Systems) literature

which includes the Fox Meyer Drug company’s

bankruptcy and Hershey’s logistics issues (Carr,

Ghosh B. and J. Woodfork J..

ERP POST-IMPLEMENTATION TRAINING PROGRAM ASSESSMENT - Identifying Key Factors that Improve Cognitive Outcomes.

DOI: 10.5220/0003899600050014

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 5-14

ISBN: 978-989-8565-07-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

2002). Published research has associated inadequate

end-user training with ERP implementation

problems (Brown and Vessey, 2003). Conversely,

among 18 factors, ‘top management support’ and

‘training and education’ are among the most

frequently cited as being most critical to the

successful implementation of ERP systems (Ngai et

al., 2008). Post-implementation success is also

improved by building firm specific resources and

capabilities that include human resources (Groeke

and Faley, 2009). The lack of user involvement

exposes the firm to focus on technical issues rather

than the nature of business process flow (Wright and

Wright, 2002). End-user training is one of the most

pervasive methods for developing human resources

within the organization to effectively utilize an

information system (Gupta et al., 2010). Vendor-

provided ERP training can constitute roughly 5% to

50% of ERP Implementation budgets (Scott, 2005).

Current research reports that ERP system users

must gain an understanding of the full impact of the

system through post-implementation training

programs in order to get business benefits from the

system (Chang and Chou, 2011). There are three

targeted goals of most end-user training programs:

(1) skill-based goals (tool procedural) that target the

user’s ability to use the system, (2) cognitive goals

(tool conceptual or business procedural) that focus

on the use of the system to solve business problems

and (3) meta-cognitive goals that focus on building

the individual’s belief regarding their own abilities

with the system (Gupta et al., 2010). The difficulty

with most ERP system training programs is that they

focus on skills based learning, which does not

achieve cognitive outcomes and hence cannot

transfer to real-life, problem-solving contexts

(Macris, 2011). Moreover, the heterogeneous, yet

interdependent interests of ERP systems

deployments on organizations imply that mere skills

based training is inadequate to prepare end users to

use the ERP system, necessitating greater emphasis

on cognitive outcomes through team-based

approaches.

Cognitive training goals focus on the mental

awareness and judgement of the user and builds

business-conceptual “big picture” knowledge that

allows the user to apply the ERP tool to solve

business problems (Gupta et al., 2010). However, it

is difficult to assess the success of such training

programs that focus on cognitive outcomes (Gupta et

al., 2010). Benefits need to be measured over time,

post training, once the user is back on their job.

Published end user training research does not report

any suitable measurement models that can be used

during the training session to make cognitive

outcome assessments, thus creating a gap in the ERP

systems research literature.

1.1 Research Goals

The goals of this research are to develop a model to

assess team based ERP post-implementation training

programs and explore the factors that contribute to

higher cognitive outcomes of such collaborative

training programs.

Using a field study among SAP ERP end-users,

who took part in a collaborative, team-based, post

implementation training session with the ERPSim

simulation tool (Leger, 2006), this study intends to

support the following research goals:

• Build and validate a fuzzy logic multi-criteria

decision making (MCDM) model of post

implementation ERP training that can successfully

predict the level of cognitive outcomes.

• Use the model to rank the participants of a

training session with ERPSim simulation software

by the level of their cognitive outcomes achieved

from the training.

• Among three factors – (i) collaborative training

content and support, (ii) team characteristics and (iii)

individual characteristics, determine which factor(s)

most increase(s) the cognitive outcomes of the

training participants.

2 BACKGROUND THEORY

ERP system users must grasp and integrate cross-

functional knowledge in post implementation

training programs so that they can communicate and

work cooperatively with users in other business

functions (Chang and Chou, 2011). Wang and

Ramiller (2009) report that ERP training programs

must require that members reflect upon their

learning and contribute their experiences,

observations and insights back into the (user)

community’s collective discourse. Since ERP

systems, through tighter integration of business

processes, require users to work together, therefore

post-implementation training can be made more

effective when conducted in team-based

collaborative settings (Uribe et al., 2003).

An example of collaborative team-oriented

training in the ERP domain involves using

simulation games (Foster and Hopkins, 2011).

ERPSim is a simulation based educational tool

developed to help teach the benefits of enterprise

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

integration using a hands on approach (Leger, 2006).

Team based training with ERPSim offers users a

wider exposure to the functionality of the ERP

system, a better sense of the strong integration of

functional areas and a collaborative environment

where users can reflect and share knowledge among

each other (Hustad and Olsen, 2011); (Seethamraju,

2008). Published research has explored the

pedagogical value of using ERPSim in educational

programs at the undergraduate and graduate levels in

higher education and found outcome benefits.

However, research with actual users from real

businesses who have undergone ERPSim training

programs is still sparse.

While the potential benefits of using ERPSim to

conduct collaborative post-implementation training

with end users are promising, the beneficial effects

of ERP training often fades away soon afterwards

(Yu, 2005). A number of prior studies on ERP

implementations suggest that ERP training should be

continuous (Chien and Hu, 2009); (Davenport

1998). ERP training should not be an event that

occurs once and for all (Chien and Hu, 2009). For

example, periodic formal training and regular review

sessions along with online content and support must

be organized and executed so as to help end-users to

have adequate knowledge if new system functions

are added. A model that can assist in assessing the

cognitive outcomes of such sessions and programs is

needed and currently not reported in the research

literature.

Theories that focus on the individual and social

aspects of learning include shared cognition theory

and situated cognition theory (Sharda et al., 2004).

Situated cognition theory stresses that users learn

particular concepts in real-world practical situations,

where those concepts are actually being used.

Simulation games, such as ERPSim, are particularly

useful to immerse the trainee (end users) in a real

world scenario where they are required to use SAP

to run the full business cycle of a manufacturing

company - plan, procure, produce, distribute and

sell. The game requires the users to interact as

suppliers to customers, receiving orders and

fulfilling those orders by planning and acquiring raw

materials and manufacturing and distributing their

products (Foster and Hopkins, 2011). Participants in

the game are grouped into teams and need to

cooperate and share knowledge with each other to

utilize data from SAP ERP to make business

decisions and track their business results. The

simulation game effectively creates the conditions in

which participants jointly experience the complexity

and ambiguity of operating in the real world and

make them apply the skills they are taught to address

various business situations. The participants have the

opportunity to understand the operation of multiple

business processes of a company and see the

integration of these processes in SAP ERP (Leger,

2006). Collaborative training content refers to the

presence of these characteristics of collaboration –

joint work, the need for business problem solving

and reflection and sharing of insights among the

team members (Alavi et al., 1995).This is in line

with the concepts of situated learning theory.

Shared cognition theory focuses on individual

learning within a social situation, allowing for social

interactions that supports the individual’s cognitive

development with help from more capable team

members and peers. All participant brings their own

experience and expertise to share their knowledge

with the team. There is a constant interaction and

collaboration among participants that allows each

individual to develop more improved skills in

solving problems, than if they worked independently

(Sharda et al., 2004). The experience allows each

participant to see the training scenarios from other

student’s perspectives and helps them to create new

meanings and explanations through shared

understanding and practical use. The broader context

of the knowledge gained from training can enhance

their ability to perform specific tasks (Chang and

Chou, 2011). This experience also creates soft skills

such as communication and negotiation skills.

Group theories suggest that many factors can

influence the outcome of group-based training. This

includes group characteristics, such as composition

(level of homogeneity and heterogeneity), amount of

group cooperation and the nature of group

communications. Group norms and beliefs and trust

are particularly important for effective team work in

the training setting (Sharda et al., 2004). The

repeated interaction between participants in the

training program creates a set of norms, trust and

mutual understanding that bind the participants

together and facilitate better interactions during

training as well as post training (Chang and Chou,

2011). The knowledge sharing and repeated

interactions during collaborative ERP user training

promote greater cooperation, bridging gaps in

understanding and increased cognitive learning

outcomes (Chang and Chou, 2011). Along with

individual characteristics, such as motivation,

interest and learning style, the group qualities can

impact outcomes of training programs. In team

based training programs, team members from

different functional areas work together and allow

team members to develop diverse knowledge and

ERPPOST-IMPLEMENTATIONTRAININGPROGRAMASSESSMENT-IdentifyingKeyFactorsthatImprove

CognitiveOutcomes

build broader perspectives that span business

functions (Seethamraju, 2008). For successful

cognitive outcomes, team characteristics must be

optimized along with training content and delivery

structures.

3 RESEARCH MODEL AND

CONSTRUCTS

The research model is displayed in Figure 1. The

research constructs are defined in the following

subsections.

Figure 1: Research model.

3.1 Cognitive Outcomes (CO)

Cognitive outcomes (CO) focus on the mental

awareness and judgements of the end-user and the

levels of application of acquired knowledge towards

operating business functions (Gupta, et.al, 2010).

Cognitive outcomes also include the transfer of

learning to new situations and understanding the

interactions of multiple parts of a complex system

such as SAP ERP. Using the ERPSim game, the

end-users are allowed the opportunity to handle the

complexity of running a real company using the

SAP ERP system. Each team makes decisions on

production, distribution and marketing variables.

Participants in the training manipulate product price,

product composition, marketing expenses and

distribution channels to maximize the profitability of

their simulated firm across two operating periods.

The ERPSim simulator stores each team’s financial

performance by capturing the net profit for the

simulation period. In this research study, the net

profit for the overall simulation duration is used as

the variable to measure cognitive outcomes. The

team with the highest profit has the highest cognitive

outcome and lowest profit has the lowest cognitive

outcome. Since, the net profit represents the most

effective way the “business” was operated by the

team hence this measure is a suitable measure to

represent the business procedural knowledge

attained by the team members from the end-user

training. The net profit value has also been used in

prior field research with ERPSim (Foster and

Hopkins, 2011).

3.2 Collaborative Training Content

(CTC) and Training Structures

(TS)

Collaborative training content (CTC) refers to

instructional methods that encourage students to

work together to accomplish shared goals, beneficial

to all (Alavi et al., 1995); (Leidner and Jarvenpaa,

1995); (Gupta et al., 2010). Learning from peers is

important component in collaborative training as

“peers contribute to task orientation, persistence

and motivation to achieve.” ERP system users must

grasp integrative knowledge in training programs so

that they can communicate and cooperative closely

with users in other business functions (Chang and

Chou, 2011). The interactions within the team

setting allow the members to interact, exchange

knowledge and fill in gaps in their understanding of

the SAP ERP system. Collaborative training content

refers to the presence of these characteristics of

collaboration – joint work, the need for business

problem solving and reflection and sharing of

insights among the team members (Alavi et al.,

1995). Collaborative environments foster

discussions and knowledge sharing. That allows

end-users to fill gaps in their understanding and

builds knowledge about how an SAP ERP system

integrates various functional departments in the

organization. “Soft skills” are also developed that

allow members to learn collective beliefs and norms

that help them develop confidence and knowledge in

solving future business problems.

Training structures (TS) refer to the scaffolds

that support the delivery of the training content. Also

referred to as appropriation support (Gupta et al.,

2010), they include the rules, resources and methods

that support the elements of the collaborative

training session. For this research study, the training

structures include level of detail in the instructions

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

given to participants, the guidance provided by the

facilitator and the nature of the facilities and

equipment used in the training session.

3.3 Team Characteristics (TC)

In collaborative learning, the team members share

goals and learn together by working jointly and

solving the problems posed in the training. The team

composition plays a critical role in the learning

environment through the size and heterogeneity of

the team. The more diversity in the team, there is

more likely to be integration of knowledge from

multiple functional areas. Research has shown that

when team members are from differing

backgrounds, the discussions and knowledge sharing

is more intense leading to create group decisions

(Sharda et al., 2004). In teams, team members from

different functional areas allow teams to develop

diverse knowledge and build broader perspectives

than span business functions (Seethamraju, 2008).

Team characteristics (TC) is measured using

questions on whether team members came from

different functional areas (heterogeneity), and the

nature of cooperation and the level of dialog

achieved within the team. Greater cooperation and

dialog among a diverse team allows them to build

identification giving them a broader vision of the

ERP implementation scope and also creates norms

that help further enhance cross functional knowledge

sharing (Chang and Chou, 2011).

3.4 Individual Characteristics (IC)

People prefer learning methods based on their

specific learning styles (Nogura and Watson, 2004).

Individual differences influence the formation of

mental models, which affects the training process.

“States” are general influences on performance that

vary over time and include temporal factors such as

motivation level and interest level (Bostrom et al.,

1990). “Traits” are static aspects of information

processing affecting a broad range of outcomes.

Cognitive traits refer to learning styles such as a

preference for procedural or abstract knowledge and

an exploratory or reflective approach to instructional

content delivery format (Bostrom et al., 1990);

(Nogura and Watson, 2004). For this research study,

the Individual characteristics (IC) variable is

measured using motivation and interest as states and

individual learning style as traits.

4 METHODOLOGY

In the ERPSim training program, the simulation

system is coupled with ERP server access to deliver

the entire training content. The simulation consists

of a make-to-order manufacturing and distribution

scenario. A total of 16 participants were divided into

8 teams each team with 2 members. The teams were

asked to utilize the SAP ERP system to meet

demand for a variety of custom products. Each team

sets production plans, distribution plans, marketing

plans and pricing levels to produce and sell their

product through multiple distribution channels. The

team performance was measured by using a team

letter (A – H) and their net profit was recorded from

the ERPSim simulator. The net profit was used to

rank team performance. In addition, the participants

were asked to fill out a survey, which had 12 items.

Each item was measured on a 1-5 Liekert scale. The

survey items were closely worded to the definition

of the constructs and each construct (TCC, TS, TC

and IC) was measured with 3 items on the survey.

Participants rated each item on a 5 points scale

consisting of strongly disagree, disagree, neutral,

agree and strongly agree.

The research methodology consisted of using

survey data from the training participants to create a

training scenario for each team in Fuzzy Decimaker

software to rank each team based on the four factors

– team characteristics, collaborative training content,

training structure and individual characteristics. The

ranking from the Fuzzy software was compared to

the ranking of the teams using the net profit measure

from the ERPSim simulation tool. To validate the

Fuzzy composite programming model, the team

rankings from the model must match the ranking

using the net profit measure.

For this research, 8 groups were ranked using net

profit from the business simulation and compared

against the four factors and their corresponding sub

factors:

• Team Characteristics (TC). TC had three sub

factors – team heterogeneity, team dialog and team

cooperation.

• Collaborative Training Content (CTC). CTC had

three sub factors – problem solving needed, joint

work needed and the reflection and sharing of

insights.

• Training Structure (TS). TS had three sub factors

– Facilities and Equipment, Instructions and

Guidance offered by the facilitator.

• Individual characteristics (IC). IC had three sub

factors – Individual’s motivation to participate in the

ERPPOST-IMPLEMENTATIONTRAININGPROGRAMASSESSMENT-IdentifyingKeyFactorsthatImprove

CognitiveOutcomes

training, the individual’s interest in the content and

learning style.

The demographics of the 16 subjects of this research

study, who participated in the ERPSim training

session, are documented in Table 1. As seen in Table

1 the average professional experience and their years

of SAP usage of the participants were 7.6 years and

2.4 years respectively. More than half of the

participants have job responsibilities that are

operational in nature and their functional areas and

industry of work varied as seen in Table 1.

Table 1: Demographics.

Years of Exp

Average: 7.6 years (Min: 0; Max: 14)

SAP Usage

Average: 2.4 years (Min: 0; Max: 14)

Job Responsibility

Operational (10); Managerial (5), Strategic

(1)

Functional Area

Sales/marketing (5); Service (3), IT (2),

Accounting/Finance (4), Other (2)

Industry

Manufacturing (7); Retail (4); Service (3);

Finance (1); Other (1)

Gender

Male (9); Female (7)

Analysing the impact of different factors on the

cognitive outcomes of ERPSim training using

statistical techniques is difficult, because the survey

data collected from the training session has multiple

issues:

• The measurement constructs are correlated (e.g.

collaborative training content and team

characteristics) and hence can also have conflicting

values in the different items.

• The measurements often scatter around a certain

range so statistical summarization of the data can

lose information.

• The need to analyze mixed data - both qualitative

data (survey data) and quantitative data (net profit

from the ERPSim simulation tool).

Hence there is no effective way to comprehensively

assess and rank different groups based on a multi-

layered criteria using statistical analysis. However,

there is a need for a comprehensive measure of

cognitive outcomes of ERP training. A Multiple

Criteria Decision Making (MCDM) tool can be used

based on fuzzy logic in evaluating cognitive

outcomes from the SAP training to resolve the

problems in assessment under this conflicting,

uncertain and hierarchical data situation.

5 FUZZY COMPOSITE

PROGRAMMING MODEL

5.1 Fuzzy Composite Index

FCP is one of MCDM techniques, which can handle

mixed indicator data (quantitative and qualitative),

and also work with conflicting, uncertain and

hierarchical criteria. FCP methodology was

developed by Bardossy and Duckstein (1992). There

have been a lot of successful applications of FCP in

Information Systems literature including ERP

Systems research (Onut and Efendigil, 2010). The

indexes are normalized using the best and worst

basic indicator values that are described by the

following equation (Lee et al., 1992)

−+

−

ijij

(When

f is best)

(1)

−+

−

ijij

(When

−

is best)

(2)

FCP is based on a Fuzzy Composite Index (FCI).

The equation for the Fuzzy Composite Index is:

∑

ijijj

}{

(3)

Where, Lj is Fuzzy Composite Index for the B+1

level group j of B level indicators;

• w

is weight of B level indicators in group j;

• p

is balancing factors among indicators for

group j;

• f

is the best value of i

fuzzy indicators for

group j;

• f

is the worst value of i

fuzzy indicators for

group j;

• f

is the value of i

fuzzy indicators for group j.

The final fuzzy composite index, which is used for

ranking, is obtained by calculating the FCI from

basic level to top level.

The weight parameters for indicators at different

levels (w

) are established based on the degree of

importance that decision makers feel each indicator

has relative to other indicators of the same group

(Bardossy and Duckstein, 1992).

The balancing factors (p

) reflect the importance

of maximal deviations between indicators in the

same group, and determine the degree of substitution

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

between indicators of the same group. Low

balancing factors (equal to 1) are used for a high

level of allowable substitution. High balancing

factors (equal to 3) are used for minimal substitution

(Bardossy and Duckstein, 1992). The best value (f

)

stands for the maximum possible value of the

indicator, and the worst value (f

) stands for the

minimum possible value of indicator.

5.2 FuzzyDeciMaker

The FuzzyDeciMaker tool was developed by the

Civil Engineering Department of the University of

Nebraska at Lincoln. It is a software tool to

implement FCP functions, which supports building

tree data structure, inputting data, calculating the

Fuzzy Composite Index for different levels and

ranking different scenarios. The indicators in the

measurement of outcomes were based on using data

from the SAP ERPSim training session and collected

using the participant survey.

6 ANALYSIS AND RESULTS

6.1 Fuzzy Classification using

FuzzyDeciMaker Tool

Since substitution is allowed for all indicators

therefore, the balancing factors for all indicators are

set to 1.

Table 2: Fuzzy model values.

Indicators TC CTC TS IC

Weight 0.25 0.25 0.25 0.25

Balancing factor 1 1 1 1

Best Value 5 5 5 5

Worst Value 1 1 1 1

6.2 Assessment Results

The ranking of the eight teams and the final FCI

values are shown at Table 3. The teams ranking

based on net profits is also shown in Table 3.

From Table 3, we can see the comprehensive

assessment results of the training outcomes for the

eight teams. Among these eight teams, H has the

best performance, while E has the worst

performance based on the net profit from the

business simulation game. The fuzzy Indicator of

each team is also shown in Table 3 and team H has

the highest FCP index value and team E has the

lowest FCP index value. Table 3 shows that the

rankings from the fuzzy analysis corresponds closely

with the rankings using net profit with only one

positional error (rank of teams F and G are

interchanged).

Table 3: FuzzyDeciMaker assessment results.

TEAM A B C

FCP Index

0.667

0.83 0.78825

Cognitive

Outcome Rank

(Net Profit)

($8K)

($58K)

($52K)

TEAM D E

FCP Index

0.75425 0.6085 0.70025

Cognitive

Outcome Rank

(Net Profit)

($42K)

($3K)

($23K)

TEAM G H

FCP Index

0.7 0.8835

Cognitive

Outcome Rank

(Net Profit)

($27K)

($64K)

6.3 Analysis of Second and Third Level

Indicators

To investigate what indicators contributed most to

the rankings, the FCI and ranking of different levels

of indicators in the hierarchical model were

analyzed.

Table 4: Second level indicators.

TC CTC TS IC

FCI # FCI # FCI # FCI #

1.0

0.75

0.835

0.937

0.585

0.835

0.917

0.812

0.667

0.835

0.876

0.812

0.752

0.832

0.791

0.375

0.665

0.417

0.835

0.75

0.917

0.915

0.626

0.75

0.585

0.75

0.625

0.585

0.747

Tables 4 and 5 show the second indicator

rankings and the third level indicator rankings for

the team characteristics factor compared with the

final fuzzy composite ranking (FCP Final Rank from

Table 3).

From Table 4, we can see that the final ranking

of cognitive training outcome based on Fuzzy

Analysis is closest to that based on Team

characteristics. The ranking based on this factor,

shows that only team G is out of place in the ranking

order. For example, for teams H and B are ranked as

first and second, respectively by both the overall FCI

ERPPOST-IMPLEMENTATIONTRAININGPROGRAMASSESSMENT-IdentifyingKeyFactorsthatImprove

CognitiveOutcomes

score and the TCC score. The overall FCI score and

the TCC score also correspond on the least effective

teams, F, A and E. The above congruence in the

scores for the top two and bottom three performing

teams indicate that team characteristics plays the

most important role on assessing cognitive outcomes

with ERP Simulation training with the fuzzy model.

Other second level indicators (Collaborative

training content, Training Structures and Individual

characteristics) have less impact on measuring the

cognitive outcomes in the fuzzy model. For each of

those dimensions, there were at least 5 mismatches

with the overall ranking (Table 4).

Table 5: Third level indicators for team characteristics.

Heterogeneity Cooperation Team Dialog

1 1 1

2 2 2

3 3 3

4 4 4

5 5 7

8 8 8

6 6 5

7 7 6

Under team characteristics factor, the ranking

based on heterogeneity and cooperation are the

closest to that based on the TC indicator and the

final ranking (Table 5). So, team heterogeneity and

cooperation plays the most important role among

third order factors inside team characteristics in

assessing cognitive outcome in the fuzzy model.

7 CONCLUSIONS

We have developed and validated a model to assess

the effectiveness of post-implementation training

programs in building cognitive skills and business

procedural knowledge. The model uses survey items

from four constructs – team characteristics,

individual characteristics, collaborative training

content and training structures and was successful in

ranking teams on outcomes, which was validated

against the net profit values from the simulator. It is

seen that to avoid personal differences amongst

departments and employees, a collaborative training

program that brings together personnel from diverse

functions and business areas needs to be used to

install the foundation needed for long term success.

As a result of team work addressing cross functional

duties during training, gaps in understanding are

filled to bridge the operating silos that still exist

even after ERP implementation. The close

cooperation during such training programs also

facilitates mass cooperation and allows greater

information flow. Within teams, heterogeneity and

their cooperation made substantial differences in the

level of cognitive outcomes of the training program.

Whereas, participation in team based training, builds

understanding of the organizational culture and

allows managers to address cross functionalities

which enable heterogeneity of existing systems

(Scott and Vessey, 2002). This allows all business

duties to function and synchronize in real-time.

When operating at the mature phase of an ERP

system implementation, a seamless workflow of data

is realized (Stephenson and Sage, 2007). In addition,

Stephenson and Sage (2007) point out the

importance of organizations and the need of

supporting the ever changing evolution of ERP

systems, and knowledge resides in all members of an

organization hence the term “Organizational

Capacity.” Realizing the importance of individual

characteristics and the ability to measure what

supports team members to collaborate in general

ensures customer satisfaction, and profit

sustainability. Benefits of ERP system are profits but

proper oversight is duly important. Other research

indicates that firms with ERP systems are less likely

to experience Internal Control Weakness (ICW),

(Morris, 2011). The proper training program is a

critical factor when considering the individual, team,

and organization that deals with consumers. Training

is the way to set the tone from the top down for any

ERP system implementation and post

implementation management.

7.1 Contributions to Research and

Practice

The major contribution of this research is to develop

and validate a fuzzy logic based multi criteria

measurement model to assess the factors present in a

ERP post-implementation training program to

develop cognitive outcomes. The level of cognitive

outcomes of ERP post implementation training

programs predicted from the fuzzy logic model was

validated against business profitability results from

the ERPSim simulator. The team composition of the

participants in the training program was seen to be

the most important to achieve increased outcomes

from the training. To build more cross functional

knowledge, the team of training participants needs to

have heterogeneity and have strong cooperation

among themselves. The results support a repeat

study with a larger number of teams and each team

having more members. The model can be used to

guide future research and applied to measure other

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

training programs. The results of the study can be

used in IS practice to implement and support ERP

systems more effectively. It is clear that better group

composition and dynamics must be orchestrated

during ERP post implementation training programs

so as to boost training outcomes. Cognitive

outcomes require the end users to build a strong

understanding of the scope of the ERP system and

understand how their function touches other

functions and by designing better group interactions,

the broader business procedural knowledge can be

enhanced among ERP system users.

REFERENCES

Alavi, M., Wheeler, B.C. and Valacich, J.S. 1995. Using

IT to Reengineer Business Education: An Exploratory

Investigation of Collaborative Tele Learning, MIS

Quarterly 19(3), pp. 293-312.

Bardossy, A. and Duckstein, L. 1992. Analysis of a karstic

aquifer management by fuzzy composite

programming, Water Resources Bulletin (28:1), 1992,

pp. 63-73.

Bingi, P., Sharma, M.K., Godla, J.K. 1999. Critical Issues

Affecting An ERP Implementation, Information

Systems Management, Summer, pp. 7-14.

Bostrom, R.P., Olfman, L. and Sein, M.K. 1990. The

Importance of Learning Style in End-User Training,

MIS Quarterly 14(1), pp. 101-119.

Brown, C.V. and Vessey, I., 2003. Leveraging the ERP

Learnings for the Next Wave of Enterprise Systems

Projects, MIS Quarterly Executive, 2(1), pp. 65-77.

Carr, D. F. 2002. Hershey’s sweet victory. Retrieved

November, 2011, from

http://www.baselinemag.com/article2/0,3959,760615,

00.asp

Carr, N. 2003. IT doesn't matter. Harvard Business

Review, 5-12.

Chang, H.H. and Chou, H.W. 2011. Drivers and effects of

Enterprise Resource planning post-implementation

learning, Behaviour and Information Technology 30

(2), March-April, pp. 251-259.

Chien, S. W., and Hu, C. 2009. The moderating effect of

employee computer self-efficacy on the relationship

between ERP competence constructs and ERP

effectiveness. Journal of Electronic Commerce in

Organizations, 7(3), pp. 65-85.

Davenport, T. 1998. Putting the enterprise in the enterprise

systems. Harvard Business Review, 76(4), 121-131.

Davenport, T., Harris, J., and Cantrell, S. (2004).

Enterprise Systems and Ongoing Process Change.

Business Process Management Journal 10(1), 16-26.

Foster, S. and Hopkins, J. 2011. ERP Simulation Game:

Establishing Engagement, Collaboration and Learning,

Proceedings of the 15

Asia Conference on

Information Systems, Brisbane, Australia, July 8-11.

Goeke, R. J. and Faley, R. H., 2009. Do SAP Successes

Outperform Themselves and Their Competitors?

Communications of the ACM, 52(10), pp. 113-117.

Gupta, S., Bostrom, R. P. and Huber, M., 2010. End-User

Training Methods: What we know, Need to Know,

The DATABASE for Advances in Information Systems

41(4), pp. 9-39.

Hirt, S., and Swanson, E., 2001. Emergent maintenance of

ERP: New roles and relationships. Journal of Software

Maintenance and Evolution 13, 373-397.

Hustad, E. and Olsen, D. H., 2011. Teaching Enterprise

Systems in Higher Education: The Learning Context

Triangle, Proceedings of the 8

International

Conference on Enterprise Systems, Accounting and

Logistics, July 11-12, 2011, Thassos Island, Greece.

Lee, Y. M. Dahab, M. and I. Bogardi, I., 1992. Nitrate risk

assessment under uncertainty, Journal of Water

Resources Planning and management (118:2), 1992,

pp 151-165

Marcris, A. M., 2011. Enhancing Enterprise Resource

Planning users’ understanding through ontology-based

training. Computers in Human Behavior. 27(4), pp.

1450-1459.

Morris, J. J., 2011. The Impact of Enterprise Resource

Planning (ERP) Systems on the Effectiveness of

Internal Controls over Financial Reporting. Journal of

Information Systems, 25(1), pp.129-157.

Leidner, D. E. and Jarvenpaa, S. L., 1995. The use of

Information Technology to enhance Management

School education: A theoretical view. MIS Quarterly

19(3), pp. 265-291.

Leger, P., 2006. Using a Simulation Game Approach to

Teach Enterprise Resource Planning Concepts,

Journal of Information Systems Education 17(4), pp.

441-447.

Ngai, E. W. T., Law, C. C. H., and Wat, F. K. T., (2008).

Examining the critical success factors in the adoption

of enterprise resource planning. Computers in Industry

59(6), 548-564.

Noguera, J. H. and Watson, E. F., 2004. Effectiveness of

Using an Enterprise System to teach process centered

concepts in business education. Journal of Enterprise

Information Management 17(1), pp. 56-74.

Onut, S. and Efendigil, T., 2010. A Theoretical Model

Design for ERP Software Selection Process under the

constraints of cost and quality: A fuzzy approach,

Journal of Intelligent and Fuzzy Systems (21), pp. 365-

378.

Scott, J. E., 2005. Post Implementation Usability of ERP

Training Manuals: The User’s Perspective,

Information Systems Management (Spring), pp. 67-77.

Scott. J. E. and Vessey, I., 2002. Managing Risks in

Enterprise Systems Implementations, Communications

of the ACM 45(4), pp. 74-81.

Seethamraju, R. 2008. Enhancing Student Learning

through ERP Business Simulation Game, Proceedings

of the AIS SIG-ED IAIM 2008 Conference,

Sharda, R., Romano Jr., N. C., Lucca, J. A., Weiser, M.,

Scheets, G., Chung, J. M. and Sleezer, C. M., 2004.

Foundation for the Study of Computer-Supported

ERPPOST-IMPLEMENTATIONTRAININGPROGRAMASSESSMENT-IdentifyingKeyFactorsthatImprove

CognitiveOutcomes

Collaborative Learning Requiring Immersive

Presence, Journal of Management Information

Systems 20(4), pp. 31-63.

Soh, C., and Sia, S. K., 2005. The challenges of

implementing 'vanilla' versions of enterprise systems.

MIS Quarterly Executive, 4(3), 373-384.

Stephenson, S. V. and Sage, A. P., 2007. Architecting for

Enterprise Resource Planning, Information Knowledge

Systems Management (6), pp. 81-121.

Uribe, D., Klein, J. D. and Sullivan, H., 2003. The Effect

of Computer-Mediated Collaborative Learning on

Solving Ill-Defined Problems, Enterprise Technology

Research and Development 51 (1), pp. 5-16.

Wang, P. and Ramiller, N. C., 2009. Community Learning

in Information Technology Innovation, MIS Quarterly

33(4), pp. 709-734.

Willis, T., and Willis-Brown, A. 2002. Extending the

value of ERP. Industrial Management and Data

Systems, 102(1), 35-39.

Wright, S. and Wright A. M. 2002. Information System

Assurance for Enterprise Resource Planning Systems:

Unique Risk Considerations, Journal of Information

Systems 16 (Suppl.), pp.99-113.

APPENDIX

Survey Items by each study construct:

Collaborative Training Content (CTC):

1. The training required joint work within my team.

2. The training required problem solving using the

SAP system.

3. The training required reflection and sharing my

insights with my team members.

Training Structures (TS):

1. The training materials provided me with detailed

instructions on what to do.

2. I am satisfied with the guidance provided by the

facilitator during the training

3. The facilities and equipment used in the training

session were excellent

Team Characteristics (TC):

1. My team member(s) came from different

functional areas than me.

2. My team member(s) engaged in lots of dialog at

each step of the simulation exercise

3. There was a lot of cooperation and teamwork

among my team member(s)

Individual Characteristics (IC):

1. I was motivated to learn as much as I can from

this training class

2. I was very interested to take this training class

3. When I learn ….

I like to deal with my feelings

I like to think about ideas

I like doing things

I like to listen and watch

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation