IDENTIFYING USABILITY ISSUES WITH AN ERP

IMPLEMENTATION

Heikki Topi, Wendy Lucas, Tamara Babaian

Bentley College, Computer Information Systems Department, Waltham, MA, 02452, USA

Keywords: usability, ERP, collaboration, enterprise system, user

Abstract: Enterprise Resource Planning (ERP) sy

stems hold great promise for integrating business processes and have

proven their worth in a variety of organizations. Yet the gains that they have enabled in terms of increased

productivity and cost savings are often achieved in the face of daunting usability problems. While one

frequently hears anecdotes about the difficulties involved in using ERP systems, there is little

documentation of the types of problems typically faced by users. The purpose of this study is to begin

addressing this gap by categorizing and describing the usability issues encountered by one division of a

Fortune 500 company in the first years of its large-scale ERP implementation. This study also demonstrates

the promise of using collaboration theory to evaluate usability characteristics of existing systems and to

design new systems. Given the impressive results already achieved by some corporations with these

systems, imagine how much more would be possible if understanding how to use them weren’t such an

overwhelming task.

1 INTRODUCTION

Enterprise Resource Planning (ERP) systems and

other large scale enterprise systems (such as Supply

Chain Management, Customer Resource Manage-

ment, etc.) are widely used in virtually all industries.

In the 1990s, enterprise systems became a core part

of the information systems architecture in a very

large number of corporations (Davenport, 1998),

even though many implementations have either

failed entirely or have not reached the goals set for

them (Davenport, 1998; Chen 2001; Scott & Vessey,

2002). Not surprisingly, enterprise systems have

been studied very actively, particularly by the

information systems community (Esteves & Pastor,

2001). Many of these studies have focused on the

factors affecting the success of ERP implementa-

tions (Sarker & Lee, 2003; Grossman & Walsh,

2004; Siau, 2004).

One of the areas that has not been widely studied

in either acade

mic or practitioner literature is the

usability of ERP systems and other large-scale enter-

prise systems. While usability problems may not

lead directly to a large-scale failure, they can

interfere with an individual’s or workgroup’s

productivity, making it harder for users to achieve

their goals as effectively or efficiently as is

desirable. They may also make system acceptance a

more difficult and lengthy process.

The long-term goal of our research is to develop

a set o

f design principles, the application of which

leads to improvements in the usability of ERP

systems. While these principles would be applicable

also to other large-scale systems, we are focusing on

ERP systems in part because of their aforementioned

widespread usage. Our contention is that improved

ERP usability can be achieved by designing the

system with collaboration as the model for user-

system interaction. The term collaboration should

not be interpreted literally: a computer system does

not have a free will and current technology is not

capable of creating a system with the capacity to

reason that is akin to a human collaborator. How-

ever, the metaphor of a system-partner, in which the

goals of the user are understood and supported by

the system, can be the framework for the design,

leading to system interfaces and interactions that are

more supportive of the user.

The user-system collaboration metaphor is

especially relevant i

n the enterprise-wide context,

where the system possesses a vast amount of

organizational data and is responsible for automating

a large number of business processes. One of the

central principles of collaborative behaviour

(Bratman, 1992) is the commitment to mutual

128

Topi H., Lucas W. and Babaian T. (2005).

IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION.

In Proceedings of the Seventh International Conference on Enterprise Information Systems, pages 128-133

DOI: 10.5220/0002522301280133

 SciTePress

support, which requires that when either party

recognizes that the other one needs help to complete

a subtask successfully, it will provide that help

(assuming it is capable of doing so). Another central

tenet is the commitment to joint activity, which

requires that both parties recognize and commit to

that activity. To make this commitment, the parties

must be aware of the context surrounding their

collaboration. A third principle is that of mutual

responsiveness, in which each party adjusts its

behaviour based on the behaviour of the other and

guided by commitment to the joint activity.

In this paper, we report on the findings of our

field study on an ERP system in one division of a

Fortune 500 company. Gaining insight into the

problems that users experience with existing ERP

systems and understanding the tasks that those

systems are intended to support are critical first steps

in working toward improved system usability. Our

findings confirm what can be surmised from the

large bodies of anecdotal evidence; namely, that

poor usability characteristics, such as unnecessarily

complex tasks and inadequate system support in

response to user errors, impact acceptance, usage,

and the usefulness of ERP systems, even in

successful implementations. These findings augment

the results of a laboratory comparison evaluating the

usability of a large number of ERP systems that was

conducted by Forrester Research (Gilbert, 2003).

None of these problems are new: textbooks in

systems analysis and design have warned against

them since at least the 1980s. The novelty of our

approach is in considering these flaws in light of

collaboration theory. Our study proposes that the

usability problems that greatly affect users’

performance can be viewed as examples of non-

collaborative behaviour by the system. Rather than

“standing" idly or silently by when the data in its

possession could help the user diagnose and fix a

problem, such as missing information on an invoice,

the system must work on the user’s behalf as a

collaborative partner.

The contributions made by this paper are as

follows:

• It is the first attempt to focus on usability prob-

lems experienced by users of a large scale ERP

implementation. This study was conducted in a

division of a Fortune 500 company that had

been using its ERP system for two and a half

years, beginning with its launch in Summer

2001.

• We provide a classification of usability prob-

lems encountered by users and show that these

flaws can be viewed as examples of non-collab-

orative behaviour by the system, thereby

providing strong evidence in support of our

contention that collaboration theory is a suitable

conceptual framework for usability design and

evaluation in the ERP context.

This work is part of a larger research project in

which we are also developing prototypes whose

design is driven by the central principles of collab-

orative behaviour (Babaian et al. 2004).

2 RESEARCH METHODOLOGY &

CONTEXT

The data used in this study was collected in ten in-

depth interviews with nine ERP users and one non-

user. The lengths of the interviews varied from 20

minutes to 90 minutes, both of which are outliers;

the length of most interviews was about an hour.

The interviews were conducted on the premises of

the employer of our interviewees, i.e., the organi-

zation that had deployed the ERP system of interest.

The interviews were semi-structured, starting with a

set of predefined questions (available from the

authors). The analysis of the data was performed on

the interview transcripts, with the interviewers’

notes from the interviews used as supporting

evidence.

To maintain the confidentiality of the target

organization, we do not describe it or the system at a

detailed level. At a general level, the system is used

to manage a challenging post purchase maintenance

process for a large and complex engineering

product. This ERP system is in use in five different

locations worldwide, including our target business

unit, where it has been used since 2001.

The respondents in the study were all employed

by the same organization and, with the exception of

the one non-user, were users of the same ERP

system. The respondents represent a variety of orga-

nizational roles, ranging from shop floor operational

workers to upper middle management. Even though

the roles vary, all of the respondents need the system

(or data generated from the system in the case of the

non-user) to fulfil their daily tasks within the

organization.

3 ANALYSIS OF THE RESULTS

The analysis of the interview data was based on the

transcripts. The analysis process began with the

creation of a classification structure derived from the

observations made during the interview process and

from an initial analysis of the transcripts. A research

assistant then classified the entire interview material

at a detailed level to identify usability problems; if

necessary, new categories were added to the

IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION

129

structure. The results of the research assistant’s work

served as the foundation from which the authors

completed the final usability problem categorization

presented below. The remainder of this section will

discuss the identified usability problem categories

and their significance from the perspective of

collaboration theory.

A qualitative analysis of the interview

transcripts led to the following categories of

usability problems:

• Identification of and access to the correct

functionality

• Transaction execution support

• System output limitations

• Support in error situations

• Terminology problems

• Overall system complexity

Following are descriptions and examples by cate-

gory of the actual problems encountered by ERP

system users.

3.1 Identification of and access to the

correct functionality

Several users noted that finding specific

functionality quickly within the system sometimes

required an unreasonable amount of effort. A simple

example of this is navigation; that is, finding a

specific transaction. Navigation problems seemed to

be typical, as illustrated by the following comments

of users, at least one of whom has extensive

knowledge and experience with multiple system

modules:

But it’s also very intensive, cumbersome; there’s

multiple transactions that unfortunately is [sic]

not linked to basically a central menu area to

walk you through. You have to kind of know what

the transaction codes are and how to execute

them.

Or maybe we have some menus, but presently it

may take us four, five or six routes to get us to

basically one screen. I don’t always see the links.

The basic problem with navigation was that very

little help was provided to the users by the system

for determining the correct transaction screens and

how they could be accessed most directly. The

problem was most severe for those users who were

not very familiar with the system and the screens

needed to perform a specific business transaction.

A more significant problem was the users’ diffi-

culty in understanding and/or remembering which

set of actions was necessary for completing a spe-

cific business process. It is very typical that the user

of such a system has to perform a number of ERP

transactions in order to complete one specific

business process. Since the ERP system in our target

organization was not designed to be aware of the

business process the user was attempting to achieve,

it was unable to guide the user through it, as

indicated by the following comment:

There's nothing that says okay, I'm here. Act on

me now. There's nothing that exists. … No,

[there’s a ] transaction for ‘A’, a transaction for

‘B’, and a transaction for ‘C’ and nothing links

them. Someone has to manually do each one of

those.

After a while, the users had memorized the most

common sequences, but it took a significant amount

of time to get to that point. In some cases, the

organization decided that understanding the correct

ordering of ERP transactions to implement a

particular business process was important enough to

document on laminated “cheat sheets,” as noted in

the following quote:

And even the training resulted in some sheets

being made out with the laminated plastic, with

walk through menus for [a specific employee

type], what to process, why you would process it

and things like that, that were very beneficial.

Throughout the interviews, the lack of system

support for understanding the business processes

that mapped to the ERP tasks was identified as one

of the most significant problems. One respondent

expressed the importance of understanding and

focusing on the business processes, rather than the

system transactions:

But I can't stress enough the fact that the

transactions are the easy part; it's the business

that's the important piece.

When evaluated in the context of collaboration

theory, the users’ difficulties with navigation and

understanding the correct transaction sequence for

particular business processes suggest that the system

design did not properly capture the composition of

many high-level business tasks for which the system

is used. As a result, the system cannot guide a user

through the steps of a composite business process,

and the user was left with the burden of searching

for and memorizing the correct sequence of opera-

tions. This indicates that the system was designed

without commitment to joint activity: the system

cannot be committed to the user’s goals if it has not

been designed to be aware of them.

ICEIS 2005 - HUMAN-COMPUTER INTERACTION

130

3.2 Transaction execution problems

Many of the usability problems that the users faced

were related to the completion of system trans-

actions. Users often found the transaction interfaces

unduly complex. This was evidenced by the cheat

sheets developed in-house and used by the

employees even after the training and by the

customized data entry forms created to compress a

transaction into fewer screens, thereby eliminating a

lot of optional entry items.

Some transaction level problems were quite

well-known, such as the one noted by this user’s

question:

…why do I have to keep entering the same data

over and over?

A thoughtful collaborator would recognize the

need for this repetitive data and would automatically

enter it, as is typically done by many modern

systems. Another, very experienced user expressed

the same problem in different words:

Well, I mean, we're so used to copying and

pasting. … In some cases, it [the ERP system]

remembers and will carry over to some of the

screens, but not in all cases.

One of basic strengths of a computer system is

in automating mundane data entry tasks, yet the ERP

system left this job to the user. The problems related

to the entry of redundant data are well known

(increased errors, inconsistency between system

components, increased storage needs, etc.). Given

that the repetition of field values between different

screens is derived from the underlying business

processes, the system should automatically provide

the repeated data in subsequent screens rather than

requiring the user to either reenter or copy and paste

it.

These transaction level problems indicate that

the system lacked the capability of modifying its

behavior based on the user’s actions and was not

supporting the user in situations where help was

clearly needed.

3.3 System output limitations

One of the issues that came up repeatedly during the

interviews was the difficulty in getting the desired

output from the system. Some respondents expressed

very strong and comprehensive opinions regarding

specific reports that they needed to perform a specif-

ic task:

It's lousy, and it doesn't tell you anything; it

doesn't give you the information you need, and it

just makes you[r] work more difficult – I found it

not easy to use, not the right information, didn't

update correctly, didn't have a lot of flexibility.

This employee was not against the use of the

system itself, but was, rather, an active user who was

simply highly frustrated with the fact that he/she was

not able to get the output that he/she knew was

available through the system.

In many cases, unfulfilled reporting needs led to

the downloading of a report or a set of raw data to an

outside application (such as a Microsoft Excel

spreadsheet) in order to get the desired outcome.

And unless I export that down into [an] Excel file

or something, the system [is not] capable of

compressing that [data], to minimize it, reduce it.

Even experienced users felt that it would be

highly useful to have an external expert helping with

data output tasks. They accepted this as a given

limitation that was related to the nature of this

specific ERP and its data warehousing solution.

What you need is a couple of experts … – hired

… we need to go out and we need to get a [data

warehousing] expert to be able to sit there and

listen to what we want, and then to be able to dig

that information out of [the ERP system] and put

it together and report that. That's the nature, I

think, of [the ERP system].

The same user believed that using query tools to

retrieve data from the system was excruciatingly

difficult.

It's just that you need to be a brain surgeon to

actually go out and …… produce your own

(queries).

The difficulties the users faced when attempting

to retrieve the desired system outputs demonstrate

that the system did not provide adequate support in

helping users access the information they needed in

a usable format.

3.4 Support in error situations

Some of the most significant and most commonly

mentioned difficulties with using the system were

caused by insufficient or misleading error messages.

For example, some of the messages were too general

to provide any useful information to the users:

IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION

131

You’ve got to go see somebody about how come

it’s red. But that’s after your transaction is

completed. … It just says transaction failed or

something like that.

Many of the interviewees described a specific

error message that appeared in a large number of

different situations, causing a great deal of confusion

and resulting in a lot of wasted time spent trying to

determine the actual cause of the problem. Both

examples below refer to this same error message:

Basically what [this error message] does is it

gives you a generic error and you have to do the

research to try to figure it out.

[This error message is] what I like to term the

system throwing up its hands and saying “I don’t

know.”

The users eventually identified several conditi-

ons that caused this particular error. Had the system

reported the possible causes to the user, it would

have saved them hours of confusion.

At times, the system failed to clearly communi-

cate the type of error:

We have a screen where we try and return parts

to the warehouse and if that's already been done,

the transaction has already been done, it says ‘[a

particularly cryptic error message]’ on the

bottom of the screen. What [expletive] does that

mean to anybody?

Other times, important errors were ignored by

the system, leading to potentially serious conse-

quences:

We had the customized front-end and we had to

hit two buttons to execute a transaction. But the

system will allow you just to hit one. So the guys

would hit one and everything would be green,

hey, I must be okay, but they never created the

other requirements that were necessary to

[complete the transaction].

Effective communication between the parties is a

prerequisite for the success of a collaborative effort.

Clear, well-defined error messages and guidance

concerning possible actions for the user to take

would be some of the simplest ways for the system

to provide adequate support.

3.5 Terminology problems

Another set of problems that affected the users’

ability to make effective use of the ERP system

arose from the fact that the system’s terminology

was different from users’.

The ‘Help’ is worthless because it's definitely

programmer’s language based. So having the

‘Help’ customized for business processes would

be [an] important piece.

In some cases, specific remedies were imple-

mented:

I put together a glossary of how the vocabulary

changed from pre-[ERP] to post, because people

didn't understand the terms.

Another user discussed the same situation with

more colorful language, clearly indicating that the

transition between the pre-ERP and post-ERP

vocabularies was not an easy one

Well, it was like the spaceship had landed, and

these outer space creatures [trainers] got off, and

started talking to us about how we were going to

do our job, because nobody understood what they

were saying. Now, they're talking about

notifications, material numbers, document

control, material masters -- you know, that wasn't

in any of our language.

The extent of the discrepancy between the two

vocabularies was surprising:

So, we had to list down the side of everything that

[ERP] brought in, okay? And then we checked

off which ones were identical to the nomenclature

that we had in legacy. Well, there were no

checkmarks.

One of the respondents recognized that the

terminology issue is not easy to solve – every

company, after all, uses their own vocabulary.

Recognition of the universality of this problem does

not, however, solve it. For every organization, the

vocabulary used for core business terms is an

essential part of the communication toolkit between

the members of the organization, and it does not

change overnight.

These terminology-related problems indicate that

the communication between collaborating parties

was clearly impaired by the lack of a common

vocabulary. Thus, the collaborative principle of

mutual responsiveness was violated.

ICEIS 2005 - HUMAN-COMPUTER INTERACTION

132

3.6 Overall system complexity

Another negative characteristic of the ERP system

that was commonly mentioned throughout the

interview process was its overall complexity, as

demonstrated by the following statement:

It's a very intimidating system. Some people are

very intimidated.

and by this, more colorful, example:

Th[is] guy could do it [assemble a very complex

engineering artifact] in his sleep. But tell him to

get a ERP GUI, logon password and explain to

him this is how [to execute a specific transaction

using ERP] and this is what's going wrong and

why you guys are doing this. Oh my god, he was

like a deer; it was like he got so upset because it

was so out of his kingdom, so out of his normal --

he shutdown on me. He actually shut down on

me.

A commonly expressed perception was that this

particular ERP system was a very complex one to

understand and use for a large portion of the users.

While this perception might have been partially

based on those users’ computer anxiety, it was clear

that general system characteristics at least contrib-

uted to their perplexity. Strong user perceptions of

system complexity again suggest that the principles

of mutual responsiveness and commitment to mutual

support have been violated.

4 CONCLUSION

This field study highlights many of the issues users

face when dealing with a large-scale ERP system.

While the eventual outcome in this case was a

success, the problems identified by the users clearly

impacted the amount of time it took them to learn

how to use the system, the number of errors

resulting from a lack of understanding about the

steps required to complete a process, and the level of

frustration felt by the users due to ill formed error

messages, unclear instructions, and the overall lack

of system helpfulness. Identifying factors affecting

the users’ ability to make the most effective use of

ERP systems is essential to understanding how the

design of these systems can be improved. The

potential impacts of enhancing usability are signifi-

cant; less frustrated users with a clearer understand-

ing of system usage will save organizations time and

money through lower training costs, faster ramp-up

times, and more complete usage of the system for all

the tasks it was meant to handle.

In addition to identifying and categorizing

usability problems of a large ERP system, this study

also demonstrates the promise of using collaboration

theory to evaluate usability characteristics of

existing systems and to design new systems. All

identified usability problem categories were explicit

violations of at least one of the principles of collab-

oration theory.

Future work must focus on identifying the rela-

tive importance of each of the problem categories in

terms of its impact on the organization, followed by

the development of means for addressing these

issues based on the guiding design principles derived

from collaboration theory.

REFERENCES

Babaian, T., Lucas, W. & Topi, H. 2004. Collaborating to

Improve ERP Usability. In Proceedings of ICEIS’04

Bratman, M. E. 1992. Shared cooperative activity. The

Philosophical Review, 101(2), pp. 327–341.

Chen, I.J. 2001. Planning for ERP systems: Analysis and

future trend. Business Process Management Journal,

7(5), pp. 374-387.

Davenport, T.H. 1998. Putting the enterprise back into the

enterprise system. Harvard Business Review, 76(4),

121-132.

Esteves, J. & Pastor, J. 2001. Enterprise resources

planning research: An annotated bibliography.

Communications of the AIS, 7 (8), 1–52

Gilbert, A. 2003. Business apps get bad marks in usability.

CNET News.com, http://news.com.com/ 2100-1017-

980648.html (current 11/7/2004).

Grosz, B. J. 1996. Collaborative systems. AI Magazine,

17(2), pp. 67–85.

Grossman, T. & Walsh, J. 2004. Avoiding the Pitfalls of

ERP System Implementation. Information Systems

Management, 21(2), p. 38-42.

Sarker, S., & Lee, A.S. 2003. Using a case study to test the

role of three key social enablers in ERP

implementation. Information & Management, 40(8),

813-829

Scott, J.E., & Vessey, I. 2002. Managing Risks in

Enterprise Systems Implementations. Communications

of the ACM, 45(4), 74-81.

Siau, K. 2004. Enterprise resource planning (ERP)

implementation methodologies. Journal of Database

Management, 15(1), 1-6

IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION

133