IDENTIFYING USABILITY ISSUES WITH AN ERP
IMPLEMENTATION
Heikki Topi, Wendy Lucas, Tamara Babaian
Bentley College, Computer Information Systems Department, Waltham, MA, 02452, USA
Keywords: usability, ERP, collaboration, enterprise system, user
Abstract: Enterprise Resource Planning (ERP) sy
stems hold great promise for integrating business processes and have
proven their worth in a variety of organizations. Yet the gains that they have enabled in terms of increased
productivity and cost savings are often achieved in the face of daunting usability problems. While one
frequently hears anecdotes about the difficulties involved in using ERP systems, there is little
documentation of the types of problems typically faced by users. The purpose of this study is to begin
addressing this gap by categorizing and describing the usability issues encountered by one division of a
Fortune 500 company in the first years of its large-scale ERP implementation. This study also demonstrates
the promise of using collaboration theory to evaluate usability characteristics of existing systems and to
design new systems. Given the impressive results already achieved by some corporations with these
systems, imagine how much more would be possible if understanding how to use them weren’t such an
overwhelming task.
1 INTRODUCTION
Enterprise Resource Planning (ERP) systems and
other large scale enterprise systems (such as Supply
Chain Management, Customer Resource Manage-
ment, etc.) are widely used in virtually all industries.
In the 1990s, enterprise systems became a core part
of the information systems architecture in a very
large number of corporations (Davenport, 1998),
even though many implementations have either
failed entirely or have not reached the goals set for
them (Davenport, 1998; Chen 2001; Scott & Vessey,
2002). Not surprisingly, enterprise systems have
been studied very actively, particularly by the
information systems community (Esteves & Pastor,
2001). Many of these studies have focused on the
factors affecting the success of ERP implementa-
tions (Sarker & Lee, 2003; Grossman & Walsh,
2004; Siau, 2004).
One of the areas that has not been widely studied
in either acade
mic or practitioner literature is the
usability of ERP systems and other large-scale enter-
prise systems. While usability problems may not
lead directly to a large-scale failure, they can
interfere with an individual’s or workgroup’s
productivity, making it harder for users to achieve
their goals as effectively or efficiently as is
desirable. They may also make system acceptance a
more difficult and lengthy process.
The long-term goal of our research is to develop
a set o
f design principles, the application of which
leads to improvements in the usability of ERP
systems. While these principles would be applicable
also to other large-scale systems, we are focusing on
ERP systems in part because of their aforementioned
widespread usage. Our contention is that improved
ERP usability can be achieved by designing the
system with collaboration as the model for user-
system interaction. The term collaboration should
not be interpreted literally: a computer system does
not have a free will and current technology is not
capable of creating a system with the capacity to
reason that is akin to a human collaborator. How-
ever, the metaphor of a system-partner, in which the
goals of the user are understood and supported by
the system, can be the framework for the design,
leading to system interfaces and interactions that are
more supportive of the user.
The user-system collaboration metaphor is
especially relevant i
n the enterprise-wide context,
where the system possesses a vast amount of
organizational data and is responsible for automating
a large number of business processes. One of the
central principles of collaborative behaviour
(Bratman, 1992) is the commitment to mutual
128
Topi H., Lucas W. and Babaian T. (2005).
IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION.
In Proceedings of the Seventh International Conference on Enterprise Information Systems, pages 128-133
DOI: 10.5220/0002522301280133
Copyright
c
SciTePress
support, which requires that when either party
recognizes that the other one needs help to complete
a subtask successfully, it will provide that help
(assuming it is capable of doing so). Another central
tenet is the commitment to joint activity, which
requires that both parties recognize and commit to
that activity. To make this commitment, the parties
must be aware of the context surrounding their
collaboration. A third principle is that of mutual
responsiveness, in which each party adjusts its
behaviour based on the behaviour of the other and
guided by commitment to the joint activity.
In this paper, we report on the findings of our
field study on an ERP system in one division of a
Fortune 500 company. Gaining insight into the
problems that users experience with existing ERP
systems and understanding the tasks that those
systems are intended to support are critical first steps
in working toward improved system usability. Our
findings confirm what can be surmised from the
large bodies of anecdotal evidence; namely, that
poor usability characteristics, such as unnecessarily
complex tasks and inadequate system support in
response to user errors, impact acceptance, usage,
and the usefulness of ERP systems, even in
successful implementations. These findings augment
the results of a laboratory comparison evaluating the
usability of a large number of ERP systems that was
conducted by Forrester Research (Gilbert, 2003).
None of these problems are new: textbooks in
systems analysis and design have warned against
them since at least the 1980s. The novelty of our
approach is in considering these flaws in light of
collaboration theory. Our study proposes that the
usability problems that greatly affect users’
performance can be viewed as examples of non-
collaborative behaviour by the system. Rather than
“standing" idly or silently by when the data in its
possession could help the user diagnose and fix a
problem, such as missing information on an invoice,
the system must work on the user’s behalf as a
collaborative partner.
The contributions made by this paper are as
follows:
It is the first attempt to focus on usability prob-
lems experienced by users of a large scale ERP
implementation. This study was conducted in a
division of a Fortune 500 company that had
been using its ERP system for two and a half
years, beginning with its launch in Summer
2001.
We provide a classification of usability prob-
lems encountered by users and show that these
flaws can be viewed as examples of non-collab-
orative behaviour by the system, thereby
providing strong evidence in support of our
contention that collaboration theory is a suitable
conceptual framework for usability design and
evaluation in the ERP context.
This work is part of a larger research project in
which we are also developing prototypes whose
design is driven by the central principles of collab-
orative behaviour (Babaian et al. 2004).
2 RESEARCH METHODOLOGY &
CONTEXT
The data used in this study was collected in ten in-
depth interviews with nine ERP users and one non-
user. The lengths of the interviews varied from 20
minutes to 90 minutes, both of which are outliers;
the length of most interviews was about an hour.
The interviews were conducted on the premises of
the employer of our interviewees, i.e., the organi-
zation that had deployed the ERP system of interest.
The interviews were semi-structured, starting with a
set of predefined questions (available from the
authors). The analysis of the data was performed on
the interview transcripts, with the interviewers’
notes from the interviews used as supporting
evidence.
To maintain the confidentiality of the target
organization, we do not describe it or the system at a
detailed level. At a general level, the system is used
to manage a challenging post purchase maintenance
process for a large and complex engineering
product. This ERP system is in use in five different
locations worldwide, including our target business
unit, where it has been used since 2001.
The respondents in the study were all employed
by the same organization and, with the exception of
the one non-user, were users of the same ERP
system. The respondents represent a variety of orga-
nizational roles, ranging from shop floor operational
workers to upper middle management. Even though
the roles vary, all of the respondents need the system
(or data generated from the system in the case of the
non-user) to fulfil their daily tasks within the
organization.
3 ANALYSIS OF THE RESULTS
The analysis of the interview data was based on the
transcripts. The analysis process began with the
creation of a classification structure derived from the
observations made during the interview process and
from an initial analysis of the transcripts. A research
assistant then classified the entire interview material
at a detailed level to identify usability problems; if
necessary, new categories were added to the
IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION
129
structure. The results of the research assistant’s work
served as the foundation from which the authors
completed the final usability problem categorization
presented below. The remainder of this section will
discuss the identified usability problem categories
and their significance from the perspective of
collaboration theory.
A qualitative analysis of the interview
transcripts led to the following categories of
usability problems:
Identification of and access to the correct
functionality
Transaction execution support
System output limitations
Support in error situations
Terminology problems
Overall system complexity
Following are descriptions and examples by cate-
gory of the actual problems encountered by ERP
system users.
3.1 Identification of and access to the
correct functionality
Several users noted that finding specific
functionality quickly within the system sometimes
required an unreasonable amount of effort. A simple
example of this is navigation; that is, finding a
specific transaction. Navigation problems seemed to
be typical, as illustrated by the following comments
of users, at least one of whom has extensive
knowledge and experience with multiple system
modules:
But it’s also very intensive, cumbersome; there’s
multiple transactions that unfortunately is [sic]
not linked to basically a central menu area to
walk you through. You have to kind of know what
the transaction codes are and how to execute
them.
Or maybe we have some menus, but presently it
may take us four, five or six routes to get us to
basically one screen. I don’t always see the links.
The basic problem with navigation was that very
little help was provided to the users by the system
for determining the correct transaction screens and
how they could be accessed most directly. The
problem was most severe for those users who were
not very familiar with the system and the screens
needed to perform a specific business transaction.
A more significant problem was the users’ diffi-
culty in understanding and/or remembering which
set of actions was necessary for completing a spe-
cific business process. It is very typical that the user
of such a system has to perform a number of ERP
transactions in order to complete one specific
business process. Since the ERP system in our target
organization was not designed to be aware of the
business process the user was attempting to achieve,
it was unable to guide the user through it, as
indicated by the following comment:
There's nothing that says okay, I'm here. Act on
me now. There's nothing that exists. … No,
[there’s a ] transaction for ‘A’, a transaction for
‘B’, and a transaction for ‘C’ and nothing links
them. Someone has to manually do each one of
those.
After a while, the users had memorized the most
common sequences, but it took a significant amount
of time to get to that point. In some cases, the
organization decided that understanding the correct
ordering of ERP transactions to implement a
particular business process was important enough to
document on laminated “cheat sheets,” as noted in
the following quote:
And even the training resulted in some sheets
being made out with the laminated plastic, with
walk through menus for [a specific employee
type], what to process, why you would process it
and things like that, that were very beneficial.
Throughout the interviews, the lack of system
support for understanding the business processes
that mapped to the ERP tasks was identified as one
of the most significant problems. One respondent
expressed the importance of understanding and
focusing on the business processes, rather than the
system transactions:
But I can't stress enough the fact that the
transactions are the easy part; it's the business
that's the important piece.
When evaluated in the context of collaboration
theory, the users’ difficulties with navigation and
understanding the correct transaction sequence for
particular business processes suggest that the system
design did not properly capture the composition of
many high-level business tasks for which the system
is used. As a result, the system cannot guide a user
through the steps of a composite business process,
and the user was left with the burden of searching
for and memorizing the correct sequence of opera-
tions. This indicates that the system was designed
without commitment to joint activity: the system
cannot be committed to the user’s goals if it has not
been designed to be aware of them.
ICEIS 2005 - HUMAN-COMPUTER INTERACTION
130
3.2 Transaction execution problems
Many of the usability problems that the users faced
were related to the completion of system trans-
actions. Users often found the transaction interfaces
unduly complex. This was evidenced by the cheat
sheets developed in-house and used by the
employees even after the training and by the
customized data entry forms created to compress a
transaction into fewer screens, thereby eliminating a
lot of optional entry items.
Some transaction level problems were quite
well-known, such as the one noted by this user’s
question:
…why do I have to keep entering the same data
over and over?
A thoughtful collaborator would recognize the
need for this repetitive data and would automatically
enter it, as is typically done by many modern
systems. Another, very experienced user expressed
the same problem in different words:
Well, I mean, we're so used to copying and
pasting. … In some cases, it [the ERP system]
remembers and will carry over to some of the
screens, but not in all cases.
One of basic strengths of a computer system is
in automating mundane data entry tasks, yet the ERP
system left this job to the user. The problems related
to the entry of redundant data are well known
(increased errors, inconsistency between system
components, increased storage needs, etc.). Given
that the repetition of field values between different
screens is derived from the underlying business
processes, the system should automatically provide
the repeated data in subsequent screens rather than
requiring the user to either reenter or copy and paste
it.
These transaction level problems indicate that
the system lacked the capability of modifying its
behavior based on the user’s actions and was not
supporting the user in situations where help was
clearly needed.
3.3 System output limitations
One of the issues that came up repeatedly during the
interviews was the difficulty in getting the desired
output from the system. Some respondents expressed
very strong and comprehensive opinions regarding
specific reports that they needed to perform a specif-
ic task:
It's lousy, and it doesn't tell you anything; it
doesn't give you the information you need, and it
just makes you[r] work more difficult – I found it
not easy to use, not the right information, didn't
update correctly, didn't have a lot of flexibility.
This employee was not against the use of the
system itself, but was, rather, an active user who was
simply highly frustrated with the fact that he/she was
not able to get the output that he/she knew was
available through the system.
In many cases, unfulfilled reporting needs led to
the downloading of a report or a set of raw data to an
outside application (such as a Microsoft Excel
®
spreadsheet) in order to get the desired outcome.
And unless I export that down into [an] Excel file
or something, the system [is not] capable of
compressing that [data], to minimize it, reduce it.
Even experienced users felt that it would be
highly useful to have an external expert helping with
data output tasks. They accepted this as a given
limitation that was related to the nature of this
specific ERP and its data warehousing solution.
What you need is a couple of experts … – hired
… we need to go out and we need to get a [data
warehousing] expert to be able to sit there and
listen to what we want, and then to be able to dig
that information out of [the ERP system] and put
it together and report that. That's the nature, I
think, of [the ERP system].
The same user believed that using query tools to
retrieve data from the system was excruciatingly
difficult.
It's just that you need to be a brain surgeon to
actually go out and …… produce your own
(queries).
The difficulties the users faced when attempting
to retrieve the desired system outputs demonstrate
that the system did not provide adequate support in
helping users access the information they needed in
a usable format.
3.4 Support in error situations
Some of the most significant and most commonly
mentioned difficulties with using the system were
caused by insufficient or misleading error messages.
For example, some of the messages were too general
to provide any useful information to the users:
IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION
131
You’ve got to go see somebody about how come
it’s red. But that’s after your transaction is
completed. … It just says transaction failed or
something like that.
Many of the interviewees described a specific
error message that appeared in a large number of
different situations, causing a great deal of confusion
and resulting in a lot of wasted time spent trying to
determine the actual cause of the problem. Both
examples below refer to this same error message:
Basically what [this error message] does is it
gives you a generic error and you have to do the
research to try to figure it out.
[This error message is] what I like to term the
system throwing up its hands and saying “I don’t
know.”
The users eventually identified several conditi-
ons that caused this particular error. Had the system
reported the possible causes to the user, it would
have saved them hours of confusion.
At times, the system failed to clearly communi-
cate the type of error:
We have a screen where we try and return parts
to the warehouse and if that's already been done,
the transaction has already been done, it says ‘[a
particularly cryptic error message]’ on the
bottom of the screen. What [expletive] does that
mean to anybody?
Other times, important errors were ignored by
the system, leading to potentially serious conse-
quences:
We had the customized front-end and we had to
hit two buttons to execute a transaction. But the
system will allow you just to hit one. So the guys
would hit one and everything would be green,
hey, I must be okay, but they never created the
other requirements that were necessary to
[complete the transaction].
Effective communication between the parties is a
prerequisite for the success of a collaborative effort.
Clear, well-defined error messages and guidance
concerning possible actions for the user to take
would be some of the simplest ways for the system
to provide adequate support.
3.5 Terminology problems
Another set of problems that affected the users’
ability to make effective use of the ERP system
arose from the fact that the system’s terminology
was different from users’.
The ‘Help’ is worthless because it's definitely
programmer’s language based. So having the
‘Help’ customized for business processes would
be [an] important piece.
In some cases, specific remedies were imple-
mented:
I put together a glossary of how the vocabulary
changed from pre-[ERP] to post, because people
didn't understand the terms.
Another user discussed the same situation with
more colorful language, clearly indicating that the
transition between the pre-ERP and post-ERP
vocabularies was not an easy one
Well, it was like the spaceship had landed, and
these outer space creatures [trainers] got off, and
started talking to us about how we were going to
do our job, because nobody understood what they
were saying. Now, they're talking about
notifications, material numbers, document
control, material masters -- you know, that wasn't
in any of our language.
The extent of the discrepancy between the two
vocabularies was surprising:
So, we had to list down the side of everything that
[ERP] brought in, okay? And then we checked
off which ones were identical to the nomenclature
that we had in legacy. Well, there were no
checkmarks.
One of the respondents recognized that the
terminology issue is not easy to solve – every
company, after all, uses their own vocabulary.
Recognition of the universality of this problem does
not, however, solve it. For every organization, the
vocabulary used for core business terms is an
essential part of the communication toolkit between
the members of the organization, and it does not
change overnight.
These terminology-related problems indicate that
the communication between collaborating parties
was clearly impaired by the lack of a common
vocabulary. Thus, the collaborative principle of
mutual responsiveness was violated.
ICEIS 2005 - HUMAN-COMPUTER INTERACTION
132
3.6 Overall system complexity
Another negative characteristic of the ERP system
that was commonly mentioned throughout the
interview process was its overall complexity, as
demonstrated by the following statement:
It's a very intimidating system. Some people are
very intimidated.
and by this, more colorful, example:
Th[is] guy could do it [assemble a very complex
engineering artifact] in his sleep. But tell him to
get a ERP GUI, logon password and explain to
him this is how [to execute a specific transaction
using ERP] and this is what's going wrong and
why you guys are doing this. Oh my god, he was
like a deer; it was like he got so upset because it
was so out of his kingdom, so out of his normal --
he shutdown on me. He actually shut down on
me.
A commonly expressed perception was that this
particular ERP system was a very complex one to
understand and use for a large portion of the users.
While this perception might have been partially
based on those users’ computer anxiety, it was clear
that general system characteristics at least contrib-
uted to their perplexity. Strong user perceptions of
system complexity again suggest that the principles
of mutual responsiveness and commitment to mutual
support have been violated.
4 CONCLUSION
This field study highlights many of the issues users
face when dealing with a large-scale ERP system.
While the eventual outcome in this case was a
success, the problems identified by the users clearly
impacted the amount of time it took them to learn
how to use the system, the number of errors
resulting from a lack of understanding about the
steps required to complete a process, and the level of
frustration felt by the users due to ill formed error
messages, unclear instructions, and the overall lack
of system helpfulness. Identifying factors affecting
the users’ ability to make the most effective use of
ERP systems is essential to understanding how the
design of these systems can be improved. The
potential impacts of enhancing usability are signifi-
cant; less frustrated users with a clearer understand-
ing of system usage will save organizations time and
money through lower training costs, faster ramp-up
times, and more complete usage of the system for all
the tasks it was meant to handle.
In addition to identifying and categorizing
usability problems of a large ERP system, this study
also demonstrates the promise of using collaboration
theory to evaluate usability characteristics of
existing systems and to design new systems. All
identified usability problem categories were explicit
violations of at least one of the principles of collab-
oration theory.
Future work must focus on identifying the rela-
tive importance of each of the problem categories in
terms of its impact on the organization, followed by
the development of means for addressing these
issues based on the guiding design principles derived
from collaboration theory.
REFERENCES
Babaian, T., Lucas, W. & Topi, H. 2004. Collaborating to
Improve ERP Usability. In Proceedings of ICEIS’04
Bratman, M. E. 1992. Shared cooperative activity. The
Philosophical Review, 101(2), pp. 327–341.
Chen, I.J. 2001. Planning for ERP systems: Analysis and
future trend. Business Process Management Journal,
7(5), pp. 374-387.
Davenport, T.H. 1998. Putting the enterprise back into the
enterprise system. Harvard Business Review, 76(4),
121-132.
Esteves, J. & Pastor, J. 2001. Enterprise resources
planning research: An annotated bibliography.
Communications of the AIS, 7 (8), 1–52
Gilbert, A. 2003. Business apps get bad marks in usability.
CNET News.com, http://news.com.com/ 2100-1017-
980648.html (current 11/7/2004).
Grosz, B. J. 1996. Collaborative systems. AI Magazine,
17(2), pp. 67–85.
Grossman, T. & Walsh, J. 2004. Avoiding the Pitfalls of
ERP System Implementation. Information Systems
Management, 21(2), p. 38-42.
Sarker, S., & Lee, A.S. 2003. Using a case study to test the
role of three key social enablers in ERP
implementation. Information & Management, 40(8),
813-829
Scott, J.E., & Vessey, I. 2002. Managing Risks in
Enterprise Systems Implementations. Communications
of the ACM, 45(4), 74-81.
Siau, K. 2004. Enterprise resource planning (ERP)
implementation methodologies. Journal of Database
Management, 15(1), 1-6
IDENTIFYING USABILITY ISSUES WITH AN ERP IMPLEMENTATION
133