User Participation and Involvement in the Development

of HR Self-service Applications within the Dutch

Government

Gerwin Koopman

and Ronald Batenburg

Syntess Software, PO Box 127, 4300 AC Zierikzee, The Netherlands

Utrecht University, Department of Information and Computing Sciences, PO Box 80089

3584 CS Utrecht, The Netherlands

Abstract. This paper departs from the notion that user participation and

involvement are important factors for IS success. Five case studies portrayed

are based on interviews with civil servants employed at different governmental

organizations. In line with the findings from the literature, respondents argued

that users should be involved early. A number of important lessons was learned

by the respondents. The first was to use expectancy management, i.e. keep users

informed about developments and motives for certain decisions. Second,

employees should use the self-service applications without much support from

the HR-departments. A third important aspect is the distinct decision process

public organizations deal with.

1 Introduction

Although IS/IT-problems exist in both the private sector and the public sector, failures

in the latter are more visible. Reasons for this visibility are the public accountability

and lack of commercial confidentiality governmental organisations have to deal with

[1]. A good example of the public accountability could be observed in 2007, when

The Dutch House of Representatives asked the Dutch Government questions about

their ICT-expenditures. Concerns were raised about how much money was wasted by

governmental ICT projects that resulted in failures. The Netherlands Court of Audit

was instructed to come up with a report on governmental ICT projects and the

possible reasons for failures. When the report [2] was finished, it named several

difficulties that can be faced when executing ICT projects for governmental

organisations. Among this list is the impact of the changes caused by the

implementation of the IT-system. Users’ current way of working may be completely

changed by changing work processes when the system is introduced. Users therefore

need to be informed and trained to completely benefit from the system. Another cause

for problems is the need for clear goals and demands. If the software developer does

We greatly acknowledge the support of Gerrit Kreffer (Ministry BZK, The Netherlands).

Koopman G. and Batenburg R. (2008).

User Participation and Involvement in the Development of HR Self-service Applications within the Dutch Government1.

In Proceedings of the 2nd International Workshop on Human Resource Information Systems, pages 16-29

DOI: 10.5220/0001744900160029

 SciTePress

not receive clear demands and wishes, the actual end-product might not be what the

government thought it would receive.

In both of the mentioned problems users play an important role in making the system

a success. There is already a lot of agreement on the fact that users should be involved

to produce usable software programs. It is recommended in ISO standard 13407 to get

better insights in the requirements for a software application. Most attention to user

involvement is still on the usability testing of systems, which happens on a later stage

in the development process. However, the sooner the end-user is involved, the more

efficient it is [3,4,5].

One of the challenges in involving users in IT developments is the time factor that

plays a very important role in governmental IT projects. Most of the decisions to

implement or develop new IT have a political background. This means the project will

have to be delivered at the end of the current cabinet’s term. This conflicts with the

idea that user involvement will take a serious amount of extra time needed in the

development of a new system [6]. The systems that have the specific attention of this

paper on first sight also seem to conflict with this additional time needed in IT

projects when involving users. Main reasons for implementing E-HRM systems and

Shared Service Centres are increasing efficiency and productivity [7,8]. For the

current Dutch cabinet this is very important because it wants to decrease the number

of civil servants with a number of 12,800 to achieve a cost cutback of 630 million

Euros in four years [9].

In this paper we address the question which methods are currently used within the

Dutch governmental institutions to involve end-users when developing self-service

applications, and what the relationship is between end-user participation and

involvement and the success of e-HRM applications. Four Dutch Ministries are

investigated that are implementing e-HRM, which offers the possibility to compare

the different development approaches that are followed. Theoretical notions on end-

user involvement are validated by semi-structured interviews or topic interviews with

stakeholders from within the Ministries. Conclusions are drawn with regard to which

methods are used to involve users, and what their relative success contribution is.

2 Background and Theory

2.1 The Benefits of End-user Involvement

Reviewing several studies on user involvement [10] identified a number of benefits of

participating end-users:

More Accurate user Requirements. Numerous problems or defects in software

applications can be traced back to poorly capturing requirements at the beginning of

the development process [11]. Pekkola et al [12] also argue one of the reasons for

information system development projects to fail are incomplete requirements. In their

studies they found user participation useful in gathering “credible, trustworthy and

realistic descriptions of requirements”. In turn these accurate user requirements result

in an improved system quality [13].

Avoidance of Unnecessary or Unusable Costly System Features. Two of the usability

guidelines given by Nielsen [14] are “Designers are not users” and “Less is more”.

Designers might think of certain features to incorporate in the application without

consulting end-users. Functionalities that are completely logical for developers, might

be completely incomprehensible for users. This might result in users having to spent

too much time on learning how to use these functionalities or even not using them.

Designers might also have the tendency of incorporating too many options to satisfy

‘every’ end-user. Besides the fact that users might never know of these options and

use them, they can also work contra productive by being overwhelming to users. A lot

of time and effort for developing these features can be saved by participating end-

users.

Improved Levels of System Acceptance. The levels of system acceptance can be

positively influenced by user involvement in several ways. Among the list Ives &

Olsen [15] have found in other literature are for instance the development of realistic

expectations about the information system [16]. Also decreasing user resistance

against the new system and actually creating commitment for it are other results of

user participation [17]. Cherry & Macredie [18] state participatory design as a means

to overcome the acceptability problems systems might encounter without the

participation of users.

Better Understanding of the System by the User. Logically during the participation

users will learn about the system by experiencing the development [17]. This

familiarisation also leads to the increase in chances users will come up with

suggestions during the development, because they will feel more confident [19]. In the

end this greater understanding should lead to a more effective use of the application.

Increased Participation in Decision-making within the Organization. Clement & Van

den Besselaar [20] point to the fact that participation is not only restricted to the

design of an IT-system. The application will probably change the way tasks are

executed, thus affecting the entire organization and by participating, employees have

the possibility to influence this [19]. It might thus not be restricted to the design of the

application, but also to other decision-making processes within the organization.

Although involving users is considered to be useful, it also introduces a number of

difficulties. Firstly, a large amount of a user’s knowledge about the process or task the

software application will have to support, has become tacit [21]. It might therefore be

hard to get information from these users about the way they work. To overcome this

kind of problems it is possible to perform field studies, which have the advantage that

users do not have to articulate their needs [13]. Other researchers also suggest the use

of (paper) prototypes to counter the difficulties users might have in articulating their

needs [12, 14]. Users can also be reluctant to have developers observing them while

they work, however [22]. They might express concerns about justifying the time they

would have to spent with the design team or disturbing their co-workers. Solutions to

this problem are getting commitment from management [6] and having sessions in

separate rooms so no colleagues would have to feel bothered. Besides these problems

Butler [22] mentions the fact that these sessions are considered to consume a lot of

time, as well in planning them, as in executing them. Several researchers also point

out the fact that involving users most of the time delivers a large amount of raw data

that is difficult to analyse and to use in decision making [23, 24]. This will make

projects where users participate more time-consuming and thus something

development teams want to cut back on. Grudin [6] also noted the judgement of

developers that user involvement would take too much time. However, as already

stated, allocating more time upfront will result in a faster cycle time for software

developers [5].

Some members of the design team might simply not have the abilities needed to

communicate efficiently with users [6]. They might find it difficult to understand the

work situations of users or miss the empathy needed when communicating with users

that do not possess the computer skills they have themselves. As a solution to the

problematic communication between users and developers, mediators could be

brought into action [12]. They can act as a bridge between both groups, translating the

different concepts from one group to the other. Mock-ups and prototypes from the

design team are for instance discussed with users, while user input and feedback is

given to the design team. Developers can then focus on the design and implementation

of the application instead of having to spent time and effort on user participation

methods.

A challenge that occurs even before all of these mentioned is the selection of user

representatives and obtaining access to them [6]. Even when an application is

developed specifically for one organisation, developers might fear the risk of missing

certain user (groups) in their selection. A possible solution is to define a few personas

based on intended users. A persona is defined as “an archetype of a user that is given a

name and a face, and it is carefully described in terms of needs, goals and tasks” [25].

This can be useful in organisations that have large groups of users, which makes it

tricky to randomly take a small selection out of the total group.

2.2 Early Involvement and Participation of Users

User participation can take on a number of forms in the development of a software

product. Kujala [13] suggests four main approaches are detectible, which are user-

centred design, participatory design, ethnography and contextual design.

Gould & Lewis [26] in their research on user-centred design recommend the early

focus on users and direct contact between development team and end-users. This

means interviews and discussions with end-users, even before any design has been

made. Also people should be observed when performing tasks, as well in the present

situation as with prototypes that are developed during the project. Also the design

should be iterative, this could for instance be realised by using prototypes that can be

reviewed by users.

Participatory design is considered to be a design philosophy instead of a

methodology [18]. It is not prescriptive and therefore the set of techniques that could

be used should be considered open-ended. The approach does have some identifiable

principles however, firstly it aims at the production of information systems that

improve the work environment. Secondly, users should be actively involved at each

stage of the development and finally the development should be under constant

review (iterative design). Cherry & Macredie [18] also mention four important

techniques, cooperative prototyping being the main technique. The other techniques

are brainstorming, workshops and organisational gaming.

Ethnography consists of observing and describing the activities of a group, in an

attempt to understand these activities [27]. In the design of information systems it is

defined as developing “a thorough understanding of current work practices as a basis

for the design of computer support” [28]. The reason for this is the occurrence of

differences in what users say they do, and what they actually do [14]. The approach is

descriptive of nature, is from a member’s point-of-view, takes place in natural settings

and behaviours should be explained from their context [29]. A typical method of

ethnography is thus observing end-users while they perform their daily work.

Similar to ethnography is contextual design. Its goal is to help a cross-functional

team to agree on what users need and design a system for them [30]. The approach

focuses on the improvement of the current way of working within an organisation. It

thus is not only limited to the design of a system, but also incorporates redesigning

the work processes. Users are the main source for data to support decisions on what

developments should take place. Specific methods to obtain information from users

are (paper) prototyping and contextual inquiry. The latter method is a combination of

observing users and interviewing them at the same moment [30]. Co-development,

ethnographic methods and contextual inquiry are participatory methods that are

located early in the development cycle [31]. Most of the approaches actually span the

entire development.

2.3 Employee Self-service

In search of defining Employee Self-service (ESS) we need to address the field of e-

HRM implying the benefits from the development in technologies that support the HR

activities. To quote Ruël et al [32]: “e-HRM is a way of implementing HR strategies,

policies, and practices in organizations through a conscious and directed support of

and/or with the full use of web-technology-based channels”. In this definition the term

web-technology is mentioned, that is the basis for a category of e-HRM systems the

so-called Employee Self-service applications. Employee self-service is defined by

Konradt et al. [33] as “corporate web portal that enables managers and employees to

view, create and maintain relevant personnel information”. They identify four

different basis channel functions the ESS can support (1) informing employees about

rules and regulations, (2) providing interaction and the access to personal information,

(3) supporting transactions, like applications for leave, and (4) delivering for instance

payslips or training videos.

All of the above tasks are normally done by the organisations’ HR departments.

Gale [34], in her study on three successful ESS implementations, describes reducing

the workload of these personnel departments is a major reason for implementation.

For instance, changing personal information of employees in often several databases

normally had to be done by HR employees. This can now be done by employees

themselves by filling in web based forms, resulting in (real-time) updates of the

databases of the HR systems. The web based nature of the ESS also offers the

possibility to significantly decrease the paperwork that needs to be handled. However,

the benefits are not only on the organisations’ side, employees also profit from the

implementation of ESS. They have instant access to information and the effort needed

for certain transactions, like expense claims, is reduced. Managers also benefit from

the up-to-date information and easy access to for instance reports, resulting in a better

overview over their resources.

Konradt et al. [33] used the Technology Acceptance Model (TAM) [35] to describe

the influences of a systems’ usefulness and ease of use on user satisfaction and system

use. The research model used in their investigation states that the ease of use related

positively to user satisfaction, as well as to usefulness. Usefulness in turn positively

influenced both system use and user satisfaction. A final relationship was described

between user satisfaction and system use. A number of implications were drawn from

these findings to ensure the success of an ESS implementation. Informing and

involving employees during the development is advised to influence the ease of use

and usefulness of the application. If users do not accept the system, the workload

reduction for HR department will not be realised. Instead of the normal workload, HR

employees will be flooded with help requests by users who do not understand the

system or are even reluctant to work with it.

3 Data and Methods: Evaluating Four ESS-Projects

To determine in what ways users are involved or enabled to participate in the

development of (ESS) software applications, interviews were held at four

organisations of the Dutch government with regard to four different types of ESS-

projects.

3.1 Emplaza

This self-service human resources application is used by approximately 5,500 civil

servants within the Ministry of the Interior and Kingdom Relationships. The

application is also used by the Ministry of Agriculture, Nature and Food Quality and

the Ministry of Economic Affairs, resulting in a total number of about 17,000 users.

The software supports up to twenty HR-processes, for instance applying for leave or

filing an appraisal conversation. The application is actually a sort of web application

and functions as layer over the actual administrative IT-system. It is built and

managed by an external party.

At the time of the interview a new release of the application (version 4.3) was

under development. This will be the base for this case description. Since the

application is not entirely new some of the reactions of the users can be expected

based on experiences from the previous releases. These experiences also influenced

the way in which new releases or features are developed. This time however, the

release has taken more than a year to develop because of some important differences

with previous situations. First of all the builders were new to the project and therefore

the advantage of having worked together (as with previous releases) was lost. Second,

release 4.3 can be considered larger and more extended in words of number of

functionalities. As a result testing the application a considerable amount of extra time

was needed to test this version.

For the development of the new releases key-users or super-users were selected to

participate. These civil servants have a lot of knowledge about the process the

application is supposed to support. By interviewing them they current way the process

is executed is determined. A next step was to establish which forms should be

available to support tasks within the process. After that the next task was to find out

how the forms and workflow should look like in Emplaza. When agreement was

found on these issues the Functional Designs were created by the software developer.

Before the actual programming started a number of applications that supported similar

HR-processes were investigated. Findings from this analysis formed the starting point

of how this should be realized in the Emplaza application.

The key-users are thus very involved in the business rules that need to be

implemented in the system. Other aspects they are asked to judge are the look-and-

feel of the user interface and the performance of the application. To do this they have

to use test-scripts that will force them through every step and part of the new

functionality so they will be able to comment on all the new developments. Members

of the HR self-service project team also test the application by looking at it from the

viewpoint of a ‘new’ user. They specifically pay attention to the help texts that are

created for the end-users to guide them through certain tasks.

A number of criteria were used in selecting employees to participate in the

development of the new release. Participants had to have a lot of knowledge and

experience in the field concerning the process at hand. Furthermore, they had to be

available to cooperate, i.e. they had to be freed from their normal tasks. Finally, they

also had to be able to think constructively about the new functionality. Most of the

time it had become clear in earlier sessions whether or not people met this latter

criterion. For testing the application managers are asked to cooperate, they are

selected on their position within the organization and thus all have a different role in

the HR-process that is going to be implemented in the system. It is tried to have two

‘camps’: those who are sceptical of ICT and those who feel positive about ICT.

End-users are actually involved just when the new release has gone ‘live’ i.e. has

gone into production. Complaints and issues that come up during the use of the

application are gathered and reviewed. These form the foundation for the change

proposals that are discussed on a interdepartmental level. During these discussions

decisions are taken on which changes really need to be implemented. If end-users are

involved earlier after all, most of them come from the central apparatus of the

Ministry. The reason for this is that they are located close to the test location. They

are chosen as randomly as possible, so no real criteria are used to select participants.

This way the development group hopes to get ‘fresh’ insights about the application.

3.2 P-Direkt

In July 2003 the Dutch government chose to start the establishment of a Shared

Service Centre (SSC) called P-Direkt. This should be a Human Resource Management

SSC for Personnel registration and Salary administration. Although the project had

some major problems, it is now still in progress with the same main goal. It should

lead to a more efficient HR-column of the government [36]. Two identified conditions

to reach this goal are joining administrative HR-tasks and the implementation of

digital self-service. The latter of these recognised conditions makes P-Direkt an

interesting subject to examine and see how user involvement or participation are

applied in this project.

The respondent for this interview is the Test manager at P-Direkt and is

responsible for the Functional Acceptation Tests and User Acceptation Tests of the

HR-Portal that is currently developed. This self-service HR-portal should eventually

be used throughout the entire government. In contrast to the Emplaza 4.3 release this

is an entirely new application. It is built using mostly standard functionalities of SAP

but, if necessary, customisation is also applied.

In the process of developing the self-service application users are involved in

different ways and at different stages. Right from the start several workgroups are

formed. These consist of civil servants from several ministries that in the end will use

the application. The members of these groups could be considered end-users, as they

will eventually use the application in their normal work. However, they have a lot of

knowledge about the HR-processes the application should support. These workgroups

are full-time dedicated to the development of the application for a longer period of

time. One workgroup for instance has been involved from the start in simplifying and

standardizing the HR-processes within the Dutch Government. After twenty-four

processes had been defined they formed the basis to build the technical system that

should support them. The workgroups were then involved in incorporating the right

business rules within this system. An example of such a rule is calculating the

maximum compensation that should be granted in different situations.

At the final part of building the application, a number of end-users are asked to

test the application. This group of end-users do have knowledge about the processes

that should be supported, however they were not earlier involved in the development

of the application. The involved departments are asked to send one or two employees

to take part in the User Acceptation Tests. Per session seven to ten participants are

asked to complete the scenarios that are designed to guide them through a certain task.

These tasks, for instance filing an expense claim, are subdivided in different steps.

This way users cannot only comment on the application in general, but also notate

findings about specific steps in the process. In this way the scenarios also contribute

to being able to easily group comments about certain steps in the process or specific

parts from all the different test users. These grouped and summarized comments and

findings are then discussed by P-Direkt and the builder of the application. During

these discussions it is decided which findings need fixes and those are then built

within two to three days. After that a new test session is held to examine whether or

not the problems were sufficiently solved.

3.3 P-Loket

The Ministry of Health, Welfare and Sport uses an application which is very similar to

Emplaza. It is called P-Loket and was also developed for the Ministry of Social

Affairs & Employment and the General Intelligence & Security Service, because they

use the same payroll application. P-Loket is a web application, and functions as a

layer on top of this payroll application (PersonnelView, or P-View). This situation is

thus very comparable to the one at the Ministry of the Interior.

P-Loket is a totally new developed application, which can be used by employees

to support them in (personnel) tasks like for instance the filing of a request for leave.

In June 2007 around 12 different forms are supported by the application, which can be

used by approximately 2,250 civil servants. These numbers should grow to about 18

forms and 5,000 employees by January 2008. The forms and processes that should be

supported were chosen based on the outcomes of the standardisation workgroup of the

P-Direkt project. The P-Direkt project is also the reason that after the 18 forms are

finished no further developments will be done. The P-Direkt application will

eventually substitute P-Loket.

From the start of the development of the application (in 2006) it was already clear

P-Direkt would be the governmental HR self-service application. However, for

reasons of more rapidly realisable efficiency benefits and to get used to self-service

applications, it was decided to still start the development of P-Loket. The first quarter

of 2006 was used as preparation and to come up with a plan of how to approach the

project. The second quarter of the year was used to prepare for the building, make a

process design and setting up authorisations. By the end of June the actual creation of

the application could start.

Building the application was done by an external software company (Allshare)

that also created the P-View application. P-Loket was also a totally new developed

application for them. However, P-View and P-Loket are quite similar web

applications, which had some advantages. The links for instance that had to be

available, were already more or less present and thus had not to be created completely

from scratch. They had one developer working full-time on the project.

Employees of the Ministry were involved in several ways during the development.

The project group that was formed at the start consisted of HR-employees, members

of the audit service and two employees of the Information & Communication

department. The latter two were experts on web (applications) and usability. Both

these experts had the task to look at the application from a user perspective. The

usability expert for instance discussed a number of prototypes (on screen) with the

builder. By asking questions like “what will happen if a user clicks this button?”

issues could already be addressed before anything was programmed. Apart from the

experts the project group members did not attend to the User Interface of the

application. Their focal point was on the business rules that should be implemented.

Next to the fact that users were represented in the project group, other civil

servants were asked to cooperate in an usability test. This test was carried out by a

third party and the main reason was to resolve usability issues the software builder

and the usability expert from the project could not agree on. The test was carried out

with one test person and a guide in one room, observers were in another room to take

notes and film the session with a camera. In selecting employees to take part in this

test, the project group tried to have a balance in computer skills, male/female ratio

and office/field staff ratio. To find eight participants, contact persons were asked if

they knew employees that fitted the necessary characteristics.

Another way to involve end-users was to have sessions with managers to discuss

the functionality that supports performance review conversations with them. Per

session the application was demonstrated to three to twelve managers. It took roughly

five weeks to complete the sessions with two hundred and fifty managers. Managers

could immediately deliver feedback in the form of questions or remarks during the

demonstration. Although this way of involving end-users took considerable time and

effort, it was considered to be very useful and contributing to the acceptance of the

application.

Also the fact that the project group was located in the same offices as end-users that

were not in the project group offered the possibility to ask these colleagues for their

opinions in an informal, ad hoc way. The project group gratefully made use of this

opportunity during the development of P-Loket.

3.4 PeopleSoft HR

The Ministry of Defence started implementing self-service on HR-processes in 2004,

but without involving end-users. As a result the users started having wrong

interpretations about the application. Therefore the Ministry started with improving

the self-service parts of the application in 2006. The application is based on the

PeopleSoft HR-system and the first processes to be supported were looking into

personal data, filing requests for leave and filing requests for foreign official tours.

Approximately 80,000 users make use of the software, of which about 65,000 are

permanent staff of the Ministry. Besides this large group of users another point of

consideration is the sometimes disrupted relationship with the formal superior. This is

due to frequent shifts within the organisation, for instance staff being posted abroad

for military operations. As a result the application should offer the possibility to

delegate certain tasks to other superiors, planners and/or secretaries.

It took about one and a half years from the beginning of the project until the

improved application went live. As a start it was determined which people and

processes should be supported. Subsequently the possibilities of the (then) current

application were investigated. There were three important points of departure: (1)

outcomes of using should be visible to the user, (2) employees use the application in

good faith (“the user does nothing wrong”), and (3) no training should be necessary to

use the application.

The main idea behind this is that the development should not only be seen as

supporting a processes by an application, but also supporting users in their actions

when using the application. Usability research was done by someone from outside the

Ministry who had no knowledge of HR-processes or PeopleSoft. This person asked

the civil servants how the current application was used. Some consultancy was done

by external parties, but it felt most of the work was done by the internal organisation

to come up with the advises and reports.

Besides the usability research, users were also involved in other ways. Employees

with reasonable knowledge and skills about IT were asked to name functional gaps in

how the support of certain processes. Next to that case studies were done by randomly

asking people in the organisation to perform tasks with the application. They only got

a short introduction of the task and the reassurance they could do nothing wrong. So

nothing about how the application worked was explained. After this users were

observed completing the tasks, while they were invited to think-aloud. The moments

when users hesitated or were in doubt, were explained as moments in the process the

application should offer help. The outcomes of this test were:

– Information on how the application was used

– Whether or not concepts and descriptions were interpreted as intended

– Functional problems

– Insights in perceptions on what has happened by performing this task (“what

will be the next steps in the organisation?”)

These findings were incorporated in the improved version of the software. It would be

valuable to perform such a test again now the build is complete, however at the

moment there is no time available to do this. Demands for support and help options

are not the same for every user, for instance because of the mentioned differences in

IT-skills. One of the tools for help within the PeopleSoft application is the “See, Try,

Know, Do” principle. Users can first look at a demonstration (see) before trying it

themselves in a simulation mode (try). A next step is then to take a test to check if

they understand everything (know), before finally actually performing the task with

the application (do). Users can use one or more of these functions to support them in

the use of the software.

In choosing people for the tests information managers were asked if they could point

out employees that met certain criteria. One criterion for instance was whether or not

they were very skilled in using IT. Although there were some criteria, no standard

profiles were used to categorise users in groups. To confront these users with the

improved application, during the development, prototypes were used.

3.5 Comparing the Four Projects

The self-service HR-applications Emplaza, P-Loket, P-Direkt and PeopleSoft support

similar (or even exactly the same) processes of the organisations, which makes it

reasonable to compare them. Figure 1 presents the different applications from the

interviews depicted in a diagram that scores them on success (y-axis) and user

participation (x-axis). Their position on both axes is based on the interviews, but is of

course subjectively determined. It is not meant to imply that any of the applications is

‘better’ than the others.

For Emplaza not only the latest version (E2) is represented in the graph, but also

earlier versions (E1). In the interview it became clear that in earlier versions a lot

more user participation was applied. The short communication lines resulting from the

programming team working on the same location as users, made sure users could be

frequently consulted. For Emplaza 4.3 end-users were involved not at all before a

working product was developed. A number of problems arose in the development and

testing of the 4.3 version. Some functionalities could for instance not be implemented

Fig. 1. Participation and application success.

on time, because tests by key users revealed to many hiccups. User satisfaction with

the functionalities that could be implemented on time however, was considered to be

reasonable high. Also, it would not be fair to let the low score on participation of end-

users to be the only reason for the low score on application success. There were

numerous other problems mentioned that contributed to the difficult development of

Emplaza 4.3. Before these problems are solved however, earlier versions can be

considered relatively more successful than the latest one.

Another application that has earlier versions depicted in the chart is the PeopleSoft

HR self-service application of the Ministry of Defence. In the use of the initial version

(D1) users encountered too many problems. So, the second version (D2) was

developed to be an improvement of the first version. A lot more attention for usability

went hand in hand with the increasing possibilities for users to participate in the

development. A lot of problems were therefore found, resulting in quite a lot of

rework. Although not all difficulties for users could be solved the second version was

considered to be superior to its predecessor. This thus shows in participation, so the

second version is placed more to the right of the graph. However, the success

positioning is not as high as might be expected with regard to the amount of user

participation. This mainly has to do with the low scores on contribution to efficiency

and the overall success rating of the application by the respondent.

From all of the different projects, the P-Loket application at the Ministry of

Health, Welfare and Sports displayed relatively the most time and effort spent on user

participation. For instance all managers were approached by demo sessions and

invited to comment on the application. Although the different approaches used are

less than with the PeopleSoft HR application at Ministry of Defense, the relative

amount of time spent is considered to be more. Hence P-Loket (P1) is more to the

right than D2. Besides the delay in the start of the project, not a lot of problems arose

during the development of the application. Also most of the problems users

experienced with the application were caught in the different tests during the

development. The amount of rework to be done, was therefore considered ‘much’, but

it could be done early in the development. Since the application introduced self-

service, it contributed a lot to the efficiency of the organisation. The positioning on

the application success axis therefore is considered to be quite high.

4 Summary and Conclusions

This paper departed from an analysis of current literature on information system

success, user satisfaction and user involvement. Research was explored that described

what factors influence the success of information systems. User participation and

involvement can be regarded as important factors for IS success [37, 38]. This claim

was projected on four employment self service (ESS) projects that were executed

within different Dutch ministries during the last five years. Interviews were held with

civil servants and project managers employed at different governmental organisations.

In line with the claim from literature study, the projects and respondents confirmed

that users should be involved early to increase the success of the software application.

This should not be too early however, because it can also delay the development

process. Both the benefits and the challenges faced with involving users, as deviated

from literature, were also found in the ESS-projects.

A number of important lessons were mentioned by the respondents. The first is

that expectancy management is important, i.e. keep users informed about

developments and motives for certain decisions. Second, it should be kept in mind

that employees should use the self-service applications without too much support

from the HR-departments. Otherwise, it would only be a shift of the workload for the

HR-department from HR-tasks to supporting users with the use of the application. In

that case, the organizations would not show the targeted improvements in efficiency.

A third important aspect to take into account is the distinct decision process

organisations like the Ministry have. This is in line with the main points of the recent

report of The Netherlands Court of Audit. The decision process within governmental

organizations includes fairly a lot of people, takes considerable time and is politically

sensitive.

References

1. Shapiro, D.: Participatory design: the will to succeed. Proceedings of the 4th decennial

conference on Critical computing: between sense and sensibility (2005) 29-38

2. Algemene Rekenkamer: Lessen uit ICT-projecten bij de overhead (2007), http://

www.rekenkamer.nl/9282000/d/p425_rapport1.pdf

3. Noyes, P.M., Starr, A.F., Frankish, C.R.: User involvement in the early stages of an aircraft

warning system. Behaviour & Information Technology 2 (1996) 67-75

4. Chatzoglou, P.D., Macaulay, L.A.: Requirements Capture and Analysis: A Survey of

Current Practice. Requirements Engineering 1 (1996) 75-87

5. Blackburn, J., Scudder, G., van Wassenhove, L.N.: Concurrent Software Development.

Communications of the ACM 11 (2000) 200-214

6. Grudin, J.: Systematic sources of suboptimal interface design in large product development

organization. Human Computer Interaction 2 (1991) 147-196

7. Verheijen, T.: Gestrikt: E-HRM komt er uiteindelijk toch. Personeelsbeleid 11 (2007) 20-23

8. Janssen, M., Joha, A.: Motives for establishing shared service centers in public

administrations. International Journal of Information Management 2 (2006) 102-115

9. Ministerie van Binnenlandse Zaken en Koninkrijksrelaties: Nota Vernieuwing Rijksdienst

(2007), http://www.minbzk.nl/aspx/download.aspx?file=/contents/pages/89897/

notavernieuwingrijksdienst.pdf

10. Damodaran, L.: User involvement in the systems design process-a practical guide for users.

Behaviour & Information Technology 6 (1996) 363 – 377

11. Borland: Effective Requirements Definition And Management Improves Systems And

Communication (2006), http://www.borland.com/resources/en/pdf/solutions/rdm_white

paper.pdf

12. Pekkola, S., Niina, K., Pasi, P.: Towards Formalised End-User Participation in Information

Systems Development Process: Bridging the Gap between Participatory Design and ISD

Methodologies. Proceedings of the ninth Participatory Design Conference (2006) 21-30

13. Kujala, S.; User involvement: a review of the benefits and challenges. Behaviour &

Information Technology 1 (2003) 1-16.

14. Nielsen, J.: Usability Engineering. Academic Press, San Diego (1993)

15. Ives, B., Olson, M.H., Baroudi, J.J.: The measurement of user information satisfaction.

Communications of the ACM 10 (1983) 785-793.

16. Gibson, H.: Determining User Involvement. Journal of System Management (1977) 20-22

17. Lucas, H.J.: Systems Quality, User Reactions, and the Use of Information Systems.

Management Informatics 4 (1974) 207-212

18. Cherry, C., Macredie, R.D.: The importance of Context in Information System Design: An

Assesment of Participatory Design. Requirements Engineering 2 (1999) 103-114

19. Robey, D., Farrow, D.: User Involvement in Information System Development: A Conflict

Model and Empirical Test. Management Science 1 (1982) 73-85

20. Clement, A., van den Besselaar, P.: A retrospective look at PD projects. Communications

of the ACM 4 (1993) 29-37

21. Wood, L.E.: Semi-structured interviewing for user-centered design. Interactions 2 (1997)

48-61

22. Butler, M.B.: Getting to know your users: usability roundtables at Lotus Development.

Interaction 1 (1996) 23-30

23. Brown, D.: The challenges of user-based design in a medical equipment market. In: Wixon,

D., Ramey, J. (eds): Field Methods Casebook for Software Design. Wiley, New York

(1996) 157-176

24. Rowley, D.E.: Organizational considerations in field-oriented product development:

Experiences of a cross-functional team. In: Wixon, D., Ramey, J. (eds.): Methods Casebook

for Software Design. New York: Wiley (1996) 125 – 144

25. Blomquist, Å., Arvola, M.: Personas in action: ethnography in an interaction design team.

Proceedings of the second Nordic conference on HCI (2002) 197 - 200

26. Gould, J.D., Lewis, C.: Designing for Usability: Key Principles and What Designers Think.

Communications of the ACM 3 (1995) 300-311

27. Littlejohn, S.W.: Theories of Human Communication. Thomson Learning, Belmont (2002)

28. Simonsen, J., Kensing, F.: Using ethnography in contextural design. Communications of

the ACM 7 (1997) 82-88

29. Blomberg, J., Giacomi, J., Mosher, A., Swenton-Hall, P.: Ethnographic field methods and

their relation to design. In: Schuler, D., Namioka, A. (eds): Participatory Design: Principles

and Practices. Lawrence Erlbaum, Hillsdale (1993) 123-155

30. Beyer, H., Holtzblatt, K.: Contextual design. Interactions 1 (1993) 32-42

31. Muller, M.; A participatory poster of participatory methods. Conference on Human Factors

in Computing Systems. CHI '01 extended abstracts on Human factors in computing systems

(2001) 99 – 100

32. Ruël, H., Bondarouk, T., Looise, J.K.: E-HRM: Innovation or Irritation. An Explorative

Empirical Study in Five Large Companies on Web-based HRM. Management Revue 3

(2004) 364-380

33. Konradt, U., Christophersen, T., Schaeffer-Kuelz, U.: Predicting user satisfaction, strain

and system usage of employee self-services. International Journal Human Computer studies

11 (2006) 1141-1153

34. Gale, F.S.: Three Stories of Self-Service Success. Workforce 1 (2003) 60-63

35. Venkatesh, V., Davis, F.D.: A Theoretical Extension of the Technology Acceptance Model:

Four Longitudinal Field Studies. Management Science 2 (2000) 186–204

36. Ministerie van Binnenlandse Zaken en Koninkrijksrelaties: Naar een doelmatige salaris- en

Personeelsadministratie. Nieuwe aanpak voor P-Direkt (2007), http://www.p-

direkt.nl/index.cfm?action=dsp_actueelitem&itemid=QKNGJL8E]

37. Barki, H., Hartwick, J.: Rethinking the Concept of User Involvement. Rethinking the

Concept of User Involvement 1 (1989) 53-63

38. DeLone, W.H., McLean, E.R.: The DeLone and McLean Model of Information Success: A

Ten-Year Update. Journal of Management Information Systems 4 (2003) 9-30