Requirements Prioritization by End-users and Consequences

on Design of a Virtual Reality Software

An Exploratory Study

Emilie Loup-Escande and Olivier Christmann

LAMPA, Arts et Métiers ParisTech, 2 Boulevard du Ronceray, Angers, France

Keywords: Prioritization, Functional Requirement, Software Design Process.

Abstract: This paper aims to understand the real contribution of end-users of a virtual reality software to the

requirements prioritization step of the software process and to study how this contribution could influence

design decisions. In our empirical study, we analyzed the lists of functionalities prioritized on a

questionnaire basis and the functionalities evoked spontaneously by users, in the context of the design of a

new software. Results show that several priority levels can be associated with a same functionality within a

single user profile and that priority levels associated with the same functionality may be common or specific

to user profiles. Results also highlight that the additional functionalities proposed by participants are new

features or precisions. In conclusion, we present our research perspective which aims to understand how the

participants perform the prioritization task.

1 INTRODUCTION

Requirements prioritization is a crucial activity in

software design. Numerous research works deal with

the study and the improvement of this part of the

software process, for example in the field of

requirements engineering and design ergonomics.

These works are mainly focused on the description

of several categories of prioritization methods:

nominal scales, ordinal scales and ratio scales (Ma,

2009). Comparison of these methods deals with the

evaluation on usability and effectiveness by

designers (e.g., (Karlsson et al., 1998)) or on their

costs-benefits (e.g., (Christian et al., 2010)); the time

needed to prioritize or the user satisfaction are some

others criteria (e.g., (Ma, 2009)). These works have

in common to only address the requirements

priorization by designers, not by users.

These studies neither aimed to analyze the

consequences of the use of prioritization methods on

design decisions, nor to analyze how the results of

prioritization by future users could impact the design

of the product or the software. Yet, it is now clear

that user involvement in design is beneficial to

provide a better balance between the designed

artefact and the users’ needs (Caelen, 2009). This is

true in standard product design as much as in

software design. This remains the same in a user-

centered perspective, where the user is involved in

the establishment of the needs and / or evaluation of

the artefact (e.g., (Bastien and Scapin, 2004)), as in

an approach called “participatory design” which

considers the user as a co-designer involved in

design decisions (e.g., (Von Hippel, 2005)). This

question underlines the issue of the involvement of

several user profiles in the design process. While

several studies have shown the interest to involve

novice users and expert users in design (e.g.,

(Popovic, 2003)), few empirical studies have sought

to demonstrate the interest of involving end-users

with different constraints, works and backgrounds,

in the design process and even less in the

requirements prioritization activity.

The main objective of this paper is to describe an

empirical study on the involvement of different user

profiles to the requirements prioritization. For that,

we will firstly examine the real contribution of

different profiles of end-users of a virtual reality

(VR) software to the prioritization of functionalities.

Secondly, we want to study the effects of this

prioritization on the design of the software. To this

end and because users can have different priorities

according to their job and background, three profiles

of users have been studied comparatively (stylists,

Loup Escande E. and Christmann O..

Requirements Prioritization by End-users and Consequences on Design of a Virtual Reality Software - An Exploratory Study.

DOI: 10.5220/0004397900050014

In Proceedings of the 8th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE-2013), pages 5-14

ISBN: 978-989-8565-62-4

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

engineers and marketers) relatively to the use of a

nominal method of prioritization.

In the next section, we present a further review

of the literature on requirements prioritization by

different stakeholders, included end-users of a

system, in the literatures of requirements

engineering and ergonomics. Then we describe the

methodology of our empirical study, and we expose

our results which characterize the activity of

prioritization of three users’ profiles of a VR system

and its consequences on the design choices.

2 REQUIREMENTS

PRIORITIZATION IN DESIGN

BY END-USERS

2.1 The “Requirement Prioritization”

Concept According to Stakeholders

Requirement prioritization consists in assigning

different priorities to the requirements in order to

obtain a relative order between them (Berander and

Andrews, 2005) and finally to determine which

requirements should be implemented first (Iqbal et

al., 2009). In that context, the requirements

prioritization is a design activity (Darses et al.,

1996). This prioritization activity takes place in the

selection stage which precedes the technical

realization (Alenljung and Persson, 2008). From this

viewpoint, requirements prioritization is a crucial

step during the software process.

In requirements engineering, the selection is

made by the requirement engineer or the project

manager or even the developer himself, based on

four recommended criteria:

 the technical feasibility of each alternative

(Alenljung and Persson, 2008);

 the degree of uncertainty and risk associated

with each alternative (Aurum and Wohlin,

2003);

 the evaluation of the costs and benefits of each

alternative (Macaulay et al., 1990);

 the degree of convergence of different

stakeholders in a design project for each

alternative (Alenljung and Persson, 2008).

In this literature, the end user does not participate

directly in the selection of alternatives, contrary to

others disciplines where prioritization can be

performed by end-users. In ergonomics, the

selection of alternatives is done – at least partly –

after the production phase of requirements and

candidate specifications. Selected alternatives are

used to produce a model which could evolve, that is,

not directly the final software as in requirement

engineering. In ergonomics as in requirement

engineering, the stages of selection of the

alternatives are prerequisites for achieving technical

realization (Maguire and Bevan, 2002).

Both in the case of an iterative user-centred

design (Bastien and Scapin, 2004) than in the case of

participatory design (Caelen, 2009) results obtained

from the evaluation of the model are elements that

will help to (re-)orient the design choices for a

prototype. Indeed, users are most concerned to

provide designers the necessary criteria to justify the

new design choices in terms of destination of the

artefact, but also in terms of profits and benefits for

them. Moreover, the involvement of users is higher

in a context of participatory design in which users,

as co-designers, have the opportunity to make design

decisions as well as designers. This suggests that

decisions will take into account, at least in part,

benefits for users, and not only designers’

constraints as in requirement engineering. (Loup-

Escande, 2010) demonstrated that the backers’

requirements and users’ needs, moderated by the

designers’ constraints, should be the specifications

for the system design.

2.2 Prioritization Methods and Tools

There are many approaches, tools and methods

recommended or used for prioritizing such as

nominal scales, ordinal scales and ratio scales

(Karlsson et al., 1998) which allow people to assign

qualitative or quantitative values to requirements.

The ‘Attributed Goal-Oriented Requirements

Analysis’ method (AGORA) (Kaiya et al., 2002) is

an example of prioritization method, based on a

graph used to decompose misunderstood goals until

their understanding by each project’ participants. In

nominal scale methods, requirements are assigned to

different priority groups. An example is the

MoScoW method, which consists in grouping all

requirements into four priority groups, that are

requirements that the project (must / should / could /

won’t) have. All requirements listed in a category

are of equal priority, which does not allow a finer

prioritization. Ordinal scales methods produce an

ordered list of requirements. For example, the simple

ranking where the most important requirement is

ranked ‘one’ and the least important is ranked ‘n’.

Simple sorting algorithms of sorting are also well

suited to the requirements prioritization: for

example, the “bubble sort” algorithm permits, by

ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

comparing the relative importance of requirements

in pairs, to obtain a list of ordered requirements

(Aho et al., 1983). Another known method called

Analytic Hierarchy Process asks users to compare all

pairs of requirements (Saaty, 2005). Ratio scale

methods provide relative difference between

requirements (e.g. the hundred dollars method ask

users to allocate a sum of money at each

requirement). In addition to an ordered list of

requirements, this method also helps to know the

relative importance of each requirement in relation

to other ones.

Users of these methods are generally specialists.

Also, methods such as the ‘bubble sort’ remain

difficult to master by people from the general public

(Ma, 2009). Different studies compared these

methods (e.g., (Ma, 2009)): the metric used is the

performance of the user (e.g., the number of

decisions to make, time spent for the prioritization)

or his satisfaction. Nevertheless, these studies are

focused on the designers and elude the users of

software which will contain implemented

requirements.

2.3 Empirical Studies Focused on

Prioritization Methods by

End-users

Prioritization and selection of functionalities can in

some cases be made in two stages. This approach is

sometimes used in ergonomics and adopted in some

design projects to develop innovative software.

Requirements are firstly identified by the engineers

with technical or cost constraints. Then, a panel of

people representing the users is asked to prioritize

the requirements according to their own criteria.

This preliminary selection of the functionalities by

the designers, prior to their prioritization by users,

can result in the removal of functionalities with a

potential high added value for users, because

perceived as complex or costly for designers.

In other projects, requirements prioritization and

selection can be performed by a group composed of

designers and users. The integration of users in the

selection of design choices helps to develop software

in which functionalities and properties of the artefact

will bring a real added value to users. We present

two studies that include end users in requirements

prioritization (Plos et al., 2007; Collinge and

Landry, 1997). These two studies did not explicitly

identify the role of end users in the prioritization

task: they were included in the group as well as

designers and backers. But priority requirements (i.e.

judged important) are different, according to the role

or the status of the project stakeholders (Berander

and Andrews, 2005).

In the first study (Plos et al., 2007), participants

(two users, three therapists, two technicians and a

stylist) had to prioritize functionalities and select

relevant functionalities to develop technical

assistance for person with motor disability. They

prioritized functional specifications by level of

importance and of flexibility. This prioritization

aimed at selecting the “relevant” features by

performing a retrospective subjective assessment of

them.

The second study aimed to clarify the problems

and the needs of different stakeholders affected by

repetitive strain injury in office activities (Collinge

and Landry, 1997). To do this, focus groups

involving ten participants were organized, composed

of various profiles (e.g., users and managers of

private/public companies, ergonomics experts …).

The facilitator asked participants to list problems

and needs related to the theme of ‘repetitive strain

injury and physical organization of the post’, and

other topics that they considered important to

broach. Then, each focus group had to review the

main requirements and prioritize them. Thus,

participants were able to define by themselves the

most important elements in terms of requirements,

which avoided a personal interpretation by the

facilitator of the previous discussions.

In both studies described above, the objective

was not to analyze the activity of requirements

prioritization by users specifically, but rather

realized by a design group including users. The

question of the impact of prioritization on design

decisions is also not addressed. The limits of these

previous empirical studies constitute the interest and

motivation of the study described in this paper.

3 METHODOLOGY

To examine the requirements prioritization by

different profiles of end-users and to evaluate the

implications for the design, we analyzed the lists of

functionalities prioritized on the basis of a

questionnaire and the functionalities evoked

spontaneously by users, in the context of a real

software design project.

3.1 Context: the Project 3D Child

This exploratory study was conducted on an

industrial project named ‘3D Child’. This project,

led by a group of companies specialized in

RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory

Study

accessories for children, was divided into two parts.

This paper covers the second sub-project in which

we were involved. It was entitled ‘3D environments

and places’: the aim was to design a virtual reality

tool for three SMEs (A, B and C) to help them to

evaluate their future products and to reduce time and

cost of designing industrial products especially in

the preliminary phases of design (i.e. prototypes).

The company A, with three sites, is a furniture

manufacturer with 1,050 employees. The company

B, with a presence in fifteen countries, designs baby

products and employs 4,700 employees. The

company C is a cabinet maker, specializing in toys

and employing nine cabinetmakers. This sub-project

resulted in the development of a software dedicated

to the presentation of interactive 3D scenes (children

bedroom and car) composed of future products in

which human characters - modelled in 3D could

move (see Figure 1). This virtual reality tool is

named ‘Appli-Viz’3D’ and is a software made for

decision support in industrial design.

Figure 1: Avatar in the “bedroom” 3D scene (Appli-

viz’3D).

3.2 Participants

This study involved twenty participants, who are

end-users of Appli-Viz’3D: eight engineers (three

from company A and five form company B), eight

stylists (four from company A, three from company

B and one from company C) and four marketers

(two from company A and two from company B).

Most users of Appli-Viz'3D were also designers in

their firms (engineers and stylists). Therefore, they

were able to know or understand the constraints and

technical potentialities of the software, compared

with users who are not designers. This has certainly

facilitated the elicitation of needs (Reich et al.,

1996). Participants were 41.3 years old on average

(S.D. = 7.8; min = 28; max = 60) and 20.1 years of

work experience (S.D. = 8.6; min = 5; max = 37).

3.3 Data Collection: Questionnaire

and Nominal Scale Method

The material used to collect data regarding the

process of the functionalities selection and

prioritization was a questionnaire. The questionnaire

contained a list of functionalities of Appli-Viz'3D

that users had to prioritize (Table 1 – white part).

These functions were taken up from a previous

demonstrator named ‘Virtual Kid’ (see Figure 2) - an

online store that helped to launch the project ‘3D

Child’. This questionnaire was elaborated by non-

engineer designers, because we thought a priori that

the designer-engineer would restrict at once the

range of possibilities.

Figure 2: Virtual Kid demonstrator.

Participants had to prioritize these functionalities

using marks from one to five (one = very important,

two = important, three = moderately important,

four = unimportant, five = useless). We chose the

nominal method of prioritization for its ease of use

that required no learning from users. We also asked

participants to add their proposals of new features

for the future tool.

The filling of the questionnaire was made

following a presentation of the 3D Child project and

a film showing the Virtual Kid application and the

presentation of its functionalities. Virtual Kid movie

should allow users to visualize the early

functionalities on the list given in the questionnaire.

In each company, we grouped the participants in the

same room during the filing of the questionnaire.

3.4 Collected Data

Collected data are prioritized lists of functionalities,

and eventually spontaneously evoked ones. These

last functionalities can be prioritized or not. In the

example shown in Table 1, spontaneously evoked

functionalities (in gray) were not prioritized.

3.5 Analysis Method

To answer to our problematic, we analyze our data

regarding:

 the relations between priority levels and

functionalities;

ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

Table 1: List of functionalities completed by a stylist

(priority levels in green, additional functionalities in grey).

Positioning 3D avatars with ergonomic postures

Moving in the scene

Changing the arrangement of the objects

Seeing the scene with the perspective of a child (small

size)

Defining the dimensions of the environment

Changing the color of the objects

Changing the texture of the objects

Changing the scenery

Seeing avatars moving in the scene

Changing the color of the walls

Allowing the avatar to interact with objects (climbing

up a ladder, opening a drawer)

Integrating ambient object directly linked to the

function of the project

 the relations between priority levels and

proposals of additional functionalities;

 the characterization of the additional

functionalities;

 the consideration of prioritization results by

designers.

3.5.1 Statistical Analysis of the Priority

Levels According to Functionalities

The quantitative analysis is based on numbers

counting. We counted the occurrences of the priority

level given by participants for each functionality

(e.g. 2 = important, see section 3.3.). We added these

numbers for each of the three users’ profiles, then

for all profiles taken into account (engineer, stylist

and marketer). The resulting data tables are

contingency tables are crossing the variable

‘functionality’ with the variable ‘priority level’ for

each profile. We defined the overall strength of the

link between two variables (e.g., priority level) by

calculating the Cramer’s V2. The link is considered

strong for V2 between 0.16 and 1.0, weak for V2

lower than 0.04 and intermediate between the two.

Then we characterized the local strength of the link

between two modalities of these two variables (e.g.,

very important …) by computing relative deviations

(RD) between modalities. There is attraction (i.e.,

similarities between variables) when the RD is

positive and repulsion (i.e., disparities between

variables) when it is negative. The attraction is said

to be remarkable for a RD greater than 0.25. For full

theoretical demonstration, see (Bernard, 2003). RD

are often used in exploratory studies since they

allow a measurement of local associations within a

set of data (e.g., (Anastassova et al., 2005)).

3.5.2 Relations between Priority Levels

and the Presence vs. the Absence of

Additional Functionalities

To analyze the relation between the priority levels

associated with early functionalities and the proposal

or the absence of additional functionalities, we

identified the most frequently used priority level in

each list (that is, the frequency of occurrence of this

level was one point higher than the frequencies of

other levels). When no level was distinguishable in

terms of frequency, we removed the lists in question

(three lists have been removed). A total of seventeen

lists were analyzed. Then we counted the lists based

on most frequently assigned levels and the presence

vs. the absence of additional functionalities.

3.5.3 Analysis of the Proposals of Additional

Functionalities

We studied the content of spontaneously evoked

functionalities to determine if they were really new

or only precisions of the functionalities already

listed. We define new functionalities as they are not

a part of a functionality previously anticipated. A

precision corresponds to a part or a detail of

functionality already proposed in the list (e.g.,

‘integrating avatar of children and adults’ is a

precision of ‘positioning 3D avatars with ergonomic

postures’ because it indicates that among the 3D

models, adults and children should be integrated).

Then we counted the number of new

functionalities and precisions to analyze the

distribution in features spontaneously evoked. To

avoid counting twice a same functionality, we

analyzed qualitatively the functionalities added by

users to identify those that were the same among all

the lists. To do this, we established equivalences

between expressions of functionalities that contained

the same terms or synonyms (e.g., ‘being able to

manipulate and the functions of the products’

(evoked by an engineer) and ‘permitting to use and

to grasp objects’ (expressed by a marketer)).

3.5.4 Consideration of the Prioritizations

by Designers

To analyze the consideration of the users’

prioritization by designers, we tried to identify,

among the prioritized functionalities, those which

were validated and implemented by the designers.

For this, we mapped the priority levels associated

with early functionalities and requirements that have

actually been really implemented in the project.

RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory

Study

4 RESULTS

All the results of our study, corresponding to the

analysis methods presented previously, are

synthesized in the following part then discussed in

the fifth section.

4.1 Priority Functionalities for Users

Not Systematically implemented

by Designers

The analysis of the relations between the two

variables ‘functionality’ and ‘priority level’ shows

an overall intermediate link (V2 = 0.07).

Table 2 summarizes the functionalities (left

column), remarkable attractions (based on relative

deviations) between priority levels and

functionalities, and the state of the functionality at

the end of the project, implemented or not

implemented (grey). For one functionality (i.e.,

“seeing the scene with the perspective of child (small

size)”), no remarkable attraction was observed

(blank line in the table 2).

Table 2: Remarkable attractions between functionalities

and priority levels (X), and state of the functionality at the

end of the project (grey = not implemented).

Functionalities

Useless

Unimportant

Md. important

Important

Very important

Positioning 3D avatars with

ergonomic postures

Moving the scene X

Changing the arrangement of the

objects

Changing the scenery X X

Defining the dimension of the

environment

Changing the color of the walls

X X X

Seeing the scene with the

perspective of a child (small size)

Changing the color of the objects X X

Changing the texture of the objects X X

Seeing avatars moving in the scene X

This table underlines the absence of direct link

between priorities associated by users and the state

of the functionalities at the end of the project

(implemented or not). For instance, the functionality

“positioning 3D avatars with ergonomic postures”,

frequently associated with the ‘very important’

priority level (RD = 1.71) was not fully

implemented. The reason given by the designer-

engineer was related to technical constraints,

particularly in terms of collisions between the avatar

and the 3D model of the product. On the user side,

some frustration has been felt to the delivery of the

artefact. Conversely, the functionality “changing the

color of the walls”, which is characterized by strong

attractions with priority levels ‘moderately

important’ (RD = 0.84), ‘unimportant’ (RD = 0.50)

and ‘useless’ (RD = 0.82) has been implemented.

Finally, some features judged essentially

‘moderately important’ and ‘useless’ have not been

implemented (e.g., “changing the color of the

objects”).

4.2 Different Priority Levels According

to Users’ Profiles

The analysis of the relations between the two

variables ‘functionality’ and ‘priority level’ shows a

strong overall association (V2 = 0.20) for the

engineer profile, an intermediate link (V2 = 0.11) for

the stylist profile and a strong overall link (V2 =

0.24) for the marketer profile.

We detail the priority levels associated with

functionalities for each user profile in Table 3, based

on analysis of relative deviations (i.e., remarkable

attractions are represented by colors).

A first observation is that several priority levels

may be associated with functionality within a same

profile, as shown in Table 3: for example, the

functionality “moving the scene” was associated

with ‘unimportant’ and ‘very important’ levels by

stylists.

A second observation is that the results suggest

that the priority levels associated with a same feature

may be common or specific depending on the profile

of users (see Table 3). For example, priorities given

for the functionality “changing the texture of the

objects” are quite similar among profiles of users,

contrary to priorities assigned to the functionality

“changing the color of the objects”.

Putting in perspective priority levels mostly

assigned by each profile and functionalities really

implemented suggests that there is no qualitative

relation between the implementation of a

functionality and the profile that judges it very

important or important.

ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

Table 3: Remarkable attractions between priority levels

associated to each functionality, according to the profile

(E=Engineers, S=Stylists, M=Marketers).

Functionalities

Profile

Useless

Unimportant

Mod. important

Important

Very important

Positioning 3D avatars with

ergonomic postures

Moving the scene

Changing the arrangement of the

objects

 

Changing the scenery

 

Defining the dimension of the

environment

 

Changing the color of the walls

 

Seeing the scene with the

perspective of a child (small size)

 

Changing the color of the objects

 

Changing the texture of the objects

 

Seeing avatars moving in the scene

 

4.3 The Users Who assigned

‘Important’ and ‘Unimportant’

Priority Levels evoked Additional

Functionalities

The majority of the lists contain proposals of

additional functionalities (12/17, 71% - 3 lists have

been removed, see part 3.5.2).

The analysis of the relations between the two

variables ‘presence or absence of proposal of

additional functionalities’ and ‘priority level’ shows

an overall intermediate link (V2 = 0.13). The

analysis of the relative deviations reveals that the

lists of functionalities with the most of ‘very

important’ priority levels don’t contain any proposal

for additional functionalities (RD = 0.36). For

example, a stylist who assigned seven times the level

‘very important’ did not add any extra functionality.

Similarly, the lists in which the priority level the

most frequently assigned is ‘moderately important’

does not contain any functionality additions (RD =

0.36).

However, when the level ‘important’ is the most

used in a list, this one also contains proposals for

additional functionalities (RD = 0.42). Similarly, the

level ‘moderately important’ is characterized by a

strong attraction with the ‘proposal of additional

functionalities’ (RD = 0.42).

When the level ‘useless’ is the most used in the

lists, they may as well include proposals for

additional functionalities, such as no proposals. For

example, a stylist who assigned five times the mark

‘five’ (useless) did not propose additional

functionalities. Conversely, an engineer proposed

three additional functionalities whereas he mainly

attributed the mark ‘five’ (i.e., useless) to the early

functionalities.

4.4 The Spontaneously evoked

Functionalities are Mainly

Precisions of anticipated

Functionalities

Our data show that thirteen out of twenty

participants evoked additional functionalities: seven

are engineers, three are stylists and three are

marketers.

The thirty-four additional functionalities evoked

by users are distributed as follows: thirteen new

functionalities (62%) and twenty-one precisions

(38%). These thirty-four functionalities are the result

of the sum of functionalities added in each list.

After grouping similar functionalities between

two or more lists, we obtained finally fifteen

additional functionalities: six new ones and nine

precisions. The distribution of the six new

functionalities and nine precisions in terms of

common vs. specific to different user profiles is

presented on the Figure 3.

RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory

Study

Figure 3: Distribution of new functionalities (N) and

precisions (P) common or specific to each profile

(E = Engineer, S = Stylist, M = Marketer).

As shown in Figure 3 concerning the six new

functionalities (N), we observe that:

 One functionality is common to engineers,

stylists and marketers (e.g. ‘allowing the

avatar to interact with objects (climbing up a

ladder, opening a drawer)’, expressed by a

designer);

 One functionality is common to stylists and

marketers (e.g. ‘making a short movie […]’,

evoked by a designer);

 Two functionalities are specific to marketers

(e.g. ‘zooming on specific functions of the

furniture during their use’);

 Two functionalities are specific to engineers

(e.g. ‘measuring spaces on the product’).

Concerning the nine precisions (P), we observe

that:

 One functionality is common to stylists and

engineers (e.g. ‘positioning avatars of

children and adults in a same scene […]’,

evoked by a stylist; ‘positioning the

child/parent pair’, expressed by an engineer);

 Four functionalities are specific to engineers

(e.g. ‘defining standard environments: car

trunk (Mini format), train door, bus, sidewalk,

supermarket check-out’);

 Four functionalities are specific to stylists

(e.g. ‘integrating typical objects: baby's

bottles, changing tables … in the scene to

assess the function of the furniture’).

5 DISCUSSION

The results show that the priority levels associated

with the same functionalities can be common to

several user profiles or different according to user

profiles. We note that several priority levels can be

assigned to one functionality by a same user profile.

This confirms the need to develop consultations

between several profiles of end users, followed by

consultations between designers and end-users

(Reich et al., 1996).

We observe that no marketer evaluated a

functionality as useless, contrary to engineers and

stylists who attributed the level ‘useless’ to some

functionalities. One possible explanation is that

engineers and stylists, who are more familiar with

making design choices, are more at ease to say that a

feature is useless for a given artefact. Marketers

seem to prefer to assume that all the functionalities

anticipated by the designers are useful at first sight,

because none of them assigned the priority level

‘useless’ to the functionalities.

Our data highlight that a functionality considered

as ‘very important’ for users might not be

implemented as is. However, knowing that this

functionality was ‘very important’ for users has

driven designers to find a compromise leading to the

implementation of a part of the functionality.

Conversely, functionalities generally considered as

‘unimportant’ or ‘useless’, but considered as ‘very

important’ by a profile of users in particular, may

have been implemented. These findings suggest that

the functionalities prioritized by the users are a

source of information for designers. Theses ones try

to take them into consideration: they care about the

usefulness of the software. But user priorities are a

set of information among others. That leads

designers to implement functionalities they consider

less costly (in financial and temporal terms), without

leaving aside the prioritizations of the users which

represent more a base of exchange than a list of

functionalities to implement as they are. This

confirms the interest to compose multidisciplinary

team, including several profiles of end-users, in the

early stages of the design process (Tichkiewitch et

al., 1993) and not only in the evaluation phases.

A last result of our study is that the majority of

users evoked additional functionalities following the

prioritization task, especially when users have

attributed most frequently the levels ‘important’ or

‘unimportant’. This evocation of post-prioritization

requirements confirms what was supposed by

(Collinge and Landry, 1997) concerning the interest

of involving users in the prioritization to clarify or

propose new features, beyond their priorities. We

suppose that a possible explanation is that providing

users with some examples of functionalities enable

users to have an initial understanding of the

technological potential of the artefact. Prioritizing

these examples allows them to imagine what could

be the future artefact, and to evoke additional

ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

functionalities. These additional functionalities were

mostly precisions of functionalities already

anticipated and, to a lesser extent, entirely new

functionalities. These last ones were common to

several user profiles (i.e. either common to

engineers, stylists and marketers or to stylists and

marketers) or specific to a profile (in this case,

marketers and engineers). The precisions were

essentially specific to engineers and stylists.

6 CONCLUSIONS

AND PERSPECTIVES

The phase of requirements prioritization step is a

key element of the software process as it will lead to

the selection of functionalities to implement. The

results presented above, and the ensuing discussion,

allow us to make several recommendations to

promote the design of artifacts with a real added

value to the user. A first prerequisite is the

integration of end users in the prioritization phase. In

the field of collaborative engineering, previous

studies have demonstrated that multidisciplinary

design teams are beneficial to the design of products.

The results of the study related in this paper allow us

to go further by claiming that multidisciplinary

teams of users are beneficial to the design of

products which are in fact useful for them. Indeed,

users have additional needs related on their job

profile, which results in different priorities. This is

particularly interesting in a context of

“participatory” design, developed in Living Labs.

Participatory design is based on a strong

involvement of users in the expression of needs or

the imagination of solutions, and on the fact that

users must make decisions as well as designers. The

prioritization phase is essential, because it allows

future users to imagine new functionalities that were

not proposed by the designers. However, giving

users a first list of functionalities is crucial for them

to imagine the future artifact to have food for

thought. These prioritized lists are necessary for to

allow designers to consider both their own

constraints and user needs. This leads them to make

compromises that benefit the real utility of the

artifact to be designed.

Despite the originality of the results presented in

this article, a limitation of our study concerns the

small project and the small sample size. That

justifies the exploratory status of study which let to

obtain trends and not general conclusions about the

differences between stakeholders. A second limit

concerns the absence, in our data collection protocol,

of elicitation interviews following the filling of the

questionnaire. The arranging of such interviews has

not been possible because of constraints concerning

the availability of the participants.

From this limit, a research perspective is to

analyze the requirements prioritization by users

adding to our original protocol elicitation interviews

to know the reasons for assigning a priority level and

how users perform this task. This would allow to

identify finely subjective criteria justifying the levels

assigned to each functionality. For example, we

would then be able to explain why a user who gave

the level ‘unimportant’ to functionality: is it because

he imagined a very infrequent use or because he

guessed he wouldn’t need this functionality (but he

did not dare to give the level ‘useless’)? We would

also understand how each assignment was

performed. Thus, we could know if people gave

‘very important’ level first or if they began with

‘useless’ functionalities. This would show that users

know immediately what would useless or instead

that users a priori know what they would need.

REFERENCES

Aho, A. V., Hopcroft, J. E., Ullman, J. D., 1983. Data

Structures and Algorithms. Addison Wesley.

Alenljung, B., Persson, A., 2008. Portraying the practice

of decision-making in requirements Engineering - A

case of large scale bespoke development.

Requirements Engineering.

Anastassova, M,. Burkhardt, J.-M., Mégard, C., Ehanno,

P., 2005a. Results from a user-centred critical

incidents study for guiding future implementation of

augmented reality in automotive maintenance.

International Journal of Industrial Ergonomics, vol.

35, p. 67-77.

Aurum, A., Wohlin, C., 2003. The fundamental nature of

requirements engineering activities as a decision-

making process. Information and Software

Technology, vol. 45, p. 945-954.

Bastien, J. M. C. and Scapin, D., 2004. La conception de

logiciels interactifs centrée sur l'utilisateur : étapes et

methodes. Falzon, P. (Ed.) Ergonomie. Paris, PUF.

Berander, P., Andrews, A., 2005. Requirements

Prioritization. Aurum, A.,Wohlin, C. (Eds.)

Engineering and Managing Software Requirements,

Springer Verlag.

Bernard, J.-M., 2003. Analysis of local or asymmetric

dependencies in contingency tables using the

imprecise Dirichlet model. International symposium

on imprecise probabilities and their applications,

(Lugano) Switzerland.

RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory

Study

Caelen, J., 2009. Conception participative par ‘moments’ :

une gestion collaborative. Le travail humain, vol. 72,

p. 79-103.

Christian, T., Mead, N. R., 2010. An Evaluation of Cost-

Benefit Using Security Requirements Prioritization

Methods. U.S. Department of Homeland Security.

Collinge, C., Landry, R., 1997. Prévention des troubles

musculosquelettiques associés à la bureautique :

analyse des besoins et portrait de la formation et de

l'information. IRSST (Ed.).

Darses, F., Falzon, P., Béguin, P., 1996. Collective design

processes. Second International Conference on the

Design of Cooperative Systems, Sophia-Antipolis

(France), INRIA.

Iqbal, A., Khan, F. M., Khan, S. A., 2009. A critical

analysis of techniques for requirement prioritization

and open research issues. International Journal of

Reviews in Computing vol. 1, p. 8-18.

Kaiya, H., Horai, H., Saeki, M., 2002. AGORA: attributed

goal-oriented requirements analysis method. IEEE

Joint International Conference on Requirements

Engineering.

Kaiya, H., Shinbara, D. Kawano, J., Saeki, M., 2005.

Improving the detection of requirements discordances

among stakeholders. Requirements Engineering, 10,

p. 289-303.

Karlsson, J., Wohlin, C., Regnell, B., 1998. An evaluation

of methods for prioritizing software requirements.

Information and Software Technology, vol. 39,

p. 939 - 947.

Loup-Escande, E., 2010. Vers une conception centrée sur

l'utilité : une analyse de la co-construction

participative et continue des besoins dans le contexte

des technologies émergentes. Thèse de doctorat,

Université d'Angers.

Ma, Q., 2009. The Effectiveness of Requirements

Prioritization Techniques for a Medium to Large

Number of Requirements: A Systematic Literature

Review. Master degree, Auckland University of

Technology.

Macaulay, L., Fowler, C., Kirby, M., Hutt, A., 1990.

USTM: a new approach to requirements specification.

Interacting with Computers, vol. 2, p. 92-118.

Maguire, M., Bevan, N., 2002. User Requirements

Analysis: A Review of Supporting Methods. IFIP 17th

World Computer Congress, Kluwer, B.V.

Plos, O., Buisine, S., Aoussat, A., Dumas, C., 2007.

Analysis and translation of user needs for assistive

technology design. International Conference on

Engineering Design.

Popovic, V., 2003. Expert and Novice Users Model and

their Application to the Design Process. Journal of the

Asian Design International Conference, Tsukuba,

Japan.

Reich, Y., Konda, S. L., Monarch, I. A., Levy, S. N.,

Subrahmanian, E., 1996. Varieties and issues of

participation and design. Design Studies, vol. 17, p.

165-180.

Saaty, T. L., 2005. Analytic Hierarchy Process. In Sons, J.

W. (Ed.) Encyclopedia of Biostatistics

Tichkiewitch, S., Tiger, A., Jeantet, A., 1993. Ingénierie

simultanée dans la conception de produits. Universités

d'été du pôle productique Rhône-Alpes.

Von Hippel, E., 2005. Democratizing Innovation. MIT

Press.

ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering