Requirements Prioritization by End-users and Consequences
on Design of a Virtual Reality Software
An Exploratory Study
Emilie Loup-Escande and Olivier Christmann
LAMPA, Arts et Métiers ParisTech, 2 Boulevard du Ronceray, Angers, France
Keywords: Prioritization, Functional Requirement, Software Design Process.
Abstract: This paper aims to understand the real contribution of end-users of a virtual reality software to the
requirements prioritization step of the software process and to study how this contribution could influence
design decisions. In our empirical study, we analyzed the lists of functionalities prioritized on a
questionnaire basis and the functionalities evoked spontaneously by users, in the context of the design of a
new software. Results show that several priority levels can be associated with a same functionality within a
single user profile and that priority levels associated with the same functionality may be common or specific
to user profiles. Results also highlight that the additional functionalities proposed by participants are new
features or precisions. In conclusion, we present our research perspective which aims to understand how the
participants perform the prioritization task.
1 INTRODUCTION
Requirements prioritization is a crucial activity in
software design. Numerous research works deal with
the study and the improvement of this part of the
software process, for example in the field of
requirements engineering and design ergonomics.
These works are mainly focused on the description
of several categories of prioritization methods:
nominal scales, ordinal scales and ratio scales (Ma,
2009). Comparison of these methods deals with the
evaluation on usability and effectiveness by
designers (e.g., (Karlsson et al., 1998)) or on their
costs-benefits (e.g., (Christian et al., 2010)); the time
needed to prioritize or the user satisfaction are some
others criteria (e.g., (Ma, 2009)). These works have
in common to only address the requirements
priorization by designers, not by users.
These studies neither aimed to analyze the
consequences of the use of prioritization methods on
design decisions, nor to analyze how the results of
prioritization by future users could impact the design
of the product or the software. Yet, it is now clear
that user involvement in design is beneficial to
provide a better balance between the designed
artefact and the users’ needs (Caelen, 2009). This is
true in standard product design as much as in
software design. This remains the same in a user-
centered perspective, where the user is involved in
the establishment of the needs and / or evaluation of
the artefact (e.g., (Bastien and Scapin, 2004)), as in
an approach called “participatory design” which
considers the user as a co-designer involved in
design decisions (e.g., (Von Hippel, 2005)). This
question underlines the issue of the involvement of
several user profiles in the design process. While
several studies have shown the interest to involve
novice users and expert users in design (e.g.,
(Popovic, 2003)), few empirical studies have sought
to demonstrate the interest of involving end-users
with different constraints, works and backgrounds,
in the design process and even less in the
requirements prioritization activity.
The main objective of this paper is to describe an
empirical study on the involvement of different user
profiles to the requirements prioritization. For that,
we will firstly examine the real contribution of
different profiles of end-users of a virtual reality
(VR) software to the prioritization of functionalities.
Secondly, we want to study the effects of this
prioritization on the design of the software. To this
end and because users can have different priorities
according to their job and background, three profiles
of users have been studied comparatively (stylists,
5
Loup Escande E. and Christmann O..
Requirements Prioritization by End-users and Consequences on Design of a Virtual Reality Software - An Exploratory Study.
DOI: 10.5220/0004397900050014
In Proceedings of the 8th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE-2013), pages 5-14
ISBN: 978-989-8565-62-4
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
engineers and marketers) relatively to the use of a
nominal method of prioritization.
In the next section, we present a further review
of the literature on requirements prioritization by
different stakeholders, included end-users of a
system, in the literatures of requirements
engineering and ergonomics. Then we describe the
methodology of our empirical study, and we expose
our results which characterize the activity of
prioritization of three users’ profiles of a VR system
and its consequences on the design choices.
2 REQUIREMENTS
PRIORITIZATION IN DESIGN
BY END-USERS
2.1 The “Requirement Prioritization”
Concept According to Stakeholders
Requirement prioritization consists in assigning
different priorities to the requirements in order to
obtain a relative order between them (Berander and
Andrews, 2005) and finally to determine which
requirements should be implemented first (Iqbal et
al., 2009). In that context, the requirements
prioritization is a design activity (Darses et al.,
1996). This prioritization activity takes place in the
selection stage which precedes the technical
realization (Alenljung and Persson, 2008). From this
viewpoint, requirements prioritization is a crucial
step during the software process.
In requirements engineering, the selection is
made by the requirement engineer or the project
manager or even the developer himself, based on
four recommended criteria:
the technical feasibility of each alternative
(Alenljung and Persson, 2008);
the degree of uncertainty and risk associated
with each alternative (Aurum and Wohlin,
2003);
the evaluation of the costs and benefits of each
alternative (Macaulay et al., 1990);
the degree of convergence of different
stakeholders in a design project for each
alternative (Alenljung and Persson, 2008).
In this literature, the end user does not participate
directly in the selection of alternatives, contrary to
others disciplines where prioritization can be
performed by end-users. In ergonomics, the
selection of alternatives is done – at least partly –
after the production phase of requirements and
candidate specifications. Selected alternatives are
used to produce a model which could evolve, that is,
not directly the final software as in requirement
engineering. In ergonomics as in requirement
engineering, the stages of selection of the
alternatives are prerequisites for achieving technical
realization (Maguire and Bevan, 2002).
Both in the case of an iterative user-centred
design (Bastien and Scapin, 2004) than in the case of
participatory design (Caelen, 2009) results obtained
from the evaluation of the model are elements that
will help to (re-)orient the design choices for a
prototype. Indeed, users are most concerned to
provide designers the necessary criteria to justify the
new design choices in terms of destination of the
artefact, but also in terms of profits and benefits for
them. Moreover, the involvement of users is higher
in a context of participatory design in which users,
as co-designers, have the opportunity to make design
decisions as well as designers. This suggests that
decisions will take into account, at least in part,
benefits for users, and not only designers’
constraints as in requirement engineering. (Loup-
Escande, 2010) demonstrated that the backers’
requirements and users’ needs, moderated by the
designers’ constraints, should be the specifications
for the system design.
2.2 Prioritization Methods and Tools
There are many approaches, tools and methods
recommended or used for prioritizing such as
nominal scales, ordinal scales and ratio scales
(Karlsson et al., 1998) which allow people to assign
qualitative or quantitative values to requirements.
The ‘Attributed Goal-Oriented Requirements
Analysis’ method (AGORA) (Kaiya et al., 2002) is
an example of prioritization method, based on a
graph used to decompose misunderstood goals until
their understanding by each project’ participants. In
nominal scale methods, requirements are assigned to
different priority groups. An example is the
MoScoW method, which consists in grouping all
requirements into four priority groups, that are
requirements that the project (must / should / could /
won’t) have. All requirements listed in a category
are of equal priority, which does not allow a finer
prioritization. Ordinal scales methods produce an
ordered list of requirements. For example, the simple
ranking where the most important requirement is
ranked ‘one’ and the least important is ranked ‘n’.
Simple sorting algorithms of sorting are also well
suited to the requirements prioritization: for
example, the “bubble sort” algorithm permits, by
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering
6
comparing the relative importance of requirements
in pairs, to obtain a list of ordered requirements
(Aho et al., 1983). Another known method called
Analytic Hierarchy Process asks users to compare all
pairs of requirements (Saaty, 2005). Ratio scale
methods provide relative difference between
requirements (e.g. the hundred dollars method ask
users to allocate a sum of money at each
requirement). In addition to an ordered list of
requirements, this method also helps to know the
relative importance of each requirement in relation
to other ones.
Users of these methods are generally specialists.
Also, methods such as the ‘bubble sort’ remain
difficult to master by people from the general public
(Ma, 2009). Different studies compared these
methods (e.g., (Ma, 2009)): the metric used is the
performance of the user (e.g., the number of
decisions to make, time spent for the prioritization)
or his satisfaction. Nevertheless, these studies are
focused on the designers and elude the users of
software which will contain implemented
requirements.
2.3 Empirical Studies Focused on
Prioritization Methods by
End-users
Prioritization and selection of functionalities can in
some cases be made in two stages. This approach is
sometimes used in ergonomics and adopted in some
design projects to develop innovative software.
Requirements are firstly identified by the engineers
with technical or cost constraints. Then, a panel of
people representing the users is asked to prioritize
the requirements according to their own criteria.
This preliminary selection of the functionalities by
the designers, prior to their prioritization by users,
can result in the removal of functionalities with a
potential high added value for users, because
perceived as complex or costly for designers.
In other projects, requirements prioritization and
selection can be performed by a group composed of
designers and users. The integration of users in the
selection of design choices helps to develop software
in which functionalities and properties of the artefact
will bring a real added value to users. We present
two studies that include end users in requirements
prioritization (Plos et al., 2007; Collinge and
Landry, 1997). These two studies did not explicitly
identify the role of end users in the prioritization
task: they were included in the group as well as
designers and backers. But priority requirements (i.e.
judged important) are different, according to the role
or the status of the project stakeholders (Berander
and Andrews, 2005).
In the first study (Plos et al., 2007), participants
(two users, three therapists, two technicians and a
stylist) had to prioritize functionalities and select
relevant functionalities to develop technical
assistance for person with motor disability. They
prioritized functional specifications by level of
importance and of flexibility. This prioritization
aimed at selecting the “relevant” features by
performing a retrospective subjective assessment of
them.
The second study aimed to clarify the problems
and the needs of different stakeholders affected by
repetitive strain injury in office activities (Collinge
and Landry, 1997). To do this, focus groups
involving ten participants were organized, composed
of various profiles (e.g., users and managers of
private/public companies, ergonomics experts …).
The facilitator asked participants to list problems
and needs related to the theme of ‘repetitive strain
injury and physical organization of the post’, and
other topics that they considered important to
broach. Then, each focus group had to review the
main requirements and prioritize them. Thus,
participants were able to define by themselves the
most important elements in terms of requirements,
which avoided a personal interpretation by the
facilitator of the previous discussions.
In both studies described above, the objective
was not to analyze the activity of requirements
prioritization by users specifically, but rather
realized by a design group including users. The
question of the impact of prioritization on design
decisions is also not addressed. The limits of these
previous empirical studies constitute the interest and
motivation of the study described in this paper.
3 METHODOLOGY
To examine the requirements prioritization by
different profiles of end-users and to evaluate the
implications for the design, we analyzed the lists of
functionalities prioritized on the basis of a
questionnaire and the functionalities evoked
spontaneously by users, in the context of a real
software design project.
3.1 Context: the Project 3D Child
This exploratory study was conducted on an
industrial project named ‘3D Child’. This project,
led by a group of companies specialized in
RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory
Study
7
accessories for children, was divided into two parts.
This paper covers the second sub-project in which
we were involved. It was entitled ‘3D environments
and places’: the aim was to design a virtual reality
tool for three SMEs (A, B and C) to help them to
evaluate their future products and to reduce time and
cost of designing industrial products especially in
the preliminary phases of design (i.e. prototypes).
The company A, with three sites, is a furniture
manufacturer with 1,050 employees. The company
B, with a presence in fifteen countries, designs baby
products and employs 4,700 employees. The
company C is a cabinet maker, specializing in toys
and employing nine cabinetmakers. This sub-project
resulted in the development of a software dedicated
to the presentation of interactive 3D scenes (children
bedroom and car) composed of future products in
which human characters - modelled in 3D could
move (see Figure 1). This virtual reality tool is
named ‘Appli-Viz’3D’ and is a software made for
decision support in industrial design.
Figure 1: Avatar in the “bedroom” 3D scene (Appli-
viz’3D).
3.2 Participants
This study involved twenty participants, who are
end-users of Appli-Viz’3D: eight engineers (three
from company A and five form company B), eight
stylists (four from company A, three from company
B and one from company C) and four marketers
(two from company A and two from company B).
Most users of Appli-Viz'3D were also designers in
their firms (engineers and stylists). Therefore, they
were able to know or understand the constraints and
technical potentialities of the software, compared
with users who are not designers. This has certainly
facilitated the elicitation of needs (Reich et al.,
1996). Participants were 41.3 years old on average
(S.D. = 7.8; min = 28; max = 60) and 20.1 years of
work experience (S.D. = 8.6; min = 5; max = 37).
3.3 Data Collection: Questionnaire
and Nominal Scale Method
The material used to collect data regarding the
process of the functionalities selection and
prioritization was a questionnaire. The questionnaire
contained a list of functionalities of Appli-Viz'3D
that users had to prioritize (Table 1 – white part).
These functions were taken up from a previous
demonstrator named ‘Virtual Kid’ (see Figure 2) - an
online store that helped to launch the project ‘3D
Child’. This questionnaire was elaborated by non-
engineer designers, because we thought a priori that
the designer-engineer would restrict at once the
range of possibilities.
Figure 2: Virtual Kid demonstrator.
Participants had to prioritize these functionalities
using marks from one to five (one = very important,
two = important, three = moderately important,
four = unimportant, five = useless). We chose the
nominal method of prioritization for its ease of use
that required no learning from users. We also asked
participants to add their proposals of new features
for the future tool.
The filling of the questionnaire was made
following a presentation of the 3D Child project and
a film showing the Virtual Kid application and the
presentation of its functionalities. Virtual Kid movie
should allow users to visualize the early
functionalities on the list given in the questionnaire.
In each company, we grouped the participants in the
same room during the filing of the questionnaire.
3.4 Collected Data
Collected data are prioritized lists of functionalities,
and eventually spontaneously evoked ones. These
last functionalities can be prioritized or not. In the
example shown in Table 1, spontaneously evoked
functionalities (in gray) were not prioritized.
3.5 Analysis Method
To answer to our problematic, we analyze our data
regarding:
the relations between priority levels and
functionalities;
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering
8
Table 1: List of functionalities completed by a stylist
(priority levels in green, additional functionalities in grey).
1
Positioning 3D avatars with ergonomic postures
2
Moving in the scene
3
Changing the arrangement of the objects
2
Seeing the scene with the perspective of a child (small
size)
2
Defining the dimensions of the environment
5
Changing the color of the objects
5
Changing the texture of the objects
3
Changing the scenery
2
Seeing avatars moving in the scene
3
Changing the color of the walls
Allowing the avatar to interact with objects (climbing
up a ladder, opening a drawer)
Integrating ambient object directly linked to the
function of the project
the relations between priority levels and
proposals of additional functionalities;
the characterization of the additional
functionalities;
the consideration of prioritization results by
designers.
3.5.1 Statistical Analysis of the Priority
Levels According to Functionalities
The quantitative analysis is based on numbers
counting. We counted the occurrences of the priority
level given by participants for each functionality
(e.g. 2 = important, see section 3.3.). We added these
numbers for each of the three users’ profiles, then
for all profiles taken into account (engineer, stylist
and marketer). The resulting data tables are
contingency tables are crossing the variable
‘functionality’ with the variable ‘priority level’ for
each profile. We defined the overall strength of the
link between two variables (e.g., priority level) by
calculating the Cramer’s V2. The link is considered
strong for V2 between 0.16 and 1.0, weak for V2
lower than 0.04 and intermediate between the two.
Then we characterized the local strength of the link
between two modalities of these two variables (e.g.,
very important …) by computing relative deviations
(RD) between modalities. There is attraction (i.e.,
similarities between variables) when the RD is
positive and repulsion (i.e., disparities between
variables) when it is negative. The attraction is said
to be remarkable for a RD greater than 0.25. For full
theoretical demonstration, see (Bernard, 2003). RD
are often used in exploratory studies since they
allow a measurement of local associations within a
set of data (e.g., (Anastassova et al., 2005)).
3.5.2 Relations between Priority Levels
and the Presence vs. the Absence of
Additional Functionalities
To analyze the relation between the priority levels
associated with early functionalities and the proposal
or the absence of additional functionalities, we
identified the most frequently used priority level in
each list (that is, the frequency of occurrence of this
level was one point higher than the frequencies of
other levels). When no level was distinguishable in
terms of frequency, we removed the lists in question
(three lists have been removed). A total of seventeen
lists were analyzed. Then we counted the lists based
on most frequently assigned levels and the presence
vs. the absence of additional functionalities.
3.5.3 Analysis of the Proposals of Additional
Functionalities
We studied the content of spontaneously evoked
functionalities to determine if they were really new
or only precisions of the functionalities already
listed. We define new functionalities as they are not
a part of a functionality previously anticipated. A
precision corresponds to a part or a detail of
functionality already proposed in the list (e.g.,
integrating avatar of children and adults’ is a
precision of ‘positioning 3D avatars with ergonomic
postures’ because it indicates that among the 3D
models, adults and children should be integrated).
Then we counted the number of new
functionalities and precisions to analyze the
distribution in features spontaneously evoked. To
avoid counting twice a same functionality, we
analyzed qualitatively the functionalities added by
users to identify those that were the same among all
the lists. To do this, we established equivalences
between expressions of functionalities that contained
the same terms or synonyms (e.g., being able to
manipulate and the functions of the products
(evoked by an engineer) and ‘permitting to use and
to grasp objects’ (expressed by a marketer)).
3.5.4 Consideration of the Prioritizations
by Designers
To analyze the consideration of the users’
prioritization by designers, we tried to identify,
among the prioritized functionalities, those which
were validated and implemented by the designers.
For this, we mapped the priority levels associated
with early functionalities and requirements that have
actually been really implemented in the project.
RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory
Study
9
4 RESULTS
All the results of our study, corresponding to the
analysis methods presented previously, are
synthesized in the following part then discussed in
the fifth section.
4.1 Priority Functionalities for Users
Not Systematically implemented
by Designers
The analysis of the relations between the two
variables ‘functionality’ and ‘priority level’ shows
an overall intermediate link (V2 = 0.07).
Table 2 summarizes the functionalities (left
column), remarkable attractions (based on relative
deviations) between priority levels and
functionalities, and the state of the functionality at
the end of the project, implemented or not
implemented (grey). For one functionality (i.e.,
seeing the scene with the perspective of child (small
size)”), no remarkable attraction was observed
(blank line in the table 2).
Table 2: Remarkable attractions between functionalities
and priority levels (X), and state of the functionality at the
end of the project (grey = not implemented).
Functionalities
Useless
Unimportant
Md. important
Important
Very important
Positioning 3D avatars with
ergonomic postures
X
Moving the scene X
Changing the arrangement of the
objects
X
Changing the scenery X X
Defining the dimension of the
environment
X
Changing the color of the walls
X X X
Seeing the scene with the
perspective of a child (small size)
Changing the color of the objects X X
Changing the texture of the objects X X
Seeing avatars moving in the scene X
This table underlines the absence of direct link
between priorities associated by users and the state
of the functionalities at the end of the project
(implemented or not). For instance, the functionality
positioning 3D avatars with ergonomic postures”,
frequently associated with the ‘very important’
priority level (RD = 1.71) was not fully
implemented. The reason given by the designer-
engineer was related to technical constraints,
particularly in terms of collisions between the avatar
and the 3D model of the product. On the user side,
some frustration has been felt to the delivery of the
artefact. Conversely, the functionality “changing the
color of the walls”, which is characterized by strong
attractions with priority levels ‘moderately
important’ (RD = 0.84), ‘unimportant’ (RD = 0.50)
and ‘useless’ (RD = 0.82) has been implemented.
Finally, some features judged essentially
‘moderately important’ and ‘useless’ have not been
implemented (e.g., “changing the color of the
objects”).
4.2 Different Priority Levels According
to Users’ Profiles
The analysis of the relations between the two
variables ‘functionality’ and ‘priority level’ shows a
strong overall association (V2 = 0.20) for the
engineer profile, an intermediate link (V2 = 0.11) for
the stylist profile and a strong overall link (V2 =
0.24) for the marketer profile.
We detail the priority levels associated with
functionalities for each user profile in Table 3, based
on analysis of relative deviations (i.e., remarkable
attractions are represented by colors).
A first observation is that several priority levels
may be associated with functionality within a same
profile, as shown in Table 3: for example, the
functionality “moving the scene” was associated
with ‘unimportant’ and ‘very important’ levels by
stylists.
A second observation is that the results suggest
that the priority levels associated with a same feature
may be common or specific depending on the profile
of users (see Table 3). For example, priorities given
for the functionality “changing the texture of the
objects” are quite similar among profiles of users,
contrary to priorities assigned to the functionality
changing the color of the objects”.
Putting in perspective priority levels mostly
assigned by each profile and functionalities really
implemented suggests that there is no qualitative
relation between the implementation of a
functionality and the profile that judges it very
important or important.
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering
10
Table 3: Remarkable attractions between priority levels
associated to each functionality, according to the profile
(E=Engineers, S=Stylists, M=Marketers).
Functionalities
Profile
Useless
Unimportant
Mod. important
Important
Very important
Positioning 3D avatars with
ergonomic postures
E
S
M
Moving the scene
E
S
M
Changing the arrangement of the
objects
E
S
M
Changing the scenery
E
S
M
Defining the dimension of the
environment
E
S
M
Changing the color of the walls
E
S
M
Seeing the scene with the
perspective of a child (small size)
E
S
M
Changing the color of the objects
E
S
M
Changing the texture of the objects
E
S
M
Seeing avatars moving in the scene
E
S
M
4.3 The Users Who assigned
‘Important’ and ‘Unimportant’
Priority Levels evoked Additional
Functionalities
The majority of the lists contain proposals of
additional functionalities (12/17, 71% - 3 lists have
been removed, see part 3.5.2).
The analysis of the relations between the two
variables ‘presence or absence of proposal of
additional functionalities’ and ‘priority level’ shows
an overall intermediate link (V2 = 0.13). The
analysis of the relative deviations reveals that the
lists of functionalities with the most of ‘very
important’ priority levels don’t contain any proposal
for additional functionalities (RD = 0.36). For
example, a stylist who assigned seven times the level
‘very important’ did not add any extra functionality.
Similarly, the lists in which the priority level the
most frequently assigned is ‘moderately important
does not contain any functionality additions (RD =
0.36).
However, when the level ‘important’ is the most
used in a list, this one also contains proposals for
additional functionalities (RD = 0.42). Similarly, the
level ‘moderately important’ is characterized by a
strong attraction with the ‘proposal of additional
functionalities’ (RD = 0.42).
When the level ‘useless’ is the most used in the
lists, they may as well include proposals for
additional functionalities, such as no proposals. For
example, a stylist who assigned five times the mark
‘five’ (useless) did not propose additional
functionalities. Conversely, an engineer proposed
three additional functionalities whereas he mainly
attributed the mark ‘five’ (i.e., useless) to the early
functionalities.
4.4 The Spontaneously evoked
Functionalities are Mainly
Precisions of anticipated
Functionalities
Our data show that thirteen out of twenty
participants evoked additional functionalities: seven
are engineers, three are stylists and three are
marketers.
The thirty-four additional functionalities evoked
by users are distributed as follows: thirteen new
functionalities (62%) and twenty-one precisions
(38%). These thirty-four functionalities are the result
of the sum of functionalities added in each list.
After grouping similar functionalities between
two or more lists, we obtained finally fifteen
additional functionalities: six new ones and nine
precisions. The distribution of the six new
functionalities and nine precisions in terms of
common vs. specific to different user profiles is
presented on the Figure 3.
RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory
Study
11
Figure 3: Distribution of new functionalities (N) and
precisions (P) common or specific to each profile
(E = Engineer, S = Stylist, M = Marketer).
As shown in Figure 3 concerning the six new
functionalities (N), we observe that:
One functionality is common to engineers,
stylists and marketers (e.g. ‘allowing the
avatar to interact with objects (climbing up a
ladder, opening a drawer)’, expressed by a
designer);
One functionality is common to stylists and
marketers (e.g. ‘making a short movie […]’,
evoked by a designer);
Two functionalities are specific to marketers
(e.g. ‘zooming on specific functions of the
furniture during their use’);
Two functionalities are specific to engineers
(e.g. ‘measuring spaces on the product’).
Concerning the nine precisions (P), we observe
that:
One functionality is common to stylists and
engineers (e.g. ‘positioning avatars of
children and adults in a same scene […]’,
evoked by a stylist; ‘positioning the
child/parent pair’, expressed by an engineer);
Four functionalities are specific to engineers
(e.g. ‘defining standard environments: car
trunk (Mini format), train door, bus, sidewalk,
supermarket check-out’);
Four functionalities are specific to stylists
(e.g. ‘integrating typical objects: baby's
bottles, changing tables … in the scene to
assess the function of the furniture’).
5 DISCUSSION
The results show that the priority levels associated
with the same functionalities can be common to
several user profiles or different according to user
profiles. We note that several priority levels can be
assigned to one functionality by a same user profile.
This confirms the need to develop consultations
between several profiles of end users, followed by
consultations between designers and end-users
(Reich et al., 1996).
We observe that no marketer evaluated a
functionality as useless, contrary to engineers and
stylists who attributed the level ‘useless’ to some
functionalities. One possible explanation is that
engineers and stylists, who are more familiar with
making design choices, are more at ease to say that a
feature is useless for a given artefact. Marketers
seem to prefer to assume that all the functionalities
anticipated by the designers are useful at first sight,
because none of them assigned the priority level
‘useless’ to the functionalities.
Our data highlight that a functionality considered
as ‘very important’ for users might not be
implemented as is. However, knowing that this
functionality was ‘very important’ for users has
driven designers to find a compromise leading to the
implementation of a part of the functionality.
Conversely, functionalities generally considered as
‘unimportant’ or ‘useless’, but considered as ‘very
important’ by a profile of users in particular, may
have been implemented. These findings suggest that
the functionalities prioritized by the users are a
source of information for designers. Theses ones try
to take them into consideration: they care about the
usefulness of the software. But user priorities are a
set of information among others. That leads
designers to implement functionalities they consider
less costly (in financial and temporal terms), without
leaving aside the prioritizations of the users which
represent more a base of exchange than a list of
functionalities to implement as they are. This
confirms the interest to compose multidisciplinary
team, including several profiles of end-users, in the
early stages of the design process (Tichkiewitch et
al., 1993) and not only in the evaluation phases.
A last result of our study is that the majority of
users evoked additional functionalities following the
prioritization task, especially when users have
attributed most frequently the levels ‘important’ or
‘unimportant’. This evocation of post-prioritization
requirements confirms what was supposed by
(Collinge and Landry, 1997) concerning the interest
of involving users in the prioritization to clarify or
propose new features, beyond their priorities. We
suppose that a possible explanation is that providing
users with some examples of functionalities enable
users to have an initial understanding of the
technological potential of the artefact. Prioritizing
these examples allows them to imagine what could
be the future artefact, and to evoke additional
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering
12
functionalities. These additional functionalities were
mostly precisions of functionalities already
anticipated and, to a lesser extent, entirely new
functionalities. These last ones were common to
several user profiles (i.e. either common to
engineers, stylists and marketers or to stylists and
marketers) or specific to a profile (in this case,
marketers and engineers). The precisions were
essentially specific to engineers and stylists.
6 CONCLUSIONS
AND PERSPECTIVES
The phase of requirements prioritization step is a
key element of the software process as it will lead to
the selection of functionalities to implement. The
results presented above, and the ensuing discussion,
allow us to make several recommendations to
promote the design of artifacts with a real added
value to the user. A first prerequisite is the
integration of end users in the prioritization phase. In
the field of collaborative engineering, previous
studies have demonstrated that multidisciplinary
design teams are beneficial to the design of products.
The results of the study related in this paper allow us
to go further by claiming that multidisciplinary
teams of users are beneficial to the design of
products which are in fact useful for them. Indeed,
users have additional needs related on their job
profile, which results in different priorities. This is
particularly interesting in a context of
“participatory” design, developed in Living Labs.
Participatory design is based on a strong
involvement of users in the expression of needs or
the imagination of solutions, and on the fact that
users must make decisions as well as designers. The
prioritization phase is essential, because it allows
future users to imagine new functionalities that were
not proposed by the designers. However, giving
users a first list of functionalities is crucial for them
to imagine the future artifact to have food for
thought. These prioritized lists are necessary for to
allow designers to consider both their own
constraints and user needs. This leads them to make
compromises that benefit the real utility of the
artifact to be designed.
Despite the originality of the results presented in
this article, a limitation of our study concerns the
small project and the small sample size. That
justifies the exploratory status of study which let to
obtain trends and not general conclusions about the
differences between stakeholders. A second limit
concerns the absence, in our data collection protocol,
of elicitation interviews following the filling of the
questionnaire. The arranging of such interviews has
not been possible because of constraints concerning
the availability of the participants.
From this limit, a research perspective is to
analyze the requirements prioritization by users
adding to our original protocol elicitation interviews
to know the reasons for assigning a priority level and
how users perform this task. This would allow to
identify finely subjective criteria justifying the levels
assigned to each functionality. For example, we
would then be able to explain why a user who gave
the level ‘unimportant’ to functionality: is it because
he imagined a very infrequent use or because he
guessed he wouldn’t need this functionality (but he
did not dare to give the level ‘useless’)? We would
also understand how each assignment was
performed. Thus, we could know if people gave
‘very important’ level first or if they began with
‘useless’ functionalities. This would show that users
know immediately what would useless or instead
that users a priori know what they would need.
REFERENCES
Aho, A. V., Hopcroft, J. E., Ullman, J. D., 1983. Data
Structures and Algorithms. Addison Wesley.
Alenljung, B., Persson, A., 2008. Portraying the practice
of decision-making in requirements Engineering - A
case of large scale bespoke development.
Requirements Engineering.
Anastassova, M,. Burkhardt, J.-M., Mégard, C., Ehanno,
P., 2005a. Results from a user-centred critical
incidents study for guiding future implementation of
augmented reality in automotive maintenance.
International Journal of Industrial Ergonomics, vol.
35, p. 67-77.
Aurum, A., Wohlin, C., 2003. The fundamental nature of
requirements engineering activities as a decision-
making process. Information and Software
Technology, vol. 45, p. 945-954.
Bastien, J. M. C. and Scapin, D., 2004. La conception de
logiciels interactifs centrée sur l'utilisateur : étapes et
methodes. Falzon, P. (Ed.) Ergonomie. Paris, PUF.
Berander, P., Andrews, A., 2005. Requirements
Prioritization. Aurum, A.,Wohlin, C. (Eds.)
Engineering and Managing Software Requirements,
Springer Verlag.
Bernard, J.-M., 2003. Analysis of local or asymmetric
dependencies in contingency tables using the
imprecise Dirichlet model. International symposium
on imprecise probabilities and their applications,
(Lugano) Switzerland.
RequirementsPrioritizationbyEnd-usersandConsequencesonDesignofaVirtualRealitySoftware-AnExploratory
Study
13
Caelen, J., 2009. Conception participative par ‘moments’ :
une gestion collaborative. Le travail humain, vol. 72,
p. 79-103.
Christian, T., Mead, N. R., 2010. An Evaluation of Cost-
Benefit Using Security Requirements Prioritization
Methods. U.S. Department of Homeland Security.
Collinge, C., Landry, R., 1997. Prévention des troubles
musculosquelettiques associés à la bureautique :
analyse des besoins et portrait de la formation et de
l'information. IRSST (Ed.).
Darses, F., Falzon, P., Béguin, P., 1996. Collective design
processes. Second International Conference on the
Design of Cooperative Systems, Sophia-Antipolis
(France), INRIA.
Iqbal, A., Khan, F. M., Khan, S. A., 2009. A critical
analysis of techniques for requirement prioritization
and open research issues. International Journal of
Reviews in Computing vol. 1, p. 8-18.
Kaiya, H., Horai, H., Saeki, M., 2002. AGORA: attributed
goal-oriented requirements analysis method. IEEE
Joint International Conference on Requirements
Engineering.
Kaiya, H., Shinbara, D. Kawano, J., Saeki, M., 2005.
Improving the detection of requirements discordances
among stakeholders. Requirements Engineering, 10,
p. 289-303.
Karlsson, J., Wohlin, C., Regnell, B., 1998. An evaluation
of methods for prioritizing software requirements.
Information and Software Technology, vol. 39,
p. 939 - 947.
Loup-Escande, E., 2010. Vers une conception centrée sur
l'utilité : une analyse de la co-construction
participative et continue des besoins dans le contexte
des technologies émergentes. Thèse de doctorat,
Université d'Angers.
Ma, Q., 2009. The Effectiveness of Requirements
Prioritization Techniques for a Medium to Large
Number of Requirements: A Systematic Literature
Review. Master degree, Auckland University of
Technology.
Macaulay, L., Fowler, C., Kirby, M., Hutt, A., 1990.
USTM: a new approach to requirements specification.
Interacting with Computers, vol. 2, p. 92-118.
Maguire, M., Bevan, N., 2002. User Requirements
Analysis: A Review of Supporting Methods. IFIP 17th
World Computer Congress, Kluwer, B.V.
Plos, O., Buisine, S., Aoussat, A., Dumas, C., 2007.
Analysis and translation of user needs for assistive
technology design. International Conference on
Engineering Design.
Popovic, V., 2003. Expert and Novice Users Model and
their Application to the Design Process. Journal of the
Asian Design International Conference, Tsukuba,
Japan.
Reich, Y., Konda, S. L., Monarch, I. A., Levy, S. N.,
Subrahmanian, E., 1996. Varieties and issues of
participation and design. Design Studies, vol. 17, p.
165-180.
Saaty, T. L., 2005. Analytic Hierarchy Process. In Sons, J.
W. (Ed.) Encyclopedia of Biostatistics
.
Tichkiewitch, S., Tiger, A., Jeantet, A., 1993. Ingénierie
simultanée dans la conception de produits. Universités
d'été du pôle productique Rhône-Alpes.
Von Hippel, E., 2005. Democratizing Innovation. MIT
Press.
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering
14