REQUIREMENTS ELICITATION METHOD FOR DESIGNING

VIRTUAL COLLABORATIVE SYSTEMS WITH

COLLABORATIVE SENSEMAKING

Alexandre Parra Carneiro da Silva and Celso Massaki Hirata

Technological Institute of Aeronautics, Department of Electrical Engineering and Computer Science, Praça Marechal

Eduardo Gomes, 50, Vila das Acácias, São José dos Campos, Brazil

Keywords: CIB, Collaborative Sensemaking, CSCW, Requirements Elicitation.

Abstract: Many approaches for requirements elicitation have been proposed to help the design of virtual collaborative

systems. The design of virtual collaborative systems with collaborative sensemaking presents a challenge

due to the needs of interaction of users with the system in order to process, interpret, and share information

collaboratively. However, most of the design approaches fail in capturing the users’ needs, because they are

not designed to capture precisely the main users’ activities interactions during the interplay between

‘collaboration’ and ‘information’ with sensemaking. Sensemaking determines the way in which people

respond to certain events and allows constructing perceptions taking into account their goals, priorities and

problems. This paper describes a requirement elicitation method that captures the interactions of potential

users in collaborative environments with collaborative sensemaking activities. The method is based on the

simulation of activities it was employed in the design of a virtual collaborative system - a collaborative

puzzle – in order to illustrate its usage.

1 INTRODUCTION

Collaboration is an essential aspect of many types of

daily activities (Paul and Reddy 2010b). One

activity that is central to people’s personal and

professional lives is information seeking (Paul and

Reddy 2010b). An important factor of Collaborative

Information Seeking (CIS) practice is to make sense

of the information found, i.e., Collaborative

Sensemaking (CS) (Feldman and Rafaeli 2002;

Weick 1995). CS is the process whereby individuals

process information, integrate and interpret it and

through social interaction they share their

understandings (Feldman and Rafaeli 2002). The

objective of process CS is to provide a shared

understanding about information, goals, priorities

and problems that individuals face in collaborative

settings to make decisions and act effectively.

Without some shared knowledge base or an effective

interaction between team members, severe gaps are

likely to occur in the understanding of reality

(Ravied et al. 2008). The research area of

Collaborative Information Behaviour (CIB) has

received increased interest in recent years (Paul and

Reddy 2010b). The CIB area concerns about the

behaviour exhibited when people work together with

information to gain a better understanding of various

activities such as CIS, CS and others that happen at

the interplay between ‘collaboration’ and

‘information’ (Paul and Reddy 2010b). The goal of

CIB is to better translate research findings into

workable design recommendations and technical

solutions.

It is hard to define requirements for an ideal

collaborative environment, because they depend on

the organization, context, problem, participants and

other factors (Baasch 2002). There are some

agreements on the characterization of a collaborative

environment. Dargan (2001) proposes seven

capabilities that a collaborative environment should

have. Baasch (2002) adds two more capabilities to

the Dargan’s list. Among the capabilities there are

some that are directly related to the interplay of

'information' and 'collaboration', such as: rapidly

find the right people with the right expertise; build,

find, and exchange information across

organizational boundaries; deliver the right

information to the right people as soon as it is

Parra Carneiro da Silva A. and Massaki Hirata C..

REQUIREMENTS ELICITATION METHOD FOR DESIGNING VIRTUAL COLLABORATIVE SYSTEMS WITH COLLABORATIVE SENSEMAKING.

DOI: 10.5220/0003929700240035

In Proceedings of the 8th International Conference on Web Information Systems and Technologies (WEBIST-2012), pages 24-35

ISBN: 978-989-8565-08-2

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

available, and keep a record of all collaborations for

further reference.

A collaborative system is one where multiple

users or agents are engaged in a shared activity,

usually from remote locations. Comparing with

other distributed systems, collaborative systems are

distinguished by the fact that the agents are working

together towards a common goal and have a critical

need to interact closely with each other (Ivan et al.

2008). To achieve this distinction, collaborative

systems should have effective mechanisms for

communication, coordination, cooperation and

awareness. According to Fuks et al. (2008),

Communication consists of the exchange of

information in negotiations among people;

Coordination is related to the management of

people, their activities and resources; and

Cooperation is the production that takes place in the

shared workspace. The participants obtain feedback

from their actions and feed through from the actions

of their companions by means of awareness

information related to the interaction among

participants. The 3C collaboration model proposed

by authors is based on work of Ellis et al. (1991).

Räsänen and Nyce (2006) argue that many

systems have failed because developers have

neglected the social context where technology is

used. Social context is formed by actors and

relationships among them. The correct identification

of actors and relationships among them may help

developers to better understand how software

technology should be inserted in such context

(Martins 2007). The success of an information

system depends on the quality of the definition of

requirements (Martins 2007). The quality of the

requirements is greatly influenced by techniques

employed during requirement elicitation (Hickey

and Davis 2002). Requirements elicitation

techniques are methods used by analysts to

determine the needs of customers and users, so that

systems can be built with a high probability of

satisfying those needs (Hickey and Davis 2003).

However, consensus exists that one elicitation

technique cannot work for all situations (Macaulay

1996; Kotonya and Sommerville 1998; Glass 2002;

Hickey and Davis 2003). There are lots of

requirements elicitation techniques described in

literature (Byrd et al. 1992; Davis 1993; Hudlicka

1996; Macaulay 1996; Kotonya and Sommerville

1998; Leffingwell 2000; Glass 2002; Lauesen 2002).

In our survey of the elicitation of requirements,

we have not found techniques that take into account

the needs of users in collaborative environments

considering CS. The understanding of user needs in

such contexts is essential to provide mechanisms for

an effective cooperation. We believe that a

technique that focuses on the understanding of the

actors during their activities in a collaborative

environment in order to fully take advantage of the

synergy among stakeholders to achieve common

goals is required.

This paper presents a method for requirements

elicitation in collaborative environments with

sensemaking. The technique is focused on the

observation of actors interacting to perform

activities that occur in the interplay between

collaboration and information. The method captures

the needs of users in collaborative environments

whose collaborative activities - which are covered

by the activity CS and CS activity itself - are

investigated. The technique is based on the

simulation of the targeted collaborative system. In

the simulation, restrictions and permissions are

defined to the participating users in order to provide

a close environment of the target system whereby

the interactions among them can be monitored. The

technique aims to obtain a more effective

requirements elicitation to the stakeholders involved.

The remainder of the article is structured as

follows. Section 2 presents definitions, concepts

about CIB, the difficulty of requirements elicitation,

and the related work. Section 3 describes the

proposed method for requirements elicitation and its

application through simulation. More specifically,

we present the experimental environment wherein

the simulation carried out, results of

experimentation, and analyses of results. In section 4

we analyse the proposed method. Section 5

concludes the article and discusses future work.

2 BACKGROUND

In recent years, researchers from a diverse range of

disciplines conducted various studies (Poltrock et al.

2003; Foster 2006; Hertzum 2008; Ravid et al. 2008;

Paul and Reddy 2010b) within organizational and

non-organizational settings and have provided many

key insights about activities that happen at the

interplay between ‘collaboration' and 'information'

(Karunakaran 2010). To integrate the various termi-

nologies associated with Collaborative Information

Behaviour (CIB) in the studies, Karunakaran et al.

(2010) define a working definition of CIB as

“totality of behaviour exhibited when people work

together to identify

an information need, retrieve,

seek

and share information, evaluate, synthesize and

make sense of the found information, and then

REQUIREMENTSELICITATIONMETHODFORDESIGNINGVIRTUALCOLLABORATIVESYSTEMSWITH

COLLABORATIVESENSEMAKING

utilize the found information”.

Among the activities that comprise the

behaviours investigated in CIB, those underlined

above, the activity sensemaking is a critical element

of collaborative work (Ravid et al. 2008; Paul and

Reddy 2010b). Sensemaking is the process through

which individuals view and interpret the world and

then act accordingly (Ravied et al. 2008).

Sensemaking determines the way in which people

respond to certain events and construe their

perceptions regarding goals, priorities and problems

they face (Ravied et al. 2008). Convergent evidence

shows that Collaborative Sensemaking (CS)

crosscuts the activities within CIB (Paul and Reddy

2010b). CS occurs when multiple actors with

possibly different perspectives about the world

engage in the process of making sense of ‘messy’

information (Ntuen et al. 2006; Paul and Morris

2009) to come at shared understandings (Ravied et

al. 2008) and then to act accordingly for coming

more near their common goal. CS is an important

aspect of Collaborative Information Seeking (CIS)

(Paul and Reddy 2010b). CIS is defined as “the

study of systems and practices that enable

individuals to collaborate during the seeking,

searching, and retrieval of information” (Foster

2006, p .330). CIS occurs when “a group or team of

people undertakes to identify and resolve a shared

information need.” (Poltrock et al. 2003, p.239).

Resolving a shared information need often consists

of finding, retrieving, sharing, understanding, and

using information together (Paul and Reddy 2010a).

Reddy and colleagues in two investigations in

healthcare providers environments identify some

reasons that lead to the occurrence of CS, such as

(Reddy and Jansen 2008; Paul and Reddy 2010b):

ambiguity of available information, role-based

distribution of information, lack of domain

expertise, complexity of information need, and lack

of immediately accessible information. CS involves

tasks whereby individuals: share information and

sense, prioritize relevant information, contextualize

awareness information with respect to activities, and

create and manipulate shared representations (Paul

and Reddy 2010a).

Due to these needs of individuals or users,

building collaborative systems that take into account

sensemaking is a challenge. The construction of

software system requires a software development

process that includes the following phases:

requirements elicitation, design, implementation,

verification and maintenance. We believe that one

of main difficulties in the process development is to

correctly

elicit the requirements due to the needs of

individuals in CS.

The goal of the requirements elicitation is to

understand the real needs of the users which must be

supported by the software to be developed

(Sommerville and Ransom 2005). During the

requirement elicitation phase, the stakeholders

exchange information about the context and the

activities that will be supported by the software

under development (Laporti et al. 2009). This phase

is seldom problem free. Viewpoint, mental model

and expectation differences among users and

analysts make this task hard and conflicting. In

many cases, the clients are not completely sure about

their real needs. In other words, the current

approaches to address requirements elicitation do

not correspond to management expectations (Laporti

et al. 2009). Sommerville (2006) points out

problems in this phase are responsible for 55% of

computer systems’ troubles and that 82% of the

effort devoted to correcting mistakes is related to

this phase. Some techniques that consider social

context are proposed to elicit requirements of

collaborative systems as shown in (Hickey and

Davis 2003; Zoowghi and Coulin 2005; Broll 2007;

Machado 2009).

Machado et al. (2009) proposes a method to

support requirements elicitation in organizations

where there exists clearly demand by improve in

existing systems. Their method combines traditional

cognitive and ethnographic methods and focuses on

the capture of the actual activities being executed in

the context of the workplace, i.e., focused at work.

Their approach assumes that there is a difference

between information obtained from stakeholders

during interviews, and the rich, dynamic and

complex reality of workplaces. However, their

technique does not take into account the

collaborative interactions from the information

needs and other activities that support the needs.

These activities are main triggers of work in

collaborative settings (Poltrock et al. 2003; Reddy

and Jansen 2008; Paul and Reddy 2010b). Moreover,

ethnography research is very time intensive and it

has high costs (Myers 1999).

Simulation is a technique, not a technology, to

replace or amplify real experiences with guided

experiences that evoke or replicate substantial

aspects of the real world in a fully interactive

manner (Gaba 2007). A critical point that has often

been missed is that process of using simulators and

simulations is a "social practice" (Laucken 2003). A

social practice can be defined as a contextual event

in space and time, conducted for one or more

purposes, in which people interact in a goal-oriented

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

fashion with each other, with technical artefacts (the

simulator), and with the environment (including

relevant devices). To regard simulation as a social

practice puts an appropriate emphasis on the reasons

why people take part in it and how they choose to

interpret the various simulation endeavours

(Dieckmann et al. 2007).

The simulation is chosen due the following

advantages: conditions can be varied and outcomes

investigated; critical situations can be investigated

without risk; it is cost effective; simulations can be

speeded up so behaviour can be studied easily over a

long period of time, and they can be slowed down to

study behaviour more closely. The simulation of the

collaborative setting can be used to address types of

knowledge, skills, attitudes, or behaviours that

people have in specific imposed stimulus, rules or

objectives. We believe that these impositions help to

show potential problems and deficiencies that users

may face in their collaborative environments. Thus,

developers can act to solve or mitigate them through

computer systems more effective in the

environment. In addition, the collaborative

environment can be more effective with the

participation of users in the simulations because they

can recognize difficulties to collaboration in poorly

designed organizational processes.

Our proposed approach for requirements

elicitation in collaborative settings is focused on the

observation of actors interacting to perform

activities that occur in the interplay between

collaboration and information. The environment

wherein the actors interact is simulated. We believe

that this approach we provide a more effective

elicitation of requirements.

3 PROPOSED METHOD

The proposed method consists of a simulation of the

activities that make up CIB. We consider the

following CIB activities: identify an information

need; search, retrieve and share information;

evaluate, synthesize and make sense of information

found, and use the information found. The

identification of information need is the main factor

that drives collaborative activities. In order to design

the collaborative environment properly, the

proposed method seeks to simulate the virtual setting

emphasizing CIB activities that are likely to occur

with emphasis on collaborative sensemaking

activity. The simulation is made in a co-located

environment with restrictions of collaborative

systems artificially enforced. The idea is to monitor

the users’ interactions in a simulated collaborative

environment, so that is possible to identify

collaborative requirements and build systems. The

simulation method consists of four activities:

 Definition of the System, its Environment,

and Restrictions and Permissions of Com-

munication, Coordination, Cooperation,

and Awareness (3C+A). Define the system

and its environment by indicating the

characteristics of collaborative setting to be

simulated such as the main collaborative

activities, potential users, roles of users, and

business rules. The activities are defined based

on their importance for achieving the common

objectives pursued in a collaborative

environment. Potential users, their roles and

business rules are originated from the

definition of key collaborative activities and

the objectives to be achieved. The

collaborative activities to be investigated in

the simulation are mainly those which have

intrinsically features of the interplay between

collaboration and information. The restrictions

and permissions of 3C+A are those that the

potential users face when interacting with the

actual system. The output of this simulation

activity is a document describing the

aforementioned items, mainly the restrictions

and permissions, of the system and its

environment.

 Planning the Simulation. Define the proce-

dures, techniques and tools that capture user

interactions during collaboration. Define also

the procedures, techniques, and tools to

analyse the data collected in order to identify

both the breaks of restrictions and needs of

3C+A in the interactions. Define monitors

(observers) and their role for monitoring the

simulation. The output is a list of procedures,

techniques and tools to capture and analyse

the data related to the users’ interactions. The

planning output includes the definition of

procedures to monitor and control the

simulations.

 Execution, Monitoring, and Control. Ex-

ecute, monitor and control the simulation, in

accordance with the plan. In this activity, the

potential users take part in the simulated

activities whereby their interactions with the

system and with other users are monitored.

The outputs are information and data collected

and a set of annotations describing changes

accomplished during simulations.

REQUIREMENTSELICITATIONMETHODFORDESIGNINGVIRTUALCOLLABORATIVESYSTEMSWITH

COLLABORATIVESENSEMAKING

 Analyses and Identification of Require-

ments: Analyses the data collected previously

according to the chosen techniques and

defined procedures. The purpose of the

analysis is both to validate the proposed

restrictions of communication, coordination,

cooperation and awareness and identify new

needs (requisites) related to restrictions and

permissions of communication, coordination,

cooperation, and awareness. The identification

of the requisites is a result of the restrictions

that were suitably proposed and proposals of

new restrictions in the collaborative

interactions that occurred in the simulations.

4 EXPERIMENT, RESULTS AND

ANALYSIS

In order to illustrate the usage and evaluate the

proposed method, we conducted an experiment in a

simulated collaborative workplace. The aim of the

method is to find the appropriate requirements of

communication, coordination, cooperation and

awareness (3C+A) for the system or subsystem to be

developed for the targeted collaborative

environment. The (actual) collaborative environment

is a Web environment whereby participants can

work together to solve a puzzle challenge. A puzzle

is a problem or enigma that tests the ingenuity of the

solver. In a basic puzzle, one is intended to put

together pieces in a logical way in order to come up

with the desired solution. Puzzles are often contrived

as a form of entertainment, but they can also stem

from serious mathematical or logistical problems.

The type of puzzle that inspired us to create a

collaborative environment for study is Tiling

Puzzles. Tiling puzzles are two-dimensional packing

problems in which a number of flat shapes have to

be assembled into a larger given shape without

overlaps (and often without gaps) (Puzzle 2010).

Unlike tiling puzzles, the collaborative puzzle

defined is not guided by the construction of a known

image, but by a single contiguous area containing

blocks of the same cor. In our collaborative puzzle, a

piece is an arrangement of rectangle blocks of

defined shapes and the goal is to set the highest

number of pieces (or blocks) which must have

cohesion. The cohesion is characterized by absence

of gaps and pieces of the same color. The change of

colors in a contiguous is a measure of the team´s

performance. Below we apply the method proposed

to collaborative environment target.

4.1 Definition of the System, Its

Environment, and Restrictions and

Permissions of Communication,

Coordination, Cooperation and

Awareness (3C+A)

As a result of the first step we have the following

scope for the simulation. The collaborative challenge

must be resolved by four participants co-located.

The common goal of participants in the challenge is

to place the pieces on the table so that they form one

cohesive contiguous area (puzzle) with a minimum

of empty areas (gaps) inside. The pieces have

different shapes and each piece is made up of

different amount of blocks of the same color.

Fourteen or more contiguous blocks of the same

color that form a contiguous area is enough to score.

Therefore, the eleven yellow contiguous blocks

showed in Figure 1 is not considered a contiguous

area. The team earns points according to the

cohesive contiguous area formed. For achieving the

maximum degree of cohesion, it is necessary that the

area formed by the pieces covers the available space

with

all blocks of the same color without any gap.

Figure 1: Sample of single contiguous area formed with

three minor contiguous areas.

The simulated collaborative setting has the

following characteristics: (C1) each participant has a

number of particular pieces initially and they can be

presented to other participants when placed on the

table; (C2) the amount of private pieces is not

necessarily the same for all participants; (C3) Each

participant has a placeholder (box) wherein his/her

private pieces are; (C4) on a public area (table) some

pieces are present from the beginning of the

challenge and they are called public pieces, and any

participant can access them; (C5) particular pieces

presented on the table become public regardless of

whether they are placed in the puzzle or not; (C6) all

pieces, private or public, are made of colourful

blocks, and (C7) the pieces have different shapes

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

and sizes and each piece is unicolor.

Private pieces are a tentative to represent the

individual tacit information whereas public pieces

try to represent information that is shared by the

group of people. The placeholder is a simulated

space that its owner has access (knowledge). The

owner is restricted in its ability to show his pieces by

time and amount. We discuss the restrictions below.

The activities identified in the collaborative

environment are: (A1) present one particular piece

on the table; (A2) present one particular piece and

place together it with other pieces on the table; (A3)

arrange public piece(s) that are on the table; (A4)

disconnect piece(s) of puzzle; (A5) compute the

performance of the group according to the pieces

placed together; (A6) come to the consensus; (A7)

evaluate information collectively; (A8) share

information; (A9) identify information need, and

(A10) make sense collaboratively. The activities A1

- A5 were identified by the characteristics of

collaborative environment of the challenge and by

the common goal that has to be achieved. The

activities A6 - A10 come from the interactions of

participants in the challenge as actions or operations

that support the activities A1 - A5. As described, we

consider the particular pieces like information and

knowledge to each participant in the challenge. The

pieces already presented on the table are information

known to (shared by) the group of participants. Each

one can simply present his/her information on the

table (activity A1), without giving meaning to them.

On the other hand, a participant may connect his/her

information with others (activity A2) or just connect

the information that is already on the table (activity

A3), thus giving the information a particular

meaning in the context. The various possibilities for

links of information are due to different

interpretations of them. The interpretations may be

the products of both activities, make sense (activity

A10) and evaluate information (activity A7), both

executed collectively. The interpretations may also

occur during the activity disconnect pieces (activity

A4). Sharing information (activity A8) on the table

by each participant is encouraged due to information

needs that arise as the resolution of the challenge

goes on. Identifying these needs (activity A9)

depends on the interpretation and evaluation of

information available to all participants, for

example, the information from the computation of

group performance (activity A5).

Possible actions/behaviours of the participants on

some activities identified have been defined. Group

members must meet the following rules (R):

(R1) Only one group member at a time has

access to the puzzle to carry out his/her

activities.

(R2) The next member to access the table to

play is chosen by the group members

themselves.

(R3) Each member to perform activities A1 -

A4 must justify the reason for his/her act.

Each group member, during his/her

participation, can act as follows:

(R4) Execute the activity A1 or A2 at most

once.

(R5) If the results of activities A2 - A5

performed by a member, are not considered by

the group, it is the responsibility of the

member to undo the work.

(R6) The pieces that are placed on the table

must remain on it regardless they are

interlinked or not to the other pieces in the

puzzle.

(R7) The solution presented by the group

members as the final solution is just one single

strongly connected area, i.e., there exists a

path from each block to every other block.

Group members may use the following

permissions (P):

(P1) Perform activities A3 and A4 as many

times as necessary in each participation and

the member can be assisted by other group

members during these activities.

(P2) The group can measure their

performance, activity A5, during the challenge

at any time.

Participants are not limited to actions/behaviours

defined. In other words, they may take other

actions/behaviours, but without violating the

aforementioned restrictions.

4.2 Planning the Simulation

We propose to divide the planning into three stages:

pre-experiment, experiment and post-experiment. In

the pre-experiment stage, four participants are

informed of the details of the challenge, such as the

characteristics, rules and how the group is evaluated.

The form of evaluation of the group, in particular, is

thoroughly explained with the help of illustrations of

possible solutions. Doubts of participants should be

answered before proceeding to the next stage. Only

after the participants understand the challenge, the

challenge can be initiated.

During the challenge the participants can ask

questions to the monitors of the simulations.

However, the participants are aware that the time

REQUIREMENTSELICITATIONMETHODFORDESIGNINGVIRTUALCOLLABORATIVESYSTEMSWITH

COLLABORATIVESENSEMAKING

spent with questions is considered as play time. The

end of the challenge occurs when the group presents

its final solution or when the challenge reaches its

limit of 30 minutes. The second stage begins at the

signal of the monitor. The monitor, turn on the video

camera to record the behaviours and actions of

group members. In other words, the stage comprises

the running of simulation to collect data from users'

interactions of interest. Before going to the last

stage, the solution presented by the group is

evaluated by the monitors and the result announced

to the group. After all members know their

performance in group, each member receives a

questionnaire to answer it. It is the final stage.

We analyse the data of questionnaires using the

method content analysis (Stemler 2001). Our

objective is to capture the perceptions of each

member about the rules imposed and collaborative

activities that they perform. With their answers, we

can know what rules, permissions and collaborative

activities are most problematic, difficult, awkward,

and challenging. With this information, we can look

for reasons for these problems. This can be achieved

through the analysis of the recorded videos of the

challenges made. The use of videos is suitable to

analyse the interactions of the actors when they

perform their activities. According to (Ruhleder and

Jordan 1997), there are several advantages of using

video for interaction analysis, and the main reason is

that video is permanent information that can be

recurrently analysed. It provides the opportunity to

several researchers to perform their own

interpretations and a collaborative multidisciplinary

analysis can create an unbiased view of the events.

Video-based Interaction Analysis (VbIA) also

avoids the say/do problem, i.e., what people say they

do and what they really do may not necessarily be

the same. VbIA exposes mechanisms and

antecedents due to the fact that the video provides

process data rather than snapshot data. Since video

records the phenomenon of interest in context, it is

possible to ask about antecedents, varieties of

solutions produced on different occasions, and

questions of what led up to any particular state. The

third source of information is the notes of the

monitors. These notes highlight the perceptions they

had of episodes that attracted their attention during

the simulations.

Twelve persons are invited to participate in the

challenge. They are chosen among the students

enrolled in the undergraduate course of engineering

at the Technological Institute of Aeronautics (ITA).

Students are both informed about the objectives of

the experiment and asked to participate. The twelve

students faced the same challenge but in three

groups of four. The three groups are established

according to the students’ choice. Two monitors

monitor and control the three simulations to be

performed. The degree of their participation is

restricted to answer questions of participants, take

notes of the simulation, check the capture of images

by the camera and apply the questionnaire to

participants. Four sets of particular pieces were

chosen and offered to members of each group. The

pieces in these sets were chosen randomly by the

monitors, meeting the characteristics C1 and C2. For

each group, the pieces are placed in four boxes of

different colors. Before starting each experiment, the

boxes are chosen by members. Thus, each member

does not know its contents in advance and also uses

the boxes during the challenge as a placeholder.

Each member is unaware of the contents of the

boxes of the other members during the challenge,

respecting the characteristics C3 and C5. Before

each experiment, nine public pieces are already

available on the table, satisfying C4. These pieces

are also chosen randomly. Before starting each

challenge, a camera is adjusted and turned on to

capture the actions and behaviours of the group

members. The camera is placed on the table inside

the room where the simulations takes place. The

monitors observe the three groups and take notes of

events that call their attention, as shown in Figure 2.

Figure 2: Group members collaborating in a puzzle

observed by a monitor.

4.3 Executing Simulation

The simulation of three collaborative challenges

generated the following data: video recordings of the

simulations, notes of the interactions that called

attention, the performance of groups that was

calculated according to the final solution presented

to the monitors, and the solutions that were

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

photographed,. All groups took the period of 30

minutes to build their solutions. The performances

of the groups were then calculated. The first, second

and third group achieved the degree of cohesion of

83.4%, 65% and 56% respectively. We stress that

the same pieces were used in three simulations, both

public and private ones.

4.4 Analysing and Identifying

Requirements

As the last step of the method we analyse the

collected data to identify requirements for building

the collaborative challenge on the Web. The analysis

was conducted as planned in the second stage. Data

collected in questionnaires responded by participants

were copied into electronic form before being

analysed. Stretches of audio-visual recorded also

were copied into electronic form. First, we classify

the data collected from the open questions of the

questionnaires in the following units of content:

(Q1) How do you evaluate the rule R1 as to the

group’s performance? Justify your evaluation. (Q2)

How do you evaluate the rule R2 as to the group's

performance? Justify your evaluation. (Q3) How do

you evaluate the rule R3 as to the group’s

performance? Justify your evaluation. (Q4) How do

you evaluate the rule R4 and permission P1 as to the

group’s performance? Justify your evaluation. (Q5)

How do you evaluate the rule R5 as to the group's

performance? Justify your evaluation. (Q6) How do

you evaluate the rule R6 as to the group's

performance? Justify your evaluation. (Q7) During

the evaluation of pieces connected and disconnected

the participants had any problem? If so, describe

them. (Q8) Making sense in a group, i.e.,

participants with different perspectives and goals

engage in making sense together of how to arrange

the pieces available and interpret them, and

understand what information (pieces) are needed

and where. Did you have any problems during this

activity? If so, describe them. (Q9) There were

problems to arrive at consensus on actions taken by

the participants? If so, describe them.

After sorting the data collected from

questionnaires we analyse the data collected. The

analysis consisted of the comparisons of the

participants' perceptions with the monitors’

perceptions, i.e., notes and analysis of the videos.

In short, the group that has the best performance

was the one with better coordination, cooperation

and awareness. G1 began to address the challenge by

defining a strategy. The strategy was based mainly

on two steps: understanding (in group) what degree

of cohesion of a contiguous area is, and how to

obtain the area with the highest possible degree of

cohesion. All members of the group at the beginning

of the challenge sought to make sense of all

information presented to them. The result of this

search yielded a unique understanding of the

problem leading to better group performance. This

was not seen in the activities of the other two

groups. We note the difficulties of G2 and G3 to

coordinate their activities because the different

perceptions of members of these groups on how to

achieve the goal of the collaborative challenge. The

cooperation of members of these groups was not that

productive due to the lack of a shared understanding

among them. Shared understanding was reached by

the G1 only after making sense of the information

together. The G2 and G3 did not explicitly define a

suitable strategy as the G1, but by analysing the

video and other data sources we realize that they had

strategies, but divergent. At the start of their

challenge, each member of groups G2 and G3

seemed to be worried for creating large contiguous

sub-areas of a specific color by himself. At some

point of the challenge, the member wanted to join

his area to build a single contiguous area, satisfying

the rule R7. Of course, there were some conflicts

and time wasting. As result of the strategy, G2

handed in a puzzle with 40 pieces and G3 a puzzle

with 49 pieces. As previously reported, the degrees

of cohesion of their solutions were, respectively

65% and 56%. The G1 needed only 24 pieces for

reaching 83.4%.

By cross-analysing the data collected we realize

that some restrictions were not beneficial to the

resolution of the collaborative puzzle. The definition

of who must access the table was based on

consensus by the group members, rule R2. However,

some participants suggested another option to

determine who should access the table. They

suggested that it would be better to determine in

advance the turn of access to the table. The rule R4

was also identified as an obstacle by several

participants. The suggestion is to change R4 for the

following: the group member who is accessing the

table may display and place various pieces in the

puzzle during his/her access. The rule R5 was also

suggested to be changed: any member who is

accessing the table, can undo previously actions if

the member asks permission and justifies his/her

changes. The rule R6 that does not allow group

members to withdraw non used pieces from the table

(that are not interconnected to the puzzle) brought

problems. This restriction overloaded the group with

pieces and resulted into difficulties to the group to

REQUIREMENTSELICITATIONMETHODFORDESIGNINGVIRTUALCOLLABORATIVESYSTEMSWITH

COLLABORATIVESENSEMAKING

make sense and decisions. The overload was

reported by the participants and was identified by

monitors. Many participants complained that the

pieces were hard to be linked, thus polluting the

public are and making the resolution of the

challenge harder.

Based on cross-analysis of data collected, we

define the following requisites of communication,

coordination, cooperation and awareness to build the

collaborative challenge for the Web:

 Communication

: (a) Use synchronous

communication through speech and

handwriting. The characteristics C1 and C3

oblige a restricted communication as the

video, e.g., use a tool as video-conference.

Thus, to support coordination, cooperation and

awareness among group members they must

have a similar tool to chat, but without the

video conferencing feature. (b)

Communication must be flexible, i.e., unicast

and broadcast. This requirement is due to the

need for each group member having to report

and seek information with others in a

particular and broadcast way. For example,

when you want to know who has pieces of a

similar format and specific color, or request to

a specific member to rearrange pieces.

 Coordination

: (a) Allow access to the virtual

table of only one group member at a time. (b)

The next group member to access the virtual

table is determined by vote and the decision

will be by consensus (to be established

through the synchronous communication as

occurred in the simulations). Another option is

to define the order of access of all members in

advance. The group can change the form of

selection of the next member. The first option

is defined because three participants reported

that they lost time with the rule R2 and that it

should be possible to determine the order of

their moves earlier. (c) The access to the table

will have a maximum time pre-determined by

the group members. The member who is

supposed to access the table can yield the

access to the next member. If the maximum

time is reached the member loses his/her

access and the table will be available for the

next group member. The time spent in

accessing the table in the simulations by the

group members varied widely. So to avoid

that a member locks the access to the table

each member will have a time limit of access.

 Cooperation

: (a) Allow that more than one

particular piece be presented in the table by a

group member at a time. Several participants

complained and we also observe in the videos

that the rule R4 was not helpful for the group

performance. Because of this the group

members decided to simply "circumvent" this

rule. As they could not place more than one

piece or arrange pieces on the table, the

members asked several consecutive accesses

to the table to perform the activities. (b) Pieces

that the group believes that are not useful,

should not remain on the table and should be

returned to the members who presented the

pieces. Thus, the computation of the degree of

cohesion of the solution becomes simpler. (c)

The calculation of the degree of cohesion of

contiguous areas can be made automatically if

a member requests. Although we believe that

the calculation is not difficult, in the

simulations we found that two groups have not

coordinated their activities. (d) Undo

interconnections and disconnections reverting

to the previous play can be executed by any

group member, but it should only be

performed by one group member. In practice

the participants will not recognize the rule R5.

 Awareness

: (a) Allow group members to

chain messages exchanged based on a specific

event and its sequences. This requirement

aims to bring understanding to group members

on decisions and their consequences. (b) Each

piece on the table must indicate to whom it

belonged. Public pieces do not have this

identification. Thus, a piece at a specific time

can be removed from table by member who

placed it. (c) The following information must

be visible to all members of the group: who is

accessing the table at a given time and how

long he/she is taking. This information can

help coordination and cooperation of the

group. (d) Each member must be notified

when the table is available to him.

5 ANALYSIS OF THE PROPOSED

METHOD

The proposed method is based on simulation of

collaborative activities that are important to achieve

the objectives of stakeholders. The method allowed

us to find unexpected behaviours. The simulation

study was defined it according to the objective,

constraints, and permissions for the users. The

restrictions were validated by performing

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

experiments with potential users of the collaborative

environment. The result was the identification of

requirements for communication, coordination,

cooperation and awareness (3C+A) for the

development of the collaborative puzzle on Web.

The effort spent in the simulation of the

collaborative puzzle can be considered low when we

compare with other existing methods, for instance,

ethnographic investigations. This is due to the fact

that the collaborative environment simulated is less

complex, i.e., has a small number of actors involved

and the resources employed in the simulation are of

low cost. However, we consider the puzzle a

collaborative challenge which allows analysing the

behaviours and interactions in a collaborative

environment based on the concepts of CIB, in

particular the concept of collaborative sensemaking.

We believe that the simulation method can be

employed for more complex collaborative

environments where there are several roles and

artefacts. In these environments the method can be

used in the key scenarios of environments that need

research due to a poor knowledge of the interactions

of the actors.

In this method, the perspective of monitors and

developers are captured through notes during the

simulations, and the perspective of users by means

of questionnaires. The two perspectives aim to

identify common findings and discrepancies. The

discrepancies can be further investigated with the

help of the video-based analysis.

If the elicitation of requirements for 3C+A is not

made in a satisfactory manner, the collaborative

environment simulation can be performed again on

specific scenarios that presented problems.

We are aware that there are various techniques

for gathering and analysing information, but the

utilization of the questionnaire, notes of observations

and video, and content analysis and video-based

interactive analysis showed to be adequate in the

collaborative puzzle simulation. The techniques

were chosen because of the following characteristics

of the collaborative challenge: number of users

collaborating is low, number of activities performed

by users is not high, and the common goal is not

complex to understand. The choice of collection and

analysis techniques of information to be used must

consider the characteristics of the environment to be

studied - for example the context and situation - as

pointed out in (Hickey and Davis 2003; Zoowghi

and Coulin 2005; Davis 2006).

Some problems reported by participants such as:

the difficulty to physically place the pieces in the

collaborative puzzle and the complexity of

computing the performance of collaborative groups

can be readily removed when the challenge occurs in

the Web. The virtual environment to be developed

can place the pieces with a drag of mouse and make

the computation of the team's performance

automatically on each piece insertion or removal.

The method, however, is unable to capture some

requirements, for instance, those related to log and

presentation of the history of usage by the

participants. The history might be useful to roll back

(undo moves) to past configurations.

Another restriction is physical limitation of the

usage of the co-located simulation. For the

collaborative puzzle, the simulation is not adequate

if the number of users is high. If the number of users

is high, coordination requirements should be

investigated deeply and elicited.

Other limitation has to do with the synchronism

of the collaboration. Our proposed method was

effectively employed for a synchronous

collaborative puzzle. We envision that the method

can be employed to other synchronous collaborative

applications; however, it may not result in the same

success if it is applied to asynchronous collaborative

applications.

In summary, our proposed method presents the

following advantages: it allows the identification of

requirements of 3C+A in an efficient manner in

terms of time and resources; it is suitable to be used

in synchronous collaborative environments, and it

allows capturing perceptions of major actors

involved in the development of the collaborative

system. On the other hand, the method has the

following disadvantages: it does not capture/save the

history of the interactions of the users involved in

collaborative activities; and it may not allow

investigating satisfactorily collaborative

environment that has many actors, roles and

resources involved – i.e. it is physically limited to be

employed.

6 CONCLUSIONS

In this research work, we propose a method of

requirements elicitation in collaborative

environments considering activities CIB, especially

the collaborative sensemaking activity. The method

utilizes the simulation of the target environment to

capture requirements of the following aspects of

collaboration: communication, coordination,

cooperation and awareness (3C+A). The simulation

environment is defined by restrictions and

permissions of the collaborative aspects. The method

REQUIREMENTSELICITATIONMETHODFORDESIGNINGVIRTUALCOLLABORATIVESYSTEMSWITH

COLLABORATIVESENSEMAKING

was illustrated on an environment of collaborative

puzzle. It allowed finding requirements for all

collaborative aspects considered in simulation

through verifications and inconsistency

identifications of the restrictions and permissions

assigned to the environment under study.

Some difficulties encountered in the co-located

simulations of collaborative environments can be

mitigated simply because these environments come

to be supported by computer systems. For instance,

in the collaborative puzzle two difficulties arose:

handle pieces of the puzzle and calculate the group

performance during the challenge. The difficulties

can be solved easily in a virtual version of the

challenge on the Web, as described in the previous

session. On the other hand, the support of computer

systems can bring new problems that are not

necessarily perceived during the simulation of

collaborative environments, for instance the

feedback from actors can be inefficient because the

computer system may have restrictions on capturing

and presenting interaction information – e.g.

gestures, expressions, tone of voice, etc., and the

actors may not have confidence or may not have the

knowledge to effectively use the system interfaces,

especially in critical collaborative environments, as

in air traffic control (Merlin and Hirata 2010).

As the next steps this research work, we are

developing the virtual collaborative puzzle and soon

after its development we will perform experiments

on it to evaluate the gains and the difficulties. In

addition, we are planning to expand the use of the

method in other collaborative environments, for

example in health care settings.

ACKNOWLEDGEMENTS

To CNPQ, for funding this research. To the

researcher Juliana de Melo Bezerra by help in the

design and participation in the experiments.

REFERENCES

Baasch, W. K., 2002. Group Collaboration in

Organizations: Architectures, Methodologies and

Tools. Virginia: Storming Media.

Broll, G., Hussmann, H., Rukzio, E. and Wimmer, R.,

2007. Using Video Clips to Support Requirements

Elicitation in Focus Groups - An Experience Report.

In: 2

International Workshop on Multimedia

Requirements Engineering. Hamburg, Germany. Los

Alamitos, CA: IEEE, 1 - 6.

Byrd, T. A., Cossick, K. L. and Zmud, R. W., 1992. A

Synthesis of Research on Requirements Analysis and

Knowledge Acquisition Techniques. MIS Quarterly,

16 (1), 117-138.

Dargan, P. A., 2001. The Ideal Collaborative

Environment. Journal of Defense Software

Engineering – Web-Based Applications, 14 (4), 11-15.

Davis, A., 1993. Software Requirements: Objects,

Functions and States. Prentice Hall.

Davis, A., Dieste, O., Hickey, A., Juristo, N. and Moreno,

A. M., 2006. Effectiveness of Requirements

Elicitation Techniques: Empirical Results Derived

from a Systematic Review. In: 14

IEEE Int.

Requirements Engineering Conf., Washington: IEEE,

176-185.

Dieckmann, P., Gaba, D. and Rall, M., 2007. Deepening

the theoretical foundations of patient simulation as

social practice. Journal of the Society for Simulation in

Healthcare, 2 (3), 183-193.

Ellis, C. A., Gibbs, S. J. and Rein, G., 1991. Groupware:

Some Issues and Experiences. Communications of the

ACM, 34 (1), 38-58.

Feldman, M. S. and Rafaeli, A., 2002. Organizational

routines as sources of connections and understandings.

Journal of Management Studies, 39 (3), 309-331.

Foster, J., 2006. Collaborative Information Seeking and

Retrieval. Annual Review of Information Science and

Technology, 8, (1), 329-356.

Fuks, H., Raposo, A., Gerosa, M. A., Pimentel, M.,

Filippo, D. and Lucena, C. J. P., 2008. Inter and Intra-

relationships between Communication, Coordination

and Cooperation in the Scope of the 3C Collaboration

Model. In: 12

Int. Conf. on Computer Supported

Cooperative Work in Design, Beijing, China. IEEE,

148-153.

Gaba, D. M., 2007. The future vision of simulation in

health care. The Journal of the Society for Simulation

in Healthcare, 13 (1), 126 - 135.

Glass, R. L., 2002. Searching for the holy grail of software

engineering. Communications of the ACM, 45 (5), 15–

16.

Hertzum, M., 2008. Collaborative Information Seeking:

The combined activity of information seeking and

collaborative grounding. Information Processing &

Management, 44 (2), 957-962.

Hickey, A. and Davis, A., 2002. The Role of

Requirements Elicitation Techniques in Achieving

Software Quality. In: Int. Workshop on Requirements

Engineering: Foundations for Software Quality,

Essen, Germany, 2002. 165-171.

Hickey, A. and Davis, A., 2003. Elicitation Technique

Selection: How Do Experts Do It?,” In: Proc. of the

IEEE Int. Conf. on Requirements Engineering,

California, USA. Washington: IEEE, 169-178.

Hudlicka, E., 1996. Requirements Elicitation with Indirect

Knowledge Elicitation Techniques: Comparison of

Three Methods. In: Proc. of the 2

Int. Conf. on

Requirements Engineering, Washington, USA.

Washington: IEEE, 4-11.

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

Ivan, I., Ciurea, C. and Visoiu, A., 2008. Properties of the

Collaborative Systems Metrics. Journal of Information

Systems & Operations Management, 2 (1), 20-29.

Karunakaran, A., Spence , P. R. and Reddy, M. C., 2010.

Towards a Model of Collaborative Information

Behavior. In: 2

Int. Workshop on Collaborative

Information Seeking - ACM Conference on Computer

Supported Cooperative Work, Savannah, USA. ACM.

Kotonya, G., and Sommerville, I., 1998. Requirements

Engineering: Processes and Techniques. NY: John

Wiley & Sons.

Laporti, V., Borges, M. R. S. and Braganholo, V., 2009.

Athena: A collaborative approach to requirements

elicitation. Journal of Computers in Industry, 60 (1),

367-380.

Laucken, U., 2003. Theoretical Psychology. Oldenburg:

Bibliotheks- und Informations system der Universita¨t,

2003.

Lauesen, S., 2002. Software Requirements: Styles and

Techniques. Addison-Wesley.

Leffingwell, D. and Widrig, D., 2000. Managing Software

Requirements. Boston:Addison-Wesley.

Macaulay, L. A., 1996. Requirements Engineering.

London: Springer-Verlag.

Machado, R. G., Borges, M. R. S. and Gomes, J. O., 2009.

Supporting the System Requirements Elicitation

through Collaborative Observations. In: Proc. of the

Int. Workshop CRIWG, Omaha, USA,

Berlin:Springer-Verlag, 364-379.

Martins, L. E. G., 2007. Activity Theory as a feasible

model for requirements elicitation processes. Scientia:

Interdisciplinary Studies in Computer Science,

Unisinos, BR. Available from: http://www.unisinos.br/

arte/files/scientia18%281%29_art04_martins.pdf [Ac-

cessed 28 November 2011].

Merlin, B., and Hirata, C., 2010. Collaborative System

Experience in a Critical Activity Context: Air Traffic

Control. In: Brazilian Symposium on Collaborative

Systems. IEEE, 17-24.

Myers, M. D., 1999. Investigation Information Systems

with Ethnographic Research. Communications of the

Association for Information Systems, 2 (4), 1-20.

Ntuen, C. A., Munya, P. and M. Trevino, 2006. An

approach to collaborative sensemaking process. In:

Proc. 11

Int. Command and Control Research and

Technology Symposium, Cambridge, UK. 20 pages.

Paul, S. A. and M. R. Morris, 2009. CoSense: enhancing

sensemaking for collaborative web search. In: Proc.

Int. Conf. on Human Factors in Computing

Systems, Boston, USA. NY: ACM, 1771-1780.

Paul, S. A., and Reddy, M. C., 2010. A Framework for

Sensemaking in Collaborative Information Seeking.

In: 2

Workshop on Collaborative Information

Retrieval, Savannah, GA.

Paul, S. A. and Reddy, M. C., 2010. Understanding

Together: Sensemaking in Collaborative Information

Seeking. In: Proc. of the 2010 ACM Conf. on

Computer Supported Cooperative Work, Savannah,

Georgia, USA. NY: ACM, 321-330.

Poltrock, S., Grudin, J., Dumais, S., Fidel, R., Bruce, H.

and Pejtersen, A. M., 2003. Information seeking and

sharing in design teams. In: Proc. of the 2003 Int.

ACM SIGGROUP Conf. on Supporting Group Work,

Sanibel Island, USA. NY: ACM, 239-247.

Puzzle, 2010. Tiling Puzzle. In: Wikipedia: The Free

Encyclopedia. OnLine, Available from: http://

en.wikipedia.org/wiki/Tiling_puzzle [Accessed 6

November 2011].

Räsänen, M. and Nyce, J. M., 2006. A New Role for

Anthropology? – Rewriting ‘Context’ and ‘Analysis’.

In: Proc. of Nordic Conf. on Human-Computer

Interaction, Oslo, Norway. NY: ACM, 175-184.

Ravid, S., Rafaeli, A. and Shtub, A., 2008. Facilitating

Collaborative Sensemaking in Distributed Project

Teams Using Computer Supported Cooperative Work

(CSCW) Tools. In: Proc. of the 2008 ACM Conf. on

Human Factors in Computing Systems, Florence, Italy.

ACM, 1-5.

Reddy, M. C. and Jansen, B. J., 2008. A model for

Understanding Collaborative Information Behavior in

Context: A Study of two Healthcare Teams.

Information Processing and Management, 44 (1), 256-

273.

Ruhleder, K. and Jordan, B., 1997. Capturing, complex,

distributed activities: video-based interaction analysis

as a component of workplace ethnography. In: Proc. of

the IFIP TC8 WG 8.2 Int. Conf. on Information

Systems and Qualitative Research. Philadelphia, USA.

London: Chapman & Hall, 246-275.

Sommerville, I. and Ransom, J., 2005. An Empirical Study

of Industrial Requirements Engineering Process

Assessment and Improvement. ACM Transaction on

Software Eng. and Methodology, 14 (1), 25-117.

Sommerville, I., 2006. Software Engineering. Boston:

Addison-Wesley.

Stemler, S., 2001. An overview of content analysis.

Practical Assessment, Research & Evaluation, 7 (17).

Available from: http://pareonline.net/getvn.asp?v=7&

n=17 [Accessed 19 November 2011]

Weick, K. E., 1995. Sensemaking in Organizations.

California: Sage Publications.

Zoowghi, D., and Coulin, C., 2005. Engineering and

Managing Software Requirements. Secaucus:

Springer-Verlag NY, 2005.

REQUIREMENTSELICITATIONMETHODFORDESIGNINGVIRTUALCOLLABORATIVESYSTEMSWITH

COLLABORATIVESENSEMAKING