USER ENJOYMENT AND PERFORMANCE IN COLLABORATIVE

AND COOPERATIVE GAMES IN SHARED 3D VIRTUAL

ENVIRONMENTS

Anastasiia Beznosyk, Peter Quax, Karin Coninx and Wim Lamotte

Hasselt University, tUL, IBBT, Expertise Centre for Digital Media

Wetenschapspark 2, B-3590 Diepenbeek, Belgium

Keywords:

Collaboration, Cooperation, Shared 3D virtual environments.

Abstract:

Increased interest in shared virtual environments has resulted in a necessity to investigate different factors

that inﬂuence user interaction, both with the environment and other users. Introducing different types of joint

activities into computer games can support a high level of realism and user engagement. This paper investigates

two types of shared work: collaboration – where users individually perform actions in the environment towards

a common goal, and cooperation – where users simultaneously act on the same objects in a shared environment.

A basic computer game which requires joint work between all players was developed and used as a case study.

We analyzed user performance and enjoyment in order to see which type of shared work is more preferable.

Furthermore, the inﬂuence of user expertise on their enjoyment was also checked. From this particular study,

we conclude that participants prefer cooperation over collaboration, as it provides more active and realistic

performance. User expertise does not signiﬁcantly inﬂuence enjoyment in this type of shared work, indicating

that inexperienced players can enjoy the game equally as the experienced.

1 INTRODUCTION AND

RELATED WORK

Collaboration at a distance has become an important

topic within the research community. It gained in-

terest due to the growing popularity of virtual com-

munities (e.g. SecondLife

) and Massive Multiplayer

Online Role Playing Games (e.g. World of War-

craft

). Their success lies beyond advanced three-

dimensional (3D) graphics and immersion, in provid-

ing the possibility to interact and play with other play-

ers, either as a team or against each other. While most

interaction in these applications is limited to com-

munication and navigation, collaboration on closely-

coupled tasks (Otto et al., 2006), – where the ac-

tions of collaborating people are directly depending

on each other, – has received less focus.

Three levels of shared work are deﬁned in vir-

tual environments (VE) (Ruddle et al., 2002). The

ﬁrst level deﬁnes collaboration as an ability to com-

municate and perceive the shared environment. The

http://www.secondlife.com

http://www.worldofwarcraft.com

second level extends this ability, allowing individ-

ual modiﬁcation of the environment. The third one

is deﬁned by simultaneous impact of multiple users

on the same objects or parts of the environment (e.g.

shared object manipulation). The last one is also often

called cooperation (Broll, 1995), and this term will be

used throughout the paper to refer to the third type

of shared work. While the ﬁrst and the second lev-

els are actively used for most existing 3D multi-user

computer games (for example, World of Warcraft),

where people co-exist and interact in order to achieve

a shared goal, we assume exactly the third type of

collaboration can increase user engagement and add

more realism to the virtual performance. One of the

recent games that has become popular quickly and

that supports the third type of collaboration is Little

Big Planet

, where players encounter a shared chal-

lenge or obstacle, and often have to manipulate the

same object simultaneously. Such popularity triggers

a necessity to fully assess the potential of coopera-

tive activities for 3D computer games. Further possi-

ble advantages over collaboration based on individual

performance have to be investigated as well in order to

http://www.littlebigplanet.com

302

Beznosyk A., Quax P., Coninx K. and Lamotte W..

USER ENJOYMENT AND PERFORMANCE IN COLLABORATIVE AND COOPERATIVE GAMES IN SHARED 3D VIRTUAL ENVIRONMENTS.

DOI: 10.5220/0003361403020307

In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP-2011), pages 302-307

ISBN: 978-989-8425-45-4

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

provide players with an enjoyable game experience.

There is a substantial amount of work investigat-

ing different aspects of collaboration and cooperation

within small groups. Although the research is not al-

ways related to computer games, we believe that most

of the ﬁndings can be applied in the context of game

development. The major part of research in this area

focuses on the implementation of collaborative and

cooperative interaction techniques (Bowman et al.,

2008), such as shared object manipulation. For exam-

ple, Pinho et al. (Pinho et al., 2008) present a frame-

work that supports development of cooperative ma-

nipulation techniques. Another cooperative manip-

ulation technique called “SkeweR”, that allows two

users to move the same virtual object simultaneously

in a VE, is presented in (Duval et al., 2006). Be-

sides interaction techniques, several researches cov-

ered such issues like providing correct perspective

views over the shared environment (Provenzano et al.,

2007) and support for various types of feedback (Ul-

lah et al., 2009).

Also, several studies focus on perceptional as-

pects of collaboration and cooperation, namely pres-

ence and co-presence within shared virtual environ-

ments (Schroeder et al., 2001; Heldal et al., 2005).

These studies investigate factors that contribute to

effective collaboration and people’s behavior in dis-

tributed VEs while performing closely-coupled tasks.

Similar work performed by Roberts et al. (Roberts

et al., 2003) focuses on concurrent interaction with

shared objects by users of a variety of display system

conﬁgurations.

While most existing research focuses either on

collaboration (Heldal et al., 2005) or coopera-

tion (Roberts et al., 2003) separately, we realize the

importance of their comparison. The contribution of

our work is the comparative analysis of collaborative

and cooperative activities, based on users’ behavior

and preferences while playing a multi-user 3D com-

puter game.

In following section, we describe the conducted

experiment. Section 3 presents the results of the user

experiment, which are discussed and concluded in

Section 4.

2 EXPERIMENT DESCRIPTION

In order to compare collaboration and cooperation,

and more speciﬁcally to assess the inﬂuence of differ-

ent factors on users’ enjoyment, an experiment was

conducted. The purpose of this experiment was to de-

termine the factors that have a positive impact on in-

teraction in a shared virtual environment for collabo-

ration and cooperation. We wanted to see which setup

is found to be more enjoyable and engaging by users

in order to provide some guidelines for further appli-

cation development (e.g. games), where this particu-

lar type of joint work can be beneﬁcial.

We believe that cooperative activities, although

not always easy and straightforward to perform, will

increase enjoyment of users’ interaction and provide

a more convincing shared experience of working to-

gether. We expect therefore cooperative activities to

be more preferable for most participants based on sev-

eral criteria (i.e. performance, how natural the inter-

action was, etc.) and provide more enjoyable interac-

tion among partners when compared to the collabora-

tive condition.

During the experiment ten groups of three people

were sequentially asked to play a basic 3D game with

the shared goal of collecting 20 digits in the environ-

ment together with two partners. The game was rep-

resented as an environment that consists of two virtual

houses containing 20 cubes (ﬁgure 1). Half of these

cubes had a digit on one of their sides. In order to

complete the game participants were asked to calcu-

late the sum of all digits, and then to select the correct

number (corresponding to the calculated sum) out of

four options, presented as a four-item menu. To ex-

plore the environment, participants had to select and

manipulate objects.

Figure 1: The task.

The experiment consisted of 2 sessions: collab-

orative and cooperative. For every session a differ-

ent environment was created with the same level of

complexity. In the collaborative session every object

could be moved or rotated by a single person. In the

cooperative session there were two types of objects:

those that can be manipulated by one user and those

that required joint manipulation by two users. If an

object of the second type was selected, a help message

was generated to inform other participants within the

same house in the virtual environment that there is a

person requiring a partner’s help. However, the mes-

sage did not indicate who triggered the call for help

and where that person was located. Participants had

to discuss this event to discover the feasibility to help

USER ENJOYMENT AND PERFORMANCE IN COLLABORATIVE AND COOPERATIVE GAMES IN SHARED 3D

VIRTUAL ENVIRONMENTS

303

each other. If none of the co-players was in the same

house, the person who initially selected such an object

got another message, indicating that his/her partners

were not in the same area.

To indicate successful object selection visual feed-

back was provided. The object highlighted in red in-

dicated it is already selected but in order to manipu-

late it two people are required. The object highlighted

in green indicated that it is selected and can be ma-

nipulated. To select, manipulate and deselect an ob-

ject the SpaceMouse or the SpaceExplorer was used.

To move or rotate object jointly with a partner, par-

ticipants had to simultaneously move an object in the

same direction using their devices.

Although we designed this game to be played re-

motely, for observational reasons, participants were

present in the same room (as shown in ﬁgure 2). Dur-

ing the experiment participants were allowed to com-

municate. Due to the co-located setup participants

were able to talk to each other without using any

communication technology (e.g. Skype). Because

every participant was focused on actions happening

on his monitor, other communication channels of real

life (facial expressions, gestures, etc.) were not taken

into account. Apart from voice communication they

had a text messenger running on their laptops which

was used to share already found digits to the part-

ners. Once all 20 digits were found, one of the group

members calculated their sum and selected the answer

from the four-item menu (ﬁgure 1). Participants were

supervised in order to avoid guessing the correct an-

swer from the menu.

Figure 2: The experiment setup.

The time to complete each session was captured.

After each session participants were asked to com-

plete an individual questionnaire regarding their ex-

perience. In this questionnaire participants had to

evaluate their performance and preference for every

condition on a 10-point likert scale. Questions were

grouped into several categories: enjoyment, level of

working together, performance and communication

activity. Additionally, we asked participants to indi-

cate the level of their expertise in collaborative virtual

environments.

3 RESULTS

The main goal of the conducted experiment was to

compare two levels of shared work: collaboration

and cooperation. Moreover, we present an analysis

that shows the inﬂuence of the users’ expertise on

their performance and enjoyment. Finally, we ana-

lyze users’ behavior based on observations. The com-

parative analysis is performed using paired-samples

t-tests.

3.1 Comparison of Collaboration and

Cooperation

3.1.1 Task Completion Time

The analysis has shown that it took signiﬁcantly

longer for participants to complete the task in the co-

operative setup (t(9) = −3.867, p = 0.004). Figure 3

shows the task completion time for every group in

both conditions.

Figure 3: Task completion time for collaboration and coop-

eration.

As long task completion time may cause boredom

and fatigue, it is important to investigate the effect

of the length of the task on the overall enjoyment of

users. We analyzed both types of joint work in or-

der to search for any correlation with the time. We

found a signiﬁcant negative correlation (R

= −0.687,

p = 0.028) between enjoyment of collaboration and

total time. For the cooperative condition no correla-

tion was found between task completion time and en-

joyment, indicating that people enjoy interaction even

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

304

if more time to complete the task was required. This

result has an important impact for game development,

where it is important to engage players for a longer

period of time. Nevertheless, it is still necessary to

investigate the inﬂuence of time on users’ enjoyment

for continuous long-term gaming.

From observations we discovered that cooperation

took longer because it was often necessary to wait for

the partner in order to perform the joint activity. It

also took some time to ﬁnd the location of a person

who needed help in the virtual environment. Taking

into account these ﬁndings we can conclude that com-

pletion time can be considered as a task-dependent

variable, but users still enjoy longer interaction when

they are cooperating with their partners.

3.1.2 Enjoyment and Performance

Participants had to rate two types of shared work from

different perspectives. We collected responses of in-

dividuals regarding their work under both conditions.

All questions were grouped into several categories

which were analyzed separately.

In table 1 all statistics are given in order to com-

pare the collaborative and cooperative conditions,

based on the individual responses. They are grouped

similarly to the categories of the questionnaire given

to participants. As we can see, participants enjoyed

the cooperative task signiﬁcantly more. They per-

formed almost equally active as no difference was

found across the conditions. Furthermore, we ad-

ditionally asked them to evaluate their group activ-

ity in the collaborative and cooperative settings. We

found that as a group they performed signiﬁcantly

more active during the cooperation (t(9) = −3.250,

p = 0.01). This can be explained by the nature of the

given task as in a cooperative setting participants are

more dependent on their co-players.

Manipulation of objects during the cooperative

task required more effort and cohesive work from par-

ticipants, and was indicated to be signiﬁcantly harder

than single-user object manipulation. But, as indi-

cated before, such difﬁculty did not signiﬁcantly in-

ﬂuence the overall user enjoyment of cooperation.

Another question that was asked related to users’

level of working together with others, rather than on

their own. Responses showed another advantage of

cooperative work – it provides a signiﬁcantly higher

feeling of working together as a group comparing to

the one in collaboration. Furthermore, participants

also indicated the cooperative condition as a more

natural way of working together as a team. We also

found that there was no signiﬁcant difference across

the two conditions regarding the level of awareness

that participants perceived, indicating that they were

equally aware of the situation in the environment dur-

ing their activity.

The activity of communication and the level of

its importance were also analyzed. As mentioned

before, cooperation required much more communi-

cation and coordination from participants. Statisti-

cal analysis has shown that individuals communicated

signiﬁcantly more when they cooperated. Users also

indicated that communication was more important for

this session. They said that the collaboration session

“did not require much communication, and the task

itself can be actually completed without communica-

tion”, while for the cooperative session “it would be

impossible to complete the task if the communication

is prohibited”.

3.2 Inﬂuence of User Expertise on

Shared Work

We asked participants to deﬁne their level of expertise

in 3D single-user (M = 5.2, SD = 2.93) and multi-

user (M = 3.67, SD = 2.43) virtual environments. Al-

though most participants did not have much experi-

ence in shared virtual activities they still enjoy play-

ing the game under both conditions. Moreover, we

found no correlation between expertise level and en-

joyment. This indicates that both experienced and less

experienced users can equally enjoy activities with

others in multi-user virtual environments. Addition-

ally, no inﬂuence of user expertise on completion time

was found.

Test groups were randomly formed, so that certain

groups contained only inexperienced or only experi-

enced users, or were mixed. That is why we addi-

tionally compared group enjoyment and average ex-

perience within each group. We found no signiﬁcant

inﬂuence of the experience on the group enjoyment.

In spite of such heterogeneity, user interaction during

both sessions was enjoyable for every participant.

Additionally, we found no correlation between ex-

pertise and level of working together, which proves

the fact that despite lack of experience users got an

equally high perception level of working together

with their collaborators for both conditions.

3.3 Users’ Behavior

Previous sections already partially covered our analy-

sis of users’ behavior. Further analysis of user obser-

vations reveals interesting information regarding the

users’ experience. This section presents the collected

and analyzed qualitative information concerning the

users’ experience while collaborating and cooperat-

ing.

USER ENJOYMENT AND PERFORMANCE IN COLLABORATIVE AND COOPERATIVE GAMES IN SHARED 3D

VIRTUAL ENVIRONMENTS

305

Table 1: Comparison of collaboration and cooperation based on individual responses.

Factor Responses

Task performance

Enjoyment t(29) = −2.421, p = 0.022 M

coll

= 6.37, SD

coll

= 1.5 M

coop

= 7.23, SD

coop

= 1.52

Activity t(29) = −0.711, p>0.05 M

coll

= 7.43, SD

coll

= 1.3 M

coop

= 7.63, SD

coop

= 1.3

Object manipulation t(29) = 2.079, p = 0.047 M

coll

= 7.1, SD

coll

= 1.9 M

coop

= 6.3, SD

coop

= 1.37

Working with others

Level of togetherness t(29) = 7.623, p = 0.000 M

coll

= 5.33, SD

coll

= 2.12 M

coop

= 8.07, SD

coop

= 1.23

Naturalness t(29) = −3.713, p = 0.01 M

coll

= 6.44, SD

coll

= 1.38 M

coop

= 7.37, SD

coop

= 1.45

Awareness t(29) = −1.925, p>0.05 M

coll

= 5.87, SD

coll

= 1.63 M

coop

= 6.43, SD

coop

= 1.70

Communication

Activity t(29) = −5.835, p = 0.000 M

coll

= 5.77, SD

coll

= 1.87 M

coop

= 7.87, SD

coop

= 1.079

Importance t(29) = −9.380, p = 0.05 M

coll

= 5.1, SD

coll

= 1.73 M

coop

= 8.33, SD

coop

= 1.06

Preferences. The analysis shows that participants

preferred the cooperative session signiﬁcantly more

than the collaborative one, rating their enjoyment

higher. Observing people playing the two games al-

lows us to draw a conclusion what actually caused

such enjoyment.

One of the main reasons why people liked both

conditions is the collaborative nature of the games.

The fact that they were not single players had a posi-

tive impact on their overall experience. Collaboration

was considered less interesting and less fun compared

to the cooperative session because of the reduced in-

teraction. Participants performed more individually,

rather than as a group to complete the task. Some

players said they did not felt part of a group, but rather

co-existed with others.

Cooperation involved more activity in order to

complete the game, resulting in a high level of inter-

action with other participants. Although it was often

necessary to wait for a helper, participants still were

excited to perform joint activities. Moreover, frustra-

tion caused by different views on the same objects,

did not inﬂuence the enjoyment. Almost all players

experienced more fun when playing the cooperative

game, as it really gave the feeling of team work. Peo-

ple pointed out that “being fun” is more crucial than

“being difﬁcult” when playing computer games.

Voice Communication. In our previous

work (Beznosyk et al., 2010) we demonstrated

that the inclusion of communication was signiﬁcantly

important for collaborative tasks in the virtual

environments. In the current experiment, voice com-

munication was included for both the collaborative

and the cooperative sessions. We wanted to determine

when it was necessary to talk with the partners, and

what kind of communication was used while working

under different conditions.

First of all, we noticed that participants who had

difﬁculties with the actual performance (e.g. ob-

ject manipulation) used communication to compen-

sate their skills. When it was too hard to manipulate

an object, they became more active in a group dis-

cussion. They communicated actively during strategy

discussion and along the task itself. During the col-

laborative session, communication was mainly used

for strategical discussions at the beginning, to do a

bit of joking or to keep others aware what the partic-

ipants were doing. Communication was indicated as

the only instrument of collaboration, as players per-

formed individually rather than as a team. We also

observed that a few pairs did not even discuss even

their strategy, which resulted in longer task comple-

tion time. They concluded that it was not necessary to

communicate, because they were able to see if there

is anyone around.

During the cooperative session, all pairs admit-

ted the necessity of communication. Players not only

talked about their strategy or role division among

the group members, but also coordinated their move-

ments actively. They indicated that the help message

they saw, when someone picked up an object which

had to be manipulated jointly, does not really help

if you cannot communicate about the object descrip-

tion and location. For the cooperative condition the

communication was almost continuous, due to some-

one needing help. Vocal conversation was also used

as an awareness mechanism in the same way as for

collaboration. People communicated regarding their

positions and actions in the virtual environment, thus

making their performance smoother.

It is obviously preferable to include voice commu-

nication when working together with others. It con-

tributes by making interaction more ﬂuent and realis-

tic. The main purpose of communication during the

collaboration was division of the work across group

members, which can also be replaced with a text chat.

At the same time, cooperation required communica-

tion mainly for real-time coordination, which would

be difﬁcult to complete having only a text chat.

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

306

4 DISCUSSION AND

CONCLUSIONS

The results obtained during this experiment, conﬁrm-

ing our initial assumptions, have an important impact

for the implementation of shared 3D virtual environ-

ments, in particular for game development. Two types

of joint work, collaboration and cooperation, were an-

alyzed and compared in order to see which one is

more preferred by players. We found that partici-

pants enjoyed cooperative tasks more, irrespective to

the level of their expertise in 3D environments. We

observed that non-experienced players were also ac-

tively involved in the task execution, and enjoyed it

not less than more experienced partners. Coopera-

tion also supported a higher level of working together.

The main advantage was that players not only had a

joint goal, that could be achieved concurrently, but

they really had to perform activities together in order

to reach the ﬁnal aim.

Additionally, we found no negative effect of the

longer task completion time of cooperative activities

on the user enjoyment, which existed for collabora-

tion. Therefore, we can assume that similar coopera-

tive activities – where users can truly work together in

a shared environment – will be beneﬁcial in computer

games. We do not reject collaborative activities as

they are also found to be successful for gaming (Seif

El-Nasr et al., 2010). Moreover, combining the two

types of joint work within the same games has poten-

tial. We believe, that this will provide more dynamic

interaction, as players will be able to switch their ac-

tivities from one type to another.

These results prove that incorporating such activ-

ities into computer games will increase users’ enjoy-

ment and engagement, providing truly shared experi-

ence. But the implementation of cooperative systems

for multi-user computer games requires more effort

from developers. Further research is necessary in or-

der to develop collaborative and cooperative interac-

tion techniques suitable for large groups, as well as

ﬁne-grained algorithms to keep interaction synchro-

nized among all participants.

ACKNOWLEDGEMENTS

The research described in this paper is directly funded

by Hasselt University BOF ﬁnancing. The authors

would like to thank Lode Vanacken for his assistance

with the implementation and all participants who con-

tributed to this research by taking part in the experi-

ment.

REFERENCES

Beznosyk, A., Raymaekers, C., Coninx, K., Quax, P., and

Lamotte, W. (2010). Investigating the inﬂuence of

communication and input devices on collaboration in

virtual environments. In Proc. GRAPP2010.

Bowman, D. A., Coquillart, S., Froehlich, B., Hirose, M.,

Kitamura, Y., Kiyokawa, K., and Stuerzlinger, W.

(2008). 3d user interfaces: New directions and per-

spectives. IEEE Computer Graphics and Applica-

tions, 28:20–36.

Broll, W. (1995). Interacting in distributed collaborative vir-

tual environments. In Proc. VRAIS 1995, pages 148–

155. IEEE.

Duval, T., Lecuyer, A., and Thomas, S. (2006). Skewer: a

3d interaction technique for 2-user collaborative ma-

nipulation of objects in virtual environments. 3D User

Interfaces, pages 69–72.

Heldal, I., Steed, A., and Schroeder, R. (2005). Evaluating

collaboration in distributed virtual environments for a

puzzle-solving task. In Proc. HCII 2005.

Otto, O., Roberts, D., and Wolff, R. (2006). A review on ef-

fective closely-coupled collaboration using immersive

cve’s. In Proc. of ACM VRCIA 2006, pages 145–154.

ACM.

Pinho, M. S., Bowman, D. A., and Freitas, C. M. (2008).

Cooperative object manipulation in collaborative vir-

tual environments. J. Braz. Comp. Soc., 14(2):54–67.

Provenzano, L., Delzons, J., Plnacoste, P., and Vandromme,

J. (2007). Designing viewpoint awareness for 3d col-

laborative virtual environment focused on real-time

manipulation of multiple shared objects. In Virtual Re-

ality, volume 4563 of LNCS, pages 147–156. Springer

Berlin / Heidelberg.

Roberts, D., Wolff, R., Otto, O., and Steed, A. (2003). Con-

structing a gazebo: supporting teamwork in a tightly

coupled, distributed task in virtual reality. Presence:

Teleoper. Virtual Environ., 12(6):644–657.

Ruddle, R. A., Savage, J. C. D., and Jones, D. M. (2002).

Symmetric and asymmetric action integration dur-

ing cooperative object manipulation in virtual en-

vironments. ACM Trans. Comput.-Hum. Interact.,

9(4):285–308.

Schroeder, R., Steed, A., Axelsson, A.-S., Heldal, I.,

Asa

Abelin, Widestr

om, J., Nilsson, A., and Slater, M.

(2001). Collaborating in networked immersive spaces:

as good as being there together? Computers & Graph-

ics, 25(5):781–788.

Seif El-Nasr, M., Aghabeigi, B., Milam, D., Erfani, M.,

Lameman, B., Maygoli, H., and Mah, S. (2010). Un-

derstanding and evaluating cooperative games. In

Proc. CHI 2010, pages 253–262. ACM.

Ullah, S., Otmane, S., Richard, P., and Mallem, M. (2009).

Human performance in cooperative virtual environ-

ments : the effect of visual aids and oral communi-

cation. IJVR, 8(4):79–86.

USER ENJOYMENT AND PERFORMANCE IN COLLABORATIVE AND COOPERATIVE GAMES IN SHARED 3D

VIRTUAL ENVIRONMENTS

307