AN EARLY EVALUATION METHOD FOR SOCIAL

PRESENCE IN SERIOUS GAME

Di Loreto Ines

LIC, Università degli Studi di Milano, Italy

Gouaich Abdelkader

LIRMM, Université Montpellier 2, France

Keywords:

Evaluation methods, Serious games, Social presence.

Abstract:

In recent years, there has been increasing interest, both in the potential of computer games as learning and

teaching tools, and in research into their use. However, most frameworks for serious games evaluation do

not explicitly consider social aspects. On the contrary, we believe that social aspects have to be considered

as an essential component of the ’virtual’ life of most current users. For this reason, in this paper we pro-

pose a framework able to analyze serious games social aspects and give an early evaluation of the designed

application. The evaluation framework is based on four elements: identity, space, persistence and actions.

These elements (and the behaviors they let to emerge) can be used as markers in order to evaluate if or not

our systems is able to facilitate the feeling of social presence and then social learning. The result of such

an evaluation can be useful to designers in order to understand if the systems lack of functionalities before

starting the implementation and thus return on the phase of design to add missing elements.

1 INTRODUCTION

In recent years, there has been increasing interest,

both in the potential of computer games as learn-

ing and teaching tools, and in research into their use

(Nicola, 2007). However, most evaluation frame-

works for serious games do not explicitly consider

social aspects. On the contrary, we believe that social

aspects have to be considered as an essential compo-

nent of the ’virtual’ life of most current users. Cur-

rent digital cultures - such as blogging and gaming -

take for granted social web features and expect them

to be available into any application. An application

that fails in presenting at least familiar social web fea-

tures would be considered as a regression and may

be rejected. The above mentioned considerations em-

phasize the need to understand social dynamics when

designing any application in our era. This is also

true for serious games, and in particular for serious

games where the social aspect impacts over the learn-

ing aspect. Besides, serious games evaluation are per-

formed a posteriori. On the contrary, we claim that

an a priori evaluation of serious games can be useful.

In fact, while users are able to evaluate the quality of

their experience, for the most part they are not able

to understand which feature/characteristic generates

a poor performance. This doesn’t mean that an user

centered design approach (see e.g.,Vredenburg et al.,

2001) is not useful when designing a serious game

with social aspects. On the contrary, a deep analy-

sis of users’ needs is at the basis of any application

development. However, between user centered de-

sign, and user satisfaction measurement we want to

add an intermediate layer. In fact, an additional level

for an early evaluation approach can be useful at dif-

ferent levels for the serious game creators. Firstly,

the early evaluation of the game design is performed

when multiple experts (experts on learning, computer

scientists and so on) still works together, thus foster-

ing communication between them if some problem

arises. Secondly, an early evaluation of the designed

application can help designers to anticipate several

problems that can arise before starting the implemen-

tation and thus return on the phase of game design

to add missing elements, restraining in this way de-

velopment cost. For the above mentioned reasons, in

this paper we propose an early evaluation framework

able to analyze social aspects of serious games and

Loreto Ines D. and Abdelkader G. (2010).

AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME.

In Proceedings of the 2nd International Conference on Computer Supported Education, pages 94-101

DOI: 10.5220/0002797500940101

 SciTePress

give an early evaluation of the designed application.

The evaluation framework is based on four elements:

identity, space, persistence and actions. These ele-

ments (and the behaviors they let emerge) can be used

as ’markers’ (or ’indicators’) in order to evaluate if or

not our systems is able to facilitate the feeling of so-

cial presence.

Following sections of this paper will be devoted to

(i) show the importance of social presence and social

learning in serious games, (ii) present the framework

and a method for early evaluation based on the frame-

work (iii) show the results of an experiment where the

early evaluation method helped the designers to detect

design weaknesses.

2 MOTIVATION

From an historical point of view, the serious games

movement started with the U.S. Army’s release of

the video game America’s Army in 2002 (Ameri-

can’s Army, 2002; Gudmundsen, 2006). The same

year the Woodrow Wilson Center for International

Scholar in Washington, D.C. founded the Serious

Games Initiative, and the term “serious games” be-

came widespread (Serious Games Initiative, 2002).

The term itself is nowadays established, but there

is no current single deﬁnition of the concept. Seri-

ous games usually refers to games used for training,

advertising, simulation, or education, which are de-

signed to run on personal computers or video game

consoles. It is important to note that there is no

clear evidence that the serious game approach is bet-

ter that classical approaches, such as e-learning, for

knowledge transmission and acquisition. However,

for some domains such as health and reeducation

some works have established the beneﬁt of serious

games to make reeducation sessions more motivating

and less exhausting (Burke et al., 2009)

. In addition,

it has been demonstrated that social presence is one of

the most signiﬁcant factors to examine in distance ed-

ucation. Many studies dealing with social presence

on CMC (computer-mediated communication) have

been done in standard educational settings and organi-

zational settings. For example Tu (Tu, 2000) analyzed

the relationship between social presence and the so-

cial learning theory. For the author, social presence

is required to enhance and foster on-line social inter-

action, which is the major vehicle of social learning.

Also for Wenger (Wenger, 2000) learning can be de-

ﬁned as an interplay between social competence and

This is the context of our research project MoJOS

(http://www.mojos.fr/) which main objective is to build se-

rious games for upper lamb reeducation after strokes.

personal experience: our belonging to social learning

systems can take various forms at various levels be-

tween local interactions and global participation. This

is also true from the serious game point of view. For

Kiili (Kiili, 2005) social games consist of both, indi-

vidual and social events. In fact, in online multiplayer

games players can collaboratively solve and explore

problems in a shared game world, but ultimately crit-

ical reﬂection and knowledge construction occurs in

a private world. It seems relevant, then, to evaluate

serious games with social aspects from both points of

views, personal and social. As a consequence of the

above mentioned reasons we claim that because so-

cial activities are important for learning we need an

evaluation framework able to take into account social

aspects. In particular, in this paper we will propose

an early evaluation framework and a method, able

to evaluate social presence ’potential’. The evalua-

tion framework endorses Witmer’s ideas (Witmer and

Singer, 1998) stating that the effectiveness of seri-

ous games, and more generally virtual environments,

is linked to the sense of presence reported by users.

As an example, in synthetics worlds (such as Second

Life) an active exploitation of our senses can create a

psychological sense of presence, or, in other worlds,

the illusion that “I’m in the virtual world and not in

my house” and, as a consequence, that “I’m there with

other people” (Biocca, 1997).

In the rest of this section we will deﬁne the con-

cept of presence (and its importance for social envi-

ronments). Next section will be devoted to present the

framework and the method for the early evaluation.

2.1 Presence and Social Presence in

Social Environments

In his paper ’Measuring Presence in Virtual Environ-

ments: A Presence Questionnaire’, Witmer (Witmer

and Singer, 1998) says that the effectiveness of vir-

tual environments (VEs) has often been linked to the

sense of presence reported by users of those VEs. For

the author, presence is deﬁned as the subjective ex-

perience of being in one place or environment, even

when one is physically situated in another. In ad-

dition, presence is a normal awareness phenomenon

that requires directed attention and is based in the in-

teraction between sensory stimulation, environmen-

tal factors that encourage involvement and enable im-

mersion, and internal tendencies to become involved.

What is interesting for our purpose is that we can ex-

tend the concept of presence to the concept of social

presence. Social presence is, in fact, deﬁned as the

“degree of salience of the other person in the inter-

action and the consequent salience (and perceived in-

AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME

timacy and immediacy) of the interpersonal relation-

ships”(Short et al., 1976, p. 65). As we have already

said, social presence has been proved to be necessary

for both, interface design of social applications (Xu

et al., 2006), and learning purposes (Kasvi, 2000). In

addition, we claim that social presence is also im-

portant in serious games. We also state that we can

identify several ’indicators’ in order to measure the

potential presence awareness of our application. In

other words: the absence of these indicators in our

virtual environment (in our case a serious game) is a

signal of a not well designed system, not able to sup-

port social interactions and social learning. As we

have said, being able to do such an evaluation at early

stage (i.e., before starting implementation) has ma-

jor advantages: it can simplify the work of designers

(that can return on their design before development)

and help to build better applications while reducing

development costs.

3 DEFINING ’INDICATORS’ FOR

SOCIAL PRESENCE

This section presents core elements of the framework

that can be used to evaluate social presence ’poten-

tial’ in an application at early stage. The framework

is based on four elements: identity, space, persistence

and actions. These elements are motivated by an em-

pirical analysis of current social software and sup-

ported by major ﬁndings from psychology and sociol-

ogy. In fact, these elements represent core features of

any Social Interactive Systems (SIS) targeted towards

young generations (and thus also serious games with

social capabilities). Consequently, they represent an

interesting evaluation criteria in order to capture at

early stage the potential presence awareness of the ap-

plication being designed. Hereafter, the semantics of

each element of the framework is described more in

details.

Identity. Our point of view about Identity is the

same as social psychology’s later approaches, which

consider individual and social identity not as stable

characteristics, but rather as a dynamic phenomenon

(Harré and Langenhove, 1991). In these approaches,

the choice about what possible self to show is driven

by strategic moves (e.g., what features are more rel-

evant and effective for self presentation) which par-

ticipants can make within a particular situation. In

describing everyday interactions, Goffman (Goffman,

1959) distinguished between two ways of expressing

information: information that is given and informa-

tion that is given off. Information that is given is the

conscious content of communication, the voluntary,

symbolic actions that are mutually understood, for ex-

ample, a person who describes their anger is giving

information about their emotional state. In talking

about their anger, however, the person also gives off

information, through para-verbal characteristics such

as tone, volume, the choice of words, and non-verbal

cues. While information that is given is considered

to be within the actor’s control, information that is

given off is perceived by the audience to be uninten-

tionally communicated. A classical example of ’iden-

tity announcement’that has intentionally and uninten-

tionally elements is avatar personalization. While we

will not enter in detail here on its implications, the

avatar is a visual claim for personal expression that

is constantly worked on. This continuous work rein-

forces the concept of presence and thus social pres-

ence. The explicit speciﬁcation of a social network of

acquaintance can be seen as collateral information. If

it’s true that social networks are built via a series of

invitations, usually members also have some control

over the visibility of their network for others. This

means that, for impression management, a user will

show only networks he/she wants to show. For in-

stance some members can decide to make their social

networks visible only to their direct acquaintances.

In this case, there is a ’given’ information (the user

chooses what to show about his/her identity), but also

a ’given off’ information (derived e.g., from the kind

of groups a user showed/joined). From a design point

of view we can say that, allowing both the kinds of

identity representation becomes the starting point for

a social, evolving identity.

Space. If we look carefully, the language we use to

describe our experience of the virtual environment is a

reﬂection of an underlying conceptual metaphor: ’Cy-

berspace as Place’ (Lakoff and Turner, 1988). This

means that we are transferring certain spatial char-

acteristics from our real world experience over the

virtual environment. The metaphor ’Cyberspace as

Place’ leads to a series of other metaphorical infer-

ences: cyberspace is like the physical world, it can

be ’zoned’, trespassed upon, interfered with, and di-

vided up into a series of small landholdings that are

just like real world property holdings. As you no-

ticed, we joined together the terms space and place.

In reality, for the good functioning of a SIS it is im-

portant to distinguish between them. Actually, the lit-

erature about space and place is fairly massive and

diverse. A converging deﬁnition of the difference be-

tween space and place does not exist, however we can

list some interesting deﬁnitions adapted from (Car-

mona et al., 2003) that try to capture differentiation

aspects between space and place: Space is alienation;

CSEDU 2010 - 2nd International Conference on Computer Supported Education

place is identiﬁcation (according to Norberg-Schulz),

spaces are the basic divisions of our surroundings;

place is our history and adaptation of them (landscape

historian JB Jackson), space is the scene of being;

place is a site where human modes of being are well

provided for (Heidegger), places are essentially cen-

ters of meaning constructed out of living experience.

By imbuing them with meaning, individuals, groups

or societies change ’spaces’ into ’places’ (urban de-

signer Edward Relph). Besides, in his book (Car-

mona et al., 2003), about urban spaces and places,

Carmona distinguishes among dimensions of an ur-

ban space. While space is divisible, place is not. Place

is complex, inextricably multi-dimensional, lived, ex-

perienced, meaningful (with of course multi - mean-

ings). This means that while space is a well-deﬁned

topographical entity, place is the result of human in-

habitation, (social) interaction, and the like. We are

located in spaces, but we act and develop individual

and social experiences in places. We claim that in or-

der to design a social serious game, it is essential to

allow by design the creation of public (at different lev-

els) places for aggregation but also the creation of pri-

vate places (Wenger et al., 2002). Besides, the level

of personalization can be used in order to allow the

shift from spaces to places. Only taking possession of

the space, and manipulating it to turn it in something

we like, we can transform it in a place.

Persistence. As we have seen, in order to create a

social identity in an online environment several ele-

ments are required. The ﬁrst of these is persistence (of

personal identity in the system). In a non-persistent

world it is not possible to have an history of actions

and thus allow, for example, the creation of a reputa-

tion like in real life. Moreover, Danet (Danet et al.,

1997) , argued that synchronicity is associated with

’ﬂow experiences’, a state of total absorption and a

lack of awareness of time passing. This idea of syn-

chronicity is linked to the idea of temporality, a linear

procession of past, present, future. This particular nu-

ance (synchronicity as process) is very interesting if

we think that interaction with media and media per-

ception is changed. In fact, advances in technology

and the speed of network connections are blurring

distinctions between synchronous and asynchronous

communications (Joinson, 2003). Synchronous and

asynchronous communications are thus processes that

happens during time. The idea of communication as

a process is totally consistent with the idea of persis-

tence and is another element supporting social aware-

ness.

Action. In this part, we discuss physical and psy-

chological mechanisms that regulate human actions

in order to understand why the action element have

to be considered as a pillar in the design of social se-

rious game. The ﬁrst theory we want to describe is

the so-called: thinking through doing. This theory

describes how thought (mind) and action (body) are

deeply integrated and how they co-produce learning

and reasoning (Klemmer and Hartmann, 2006). Jean

Piaget (Piaget, 1952) postulated that cognitive struc-

turing requires both physical and mental activity. In a

very basic sense, humans learn about the world and its

properties by interacting within it. As a second sup-

port we can cite embodied cognition. Theories and

research of embodied cognition regard bodily activ-

ity as being essential to understanding human cog-

nition (Pecher and Zwaan, 2005). While these the-

ories address cognition through action in physical en-

vironments, they also have important implications for

designing interactive systems. In fact, body engage-

ment with virtual environments constitutes an impor-

tant aspect of cognitive work. Because an action is

always an action-over-something, the kind of interac-

tion spaces and objects we create in a Social System

will inﬂuence which cognitive work the user will do

over the system.

3.1 The Overall Framework

We can think about each of the above described el-

ement as a line (an axis) that starts from the lack of

presence of the element to the fulﬁllment of its pres-

ence for a social serious game. For example, for

the concept of identity when totally missing repre-

sents anonymity while when fulﬁlled represents a so-

cial presence (with intermediate points such as per-

sonal identity construction). For the concept of space

when missing represents a topographical space while

when fulﬁlled represents social places (with interme-

diate points such as third places and personal places).

For the concept of time its totally lack is system ’am-

nesia’ while its fulﬁllment is memory (with interme-

diate points linked more or less to the concept of per-

sistence). Finally, for the concept of action its totally

lack is the obstruction of action (i.e., my user can only

look at my application) while its fulﬁllment is social

actions (with intermediate points such as public per-

sonal actions and the like). However, the total frame-

work is not simply a list of elements (i.e., its applica-

tion doesn’t mean to put one after the other the four

elements in your system) but it’s created through the

delicate balancing between them. This means that it

is up to the designer to choose which element of the

framework stress or not during the creation of a dy-

AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME

namic experience such as a serious game with social

integrated features.

4 A METHOD FOR EVALUATING

AN EXISTING GAME DESIGN

While the above described framework can also be

used in the phase of design of a social serious games,

for this paper we will focus on its application to the

phase of evaluation of the designed application. In

fact, usually a software development starts with a re-

quirement document that captures a set of require-

ments and features of the future application. This

initial requirement document is written by stakehold-

ers using natural language to express their needs. So,

during the design phase always arose the problem of

translating natural language requirements into soft-

ware requirements. In order to overcome this dif-

ﬁculty, we suggest a methodology, already used in

game developments, based on the concept of stripes.

Flynt deﬁnes a stripe as ’a set of functionality em-

bodied in a single component of the system.’(Flynt,

2004). More speciﬁcally, a stripe embodies a subset

of functionalities described in the requirements doc-

ument as a coherent succession of actions. The pro-

cess producing stripes is a top-down process starting

from the most general system features. For instance,

one can ﬁnd the login stripe that describes a set of ac-

tions allowing the user to open a session. The level

of detail and complexity grows with each stripe, but

because the detail and complexity are layered, at no

point does complexity become overwhelming thanks

to the hierarchical organization of stripes. This hi-

erarchical organization of stripes does not mean that

all stripes are independent. In fact, stripes can over-

lap when they concern similar functionalities. For ex-

ample, both login and personalization stripes can in-

volve a stripe that loads the user’s avatar and proﬁle.

However, extracting stripes from natural language de-

scription is quite difﬁcult due to inherent ambiguity

and complexity of natural language. To overcome this

difﬁculty we inserted a phase of pre-design between

concept development and classical design phase. The

idea behind this insertion is to use the phase to help

the designer to translate the design documents into re-

quirements.

As ﬁrst step the ’translator’ makes a very sim-

ple task: he takes the design and he highlights ac-

tions (i.e., verbs) in natural language. He next de-

ﬁnes for each action the elements that impact over

the framework ﬁlling a structured table. For exam-

ple he uses a ﬁeld named ’Interaction space’ (that an-

swer the question ’the action impact over?’) to iden-

Figure 1: An example of evaluated stripe.

tify space/place, and answers to the ’who’ and ’when’

questions (’who’ is doing this action? Is a persistent

action? and the like) to identify identity and persis-

tence. The application of this method is linear (in fact

you analyze the document paragraph after paragraph).

The result of this ﬁrst analysis is a set of very detailed

stripes that still have to be worked on in order to de-

rive features and components. However, this phase is

also the most interesting for early evaluation. In fact,

as we have said in classical design cycles, the eval-

uation of the system (in order to decide if or not it

is coherent with your expectation) can be done only

after the development phase (in reality, after the use

of the application). In our case we can evaluate the

system before starting the implementation.

In order to make such an evaluation we provide

weights, ranging from 0 to 1, for each stripe (i.e, we

analyze the above mentioned table for each stripe).

Next we ’ﬁll’ another table like in Fig. 1 with weights.

In our example, the modiﬁcation of the avatar impacts

over the identity but is also a persistent action. On the

contrary, it does not impact on the space of the appli-

cation. This kind of evaluation has two main advan-

tages. Firstly, it will show you if the resulting evalu-

ation is exactly what you expect from the application

(the proﬁle you have wished for) before you start the

implementation. It may happen, for example, that you

want to create a design that is balanced over all the

four elements but the result of your evaluation shows

that you lack e.g., in identity features. Secondly, it

allows you to re-deﬁne your stripes in this phase in

order to avoid the problems highlighted by the evalua-

tion (or, on the contrary, to re-think your ﬁrst design).

In next section we will describe an experiment

we held during the creation of a social serious game

where we asked our designers to apply the above de-

scribed method and to evaluate it.

5 AN EXPERIMENTATION OF

THE METHOD: SCHOOL

SOCIETY

5.1 Method

The objective of the study reported in this section

is twofold: (i) to check practicality of the proposed

framework when building an actual complex social

CSEDU 2010 - 2nd International Conference on Computer Supported Education

serious game; (ii) and to evaluate qualitatively the

beneﬁts of such methodology. A textual description

of a social serious game named School Society has

been prepared by a team of researchers composed

mainly by pedagogues (2 persons) and game design-

ers (2 persons). Computer scientists have been al-

lowed to participate but were not authorized to ad-

dress technical issues. These meetings have produced

a game design document that describes, to some ex-

tent, the “culture” (Salen and Zimmerman, 2003) and

rules of the social serious game.

To summarize, the game can be described as fol-

low: (i) every student has an avatar within the system

and lives in a house situated in a private island,(ii) the

student can use a canoe to reach one of the follow-

ing destinations:school, market place, pub, friend’s is-

land,(iii) the student takes quizzes in the school in or-

der to have a certain amount of virtual money and ex-

perience,(iv) the student buy items to decorate his/her

house in the market place and these items are visible

to friends visiting the house,(v) ﬁnally, the pub is used

to chat with other players and make new friends It is

worth noting that this is a simple description of the

game and the actual game design document is about

20 pages (4000 words and 15 ﬁgures)

A computer science engineer, with an experience

of 3 years, has been speciﬁcally hired to produce the

serious game speciﬁcations starting from the game

design document. The methodology was introduced

to the engineer during a half-day training session con-

ducted by the authors. The engineer was not allowed

to question the methodology. However, he was al-

lowed to ask questions on the game design docu-

ment by explicitly raising issues using a standard is-

sue tracking system

. Authors of the game design

document were able to answer explicitly these issues

by providing more explanations or making choices to

clear ambiguities. The identities of the engineer and

game design authors were not revealed to prevent in-

direct interactions and all interactions have occurred

explicitly through the issue tracking system. The pro-

duction of the ﬁnal serious game speciﬁcation was

performed in two phases.

• Phase 1: the engineer produced all stripes and de-

rived automatically the application proﬁle. The

result of this phase was then transmitted to game

design authors that provide feedbacks in order to

correct what is missing and guide the next phase.

At this stage, they were allowed to make changes

to the original game design document. Fig.2

shows the ﬁrst evaluated proﬁle for the applica-

tion.

The issue management system Redmine has been used.

Figure 2: The ﬁrst evaluated proﬁle.

Figure 3: The ﬁnal evaluated proﬁle.

• Phase 2: the engineer takes into account phase 1

feedbacks and makes revisions to produce a new

set of stripes and a new application proﬁle. Again

these results were presented to game design au-

thors. Fig.3 shows the ﬁnal evaluated proﬁle for

the application. As you can see the second proﬁle

is more balanced over the four axis.

It is worth to note that this balancing among all di-

mensions was decided by game designers for this par-

ticular application. Within other contexts, designers

may decide to emphasis on particular dimensions and

deliberately neglect some others. For instance, in

learning casual games, the persistence dimension can

be deliberately diminished.

The study was limited to two phases but one can

notice that this is an iterative process that can be iter-

ated more than twice.

Table 1: Duration and number of stripes produced in each

phase.

Phases Time N. of

Stripes

N. of mod.

Stripes

Phase 1 3 weeks 75 n-a

Phase 2 1 week 81 18

AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME

Table 1 presents duration of each phase. One can

notice that phase 1 is three times longer that phase

2. This is due mainly to the learning curve of the

methodology and also by the fact that most of stripes

have been produced during phase 1 and only few have

been modiﬁed and added from phase 1 and 2.

5.2 Results

The ﬁnal evaluation was conducted using interviews

with the engineer and game design authors.

5.2.1 Engineer’s Feedback

At the end of the study an interview was conducted

with the engineer. During the interview the engineer

has addressed several points about his experience with

the presented methodology. Hereafter a summary of

the main points that have been discussed:

• The engineer has found the methodology very

useful in identifying non-functional aspects of the

serious game. In fact, the culture of classical

software engineering does not highlight the im-

portance of non-functional aspects of a software

system that creates a speciﬁc atmosphere for an

application. In serious game non-functional as-

pects are crucial to capture game culture and at-

mosphere. Thanks to the methodology, all non-

functional aspects have been recorded as non-

functional stripes.

• The engineer has already some experience in de-

veloping large software systems. So, he was

aware of the cost of implementation and revision

of software systems. Consequently, he pointed

out that the early evaluation performed between

phase 1 and phase 2 has helped to revise some

fundamental decisions without having to conduct

costly implementations.

• The engineer has also pointed out that having ex-

tracted all stripes and grouping them into clusters

makes it very easy to build a detailed storyboard

of the serious game and facilitates the implemen-

tation phase. In fact, during the implementation

all stripes are translated to features that are imple-

mented by developers. This decomposition can

also help to adopt an iterative approach by decid-

ing what features to implement for each release.

5.2.2 Game Designers’ Feedback

Surprisingly, the application proﬁle has been consid-

ered by game designers as an interesting communi-

cation medium to communicate with the engineer. In

fact, they were given a feedback that represents, to

some extent, the interpretation of the engineer. For

instance after phase 1, game designers have noticed

that several points concerning social interactions have

been missed by the engineer. In fact, phase 1 game

design mentioned, but succinctly, chat between play-

ers. Since this description was not that signiﬁcant the

engineer did not create a speciﬁc stripe. When ana-

lyzing the early evaluation of phase 1, game designers

discovered this omission and have decided to add into

the game design a description of a pub to allow play-

ers socializing. This has generated speciﬁc stripes in

phase 2.

6 CONCLUSIONS AND FUTURE

WORKS

In this paper we started from the assumption that so-

cial learning and social features enabling social pres-

ence are essential when building a serious game. For

this reason we described a framework and an early

evaluation method that can be useful in order to an-

ticipate ’lacks’ in the design. The framework and the

early evaluation approach can be useful at different

levels. Firstly, the early evaluation of the game design

is performed when multiple experts (experts on learn-

ing, computer scientists and the like) have to work

together by fostering communication between them.

Secondly, an early evaluation of the designed appli-

cation can help designers to anticipate several prob-

lems that can arise when developing a serious game

with social aspects before starting the implementation

(thus returning on the game design phase to add miss-

ing elements and restraining development cost). In or-

der to support these two last statements we described

an experiment we held where a computer scientists

and pedagogues worked together in order to build a

virtual world for learning purposes. Results of this ex-

periment were in line with the idea that adding a layer

between user centered design, and user satisfaction

measurement can really help designers to build seri-

ous games in a better way. Moreover, it’s our inten-

tion to reproduce the experimentation of the method

over different serious games design, in order to bet-

ter support our ﬁndings. Besides, we also believe that

trough users’ feed-backs we could show empirical ev-

idence that the method and the framework can help to

build better serious games from the social point of

view. For this reason we are conducting a real world

experiment to relate designer’s impressions with the a

posteriori evaluation of end users.

Results of this experiment will be subject of future

publications to enrich serious game literature with ex-

perimental works to demonstrate advantages and lim-

CSEDU 2010 - 2nd International Conference on Computer Supported Education

100

itations of this approach.

REFERENCES

American’s Army (2002). The american’s army game web-

site. http://www.americasarmy.com.

Biocca, F. (1997). The cyborg’s dilemma: Progressive

embodiment in virtual environments [1]. Journal of

Computer-Mediated Communication, 3(2).

Burke, J., McNeill, M., Charles, D., Morrow, P., Crosbie, J.,

and McDonough, S. (2009). Optimising engagement

for stroke rehabilitation using serious games. The Vi-

sual Computer.

Carmona, M., Heath, T., and Tiesdell, S. (2003). Public

Places, Urban Spaces: The Dimensions of Urban De-

sign. Architectural Press, Burlington, MA.

Danet, B., Ruedenberg-Wright, L., and Rosenbaum-Tamari,

Y. (1997). Hmmm... where’s that smoke coming

from? writing, play and performance on internet relay

chat. In Journal of Computer-Mediated Communica-

tion, volume 2.

Flynt, J. P. (2004). Software Engineering for Game Devel-

opers. Software Engineering Series. Thomson.

Goffman, E. (1959). The presentation of self in everyday

life. Doubleday.

Gudmundsen, J. (2006). Movement aims to get serious

about games. USA Today, 5/19/2006.

Harré, R. and Langenhove, L. V. (1991). Varieties of posi-

tioning. Journal for the Theory of Social Behaviour,

21(4):393–407.

Joinson, A. N. (2003). Understanding the Psychology of

Internet Behaviour: Virtual Worlds, Real Lives. Pal-

grave Macmillan.

Kasvi, J. (2000). Not just fun and games – internet games as

a training medium. In Cosiga – learning with comput-

erised simulation games, pages 23–34. P Kymäläinen

and L Seppänen.

Kiili, K. (2005). Educational game design: Experiential

gaming model revised. Technical Report Research re-

port 4, Tampere University of Technology.

Klemmer, S. R. and Hartmann, B. (2006). How bodies mat-

ter: Five themes for interaction design. In In Pro-

ceedings of Design of Interactive Systems, volume 74,

pages 140–149.

Lakoff, G. and Turner, M. (1988). Categories and Analo-

gies, volume 3. University Of Chicago Press.

Nicola, W. (2007). Motivation and computer game based

learning. Proceedings ascilite Singapore 2007.

Pecher, D. and Zwaan, R. A. (2005). Grounding Cognition:

The Role of Perception and Action in Memory, Lan-

guage, and Thinking. Cambridge University Press.

Piaget, J. (1952). The origins of intelligence in children.

International University Press.

Salen, K. and Zimmerman, E. (2003). Rules of Play: Game

Design Fundamentals. The MIT Press.

Serious Games Initiative (2002). The

serious games initiative websites.

http://www.seriousgames.org/index2.html.

Short, J., Williams, E., and Christie, B. (1976). The Social

Psychology of Telecommunications. John Wiley and

Sons Ltd.

Tu, C. H. (2000). On-line learning migration: from social

learning theory to social presence theory in a cmc en-

vironment.

Vredenburg, K., Isensee, S., and Righi, C. (2001). User-

Centered Design: An Integrated Approach. Prentice

Hall PTR.

Wenger, E. (2000). Communities of practice and social

learning systems. Organization, 7(2):225–246.

Wenger, E., Mcdermott, R., and Snyder, W. M. (2002). Cul-

tivating Communities of Practice. Harvard Business

School Press, Boston.

Witmer, B. G. and Singer, M. J. (1998). Measuring pres-

ence in virtual environments: A presence question-

naire. Presence, 7:225–240.

Xu, Z., John, D., and Boucouvalas, A. C. (2006). Social

factors and interface design guidelines. In Ghaoui, C.,

editor, Encyclopedia of Human Computer Interaction,

pages 523–532. Idea Group, London.

AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME

101