AN EARLY EVALUATION METHOD FOR SOCIAL
PRESENCE IN SERIOUS GAME
Di Loreto Ines
LIC, Università degli Studi di Milano, Italy
Gouaich Abdelkader
LIRMM, Université Montpellier 2, France
Keywords:
Evaluation methods, Serious games, Social presence.
Abstract:
In recent years, there has been increasing interest, both in the potential of computer games as learning and
teaching tools, and in research into their use. However, most frameworks for serious games evaluation do
not explicitly consider social aspects. On the contrary, we believe that social aspects have to be considered
as an essential component of the virtual’ life of most current users. For this reason, in this paper we pro-
pose a framework able to analyze serious games social aspects and give an early evaluation of the designed
application. The evaluation framework is based on four elements: identity, space, persistence and actions.
These elements (and the behaviors they let to emerge) can be used as markers in order to evaluate if or not
our systems is able to facilitate the feeling of social presence and then social learning. The result of such
an evaluation can be useful to designers in order to understand if the systems lack of functionalities before
starting the implementation and thus return on the phase of design to add missing elements.
1 INTRODUCTION
In recent years, there has been increasing interest,
both in the potential of computer games as learn-
ing and teaching tools, and in research into their use
(Nicola, 2007). However, most evaluation frame-
works for serious games do not explicitly consider
social aspects. On the contrary, we believe that social
aspects have to be considered as an essential compo-
nent of the virtual’ life of most current users. Cur-
rent digital cultures - such as blogging and gaming -
take for granted social web features and expect them
to be available into any application. An application
that fails in presenting at least familiar social web fea-
tures would be considered as a regression and may
be rejected. The above mentioned considerations em-
phasize the need to understand social dynamics when
designing any application in our era. This is also
true for serious games, and in particular for serious
games where the social aspect impacts over the learn-
ing aspect. Besides, serious games evaluation are per-
formed a posteriori. On the contrary, we claim that
an a priori evaluation of serious games can be useful.
In fact, while users are able to evaluate the quality of
their experience, for the most part they are not able
to understand which feature/characteristic generates
a poor performance. This doesn’t mean that an user
centered design approach (see e.g.,Vredenburg et al.,
2001) is not useful when designing a serious game
with social aspects. On the contrary, a deep analy-
sis of users’ needs is at the basis of any application
development. However, between user centered de-
sign, and user satisfaction measurement we want to
add an intermediate layer. In fact, an additional level
for an early evaluation approach can be useful at dif-
ferent levels for the serious game creators. Firstly,
the early evaluation of the game design is performed
when multiple experts (experts on learning, computer
scientists and so on) still works together, thus foster-
ing communication between them if some problem
arises. Secondly, an early evaluation of the designed
application can help designers to anticipate several
problems that can arise before starting the implemen-
tation and thus return on the phase of game design
to add missing elements, restraining in this way de-
velopment cost. For the above mentioned reasons, in
this paper we propose an early evaluation framework
able to analyze social aspects of serious games and
94
Loreto Ines D. and Abdelkader G. (2010).
AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME.
In Proceedings of the 2nd International Conference on Computer Supported Education, pages 94-101
DOI: 10.5220/0002797500940101
Copyright
c
SciTePress
give an early evaluation of the designed application.
The evaluation framework is based on four elements:
identity, space, persistence and actions. These ele-
ments (and the behaviors they let emerge) can be used
as ’markers’ (or ’indicators’) in order to evaluate if or
not our systems is able to facilitate the feeling of so-
cial presence.
Following sections of this paper will be devoted to
(i) show the importance of social presence and social
learning in serious games, (ii) present the framework
and a method for early evaluation based on the frame-
work (iii) show the results of an experiment where the
early evaluation method helped the designers to detect
design weaknesses.
2 MOTIVATION
From an historical point of view, the serious games
movement started with the U.S. Army’s release of
the video game America’s Army in 2002 (Ameri-
can’s Army, 2002; Gudmundsen, 2006). The same
year the Woodrow Wilson Center for International
Scholar in Washington, D.C. founded the Serious
Games Initiative, and the term “serious games” be-
came widespread (Serious Games Initiative, 2002).
The term itself is nowadays established, but there
is no current single definition of the concept. Seri-
ous games usually refers to games used for training,
advertising, simulation, or education, which are de-
signed to run on personal computers or video game
consoles. It is important to note that there is no
clear evidence that the serious game approach is bet-
ter that classical approaches, such as e-learning, for
knowledge transmission and acquisition. However,
for some domains such as health and reeducation
some works have established the benefit of serious
games to make reeducation sessions more motivating
and less exhausting (Burke et al., 2009)
1
. In addition,
it has been demonstrated that social presence is one of
the most significant factors to examine in distance ed-
ucation. Many studies dealing with social presence
on CMC (computer-mediated communication) have
been done in standard educational settings and organi-
zational settings. For example Tu (Tu, 2000) analyzed
the relationship between social presence and the so-
cial learning theory. For the author, social presence
is required to enhance and foster on-line social inter-
action, which is the major vehicle of social learning.
Also for Wenger (Wenger, 2000) learning can be de-
fined as an interplay between social competence and
1
This is the context of our research project MoJOS
(http://www.mojos.fr/) which main objective is to build se-
rious games for upper lamb reeducation after strokes.
personal experience: our belonging to social learning
systems can take various forms at various levels be-
tween local interactions and global participation. This
is also true from the serious game point of view. For
Kiili (Kiili, 2005) social games consist of both, indi-
vidual and social events. In fact, in online multiplayer
games players can collaboratively solve and explore
problems in a shared game world, but ultimately crit-
ical reflection and knowledge construction occurs in
a private world. It seems relevant, then, to evaluate
serious games with social aspects from both points of
views, personal and social. As a consequence of the
above mentioned reasons we claim that because so-
cial activities are important for learning we need an
evaluation framework able to take into account social
aspects. In particular, in this paper we will propose
an early evaluation framework and a method, able
to evaluate social presence ’potential’. The evalua-
tion framework endorses Witmer’s ideas (Witmer and
Singer, 1998) stating that the effectiveness of seri-
ous games, and more generally virtual environments,
is linked to the sense of presence reported by users.
As an example, in synthetics worlds (such as Second
Life) an active exploitation of our senses can create a
psychological sense of presence, or, in other worlds,
the illusion that “I’m in the virtual world and not in
my house” and, as a consequence, that “I’m there with
other people” (Biocca, 1997).
In the rest of this section we will define the con-
cept of presence (and its importance for social envi-
ronments). Next section will be devoted to present the
framework and the method for the early evaluation.
2.1 Presence and Social Presence in
Social Environments
In his paper ’Measuring Presence in Virtual Environ-
ments: A Presence Questionnaire’, Witmer (Witmer
and Singer, 1998) says that the effectiveness of vir-
tual environments (VEs) has often been linked to the
sense of presence reported by users of those VEs. For
the author, presence is defined as the subjective ex-
perience of being in one place or environment, even
when one is physically situated in another. In ad-
dition, presence is a normal awareness phenomenon
that requires directed attention and is based in the in-
teraction between sensory stimulation, environmen-
tal factors that encourage involvement and enable im-
mersion, and internal tendencies to become involved.
What is interesting for our purpose is that we can ex-
tend the concept of presence to the concept of social
presence. Social presence is, in fact, defined as the
“degree of salience of the other person in the inter-
action and the consequent salience (and perceived in-
AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME
95
timacy and immediacy) of the interpersonal relation-
ships”(Short et al., 1976, p. 65). As we have already
said, social presence has been proved to be necessary
for both, interface design of social applications (Xu
et al., 2006), and learning purposes (Kasvi, 2000). In
addition, we claim that social presence is also im-
portant in serious games. We also state that we can
identify several ’indicators’ in order to measure the
potential presence awareness of our application. In
other words: the absence of these indicators in our
virtual environment (in our case a serious game) is a
signal of a not well designed system, not able to sup-
port social interactions and social learning. As we
have said, being able to do such an evaluation at early
stage (i.e., before starting implementation) has ma-
jor advantages: it can simplify the work of designers
(that can return on their design before development)
and help to build better applications while reducing
development costs.
3 DEFINING ’INDICATORS’ FOR
SOCIAL PRESENCE
This section presents core elements of the framework
that can be used to evaluate social presence ’poten-
tial’ in an application at early stage. The framework
is based on four elements: identity, space, persistence
and actions. These elements are motivated by an em-
pirical analysis of current social software and sup-
ported by major findings from psychology and sociol-
ogy. In fact, these elements represent core features of
any Social Interactive Systems (SIS) targeted towards
young generations (and thus also serious games with
social capabilities). Consequently, they represent an
interesting evaluation criteria in order to capture at
early stage the potential presence awareness of the ap-
plication being designed. Hereafter, the semantics of
each element of the framework is described more in
details.
Identity. Our point of view about Identity is the
same as social psychology’s later approaches, which
consider individual and social identity not as stable
characteristics, but rather as a dynamic phenomenon
(Harré and Langenhove, 1991). In these approaches,
the choice about what possible self to show is driven
by strategic moves (e.g., what features are more rel-
evant and effective for self presentation) which par-
ticipants can make within a particular situation. In
describing everyday interactions, Goffman (Goffman,
1959) distinguished between two ways of expressing
information: information that is given and informa-
tion that is given off. Information that is given is the
conscious content of communication, the voluntary,
symbolic actions that are mutually understood, for ex-
ample, a person who describes their anger is giving
information about their emotional state. In talking
about their anger, however, the person also gives off
information, through para-verbal characteristics such
as tone, volume, the choice of words, and non-verbal
cues. While information that is given is considered
to be within the actor’s control, information that is
given off is perceived by the audience to be uninten-
tionally communicated. A classical example of ’iden-
tity announcement’that has intentionally and uninten-
tionally elements is avatar personalization. While we
will not enter in detail here on its implications, the
avatar is a visual claim for personal expression that
is constantly worked on. This continuous work rein-
forces the concept of presence and thus social pres-
ence. The explicit specification of a social network of
acquaintance can be seen as collateral information. If
it’s true that social networks are built via a series of
invitations, usually members also have some control
over the visibility of their network for others. This
means that, for impression management, a user will
show only networks he/she wants to show. For in-
stance some members can decide to make their social
networks visible only to their direct acquaintances.
In this case, there is a ’given’ information (the user
chooses what to show about his/her identity), but also
a ’given off’ information (derived e.g., from the kind
of groups a user showed/joined). From a design point
of view we can say that, allowing both the kinds of
identity representation becomes the starting point for
a social, evolving identity.
Space. If we look carefully, the language we use to
describe our experience of the virtual environment is a
reflection of an underlying conceptual metaphor: ’Cy-
berspace as Place’ (Lakoff and Turner, 1988). This
means that we are transferring certain spatial char-
acteristics from our real world experience over the
virtual environment. The metaphor ’Cyberspace as
Place’ leads to a series of other metaphorical infer-
ences: cyberspace is like the physical world, it can
be ’zoned’, trespassed upon, interfered with, and di-
vided up into a series of small landholdings that are
just like real world property holdings. As you no-
ticed, we joined together the terms space and place.
In reality, for the good functioning of a SIS it is im-
portant to distinguish between them. Actually, the lit-
erature about space and place is fairly massive and
diverse. A converging definition of the difference be-
tween space and place does not exist, however we can
list some interesting definitions adapted from (Car-
mona et al., 2003) that try to capture differentiation
aspects between space and place: Space is alienation;
CSEDU 2010 - 2nd International Conference on Computer Supported Education
96
place is identification (according to Norberg-Schulz),
spaces are the basic divisions of our surroundings;
place is our history and adaptation of them (landscape
historian JB Jackson), space is the scene of being;
place is a site where human modes of being are well
provided for (Heidegger), places are essentially cen-
ters of meaning constructed out of living experience.
By imbuing them with meaning, individuals, groups
or societies change spaces’ into ’places’ (urban de-
signer Edward Relph). Besides, in his book (Car-
mona et al., 2003), about urban spaces and places,
Carmona distinguishes among dimensions of an ur-
ban space. While space is divisible, place is not. Place
is complex, inextricably multi-dimensional, lived, ex-
perienced, meaningful (with of course multi - mean-
ings). This means that while space is a well-defined
topographical entity, place is the result of human in-
habitation, (social) interaction, and the like. We are
located in spaces, but we act and develop individual
and social experiences in places. We claim that in or-
der to design a social serious game, it is essential to
allow by design the creation of public (at different lev-
els) places for aggregation but also the creation of pri-
vate places (Wenger et al., 2002). Besides, the level
of personalization can be used in order to allow the
shift from spaces to places. Only taking possession of
the space, and manipulating it to turn it in something
we like, we can transform it in a place.
Persistence. As we have seen, in order to create a
social identity in an online environment several ele-
ments are required. The first of these is persistence (of
personal identity in the system). In a non-persistent
world it is not possible to have an history of actions
and thus allow, for example, the creation of a reputa-
tion like in real life. Moreover, Danet (Danet et al.,
1997) , argued that synchronicity is associated with
’flow experiences’, a state of total absorption and a
lack of awareness of time passing. This idea of syn-
chronicity is linked to the idea of temporality, a linear
procession of past, present, future. This particular nu-
ance (synchronicity as process) is very interesting if
we think that interaction with media and media per-
ception is changed. In fact, advances in technology
and the speed of network connections are blurring
distinctions between synchronous and asynchronous
communications (Joinson, 2003). Synchronous and
asynchronous communications are thus processes that
happens during time. The idea of communication as
a process is totally consistent with the idea of persis-
tence and is another element supporting social aware-
ness.
Action. In this part, we discuss physical and psy-
chological mechanisms that regulate human actions
in order to understand why the action element have
to be considered as a pillar in the design of social se-
rious game. The first theory we want to describe is
the so-called: thinking through doing. This theory
describes how thought (mind) and action (body) are
deeply integrated and how they co-produce learning
and reasoning (Klemmer and Hartmann, 2006). Jean
Piaget (Piaget, 1952) postulated that cognitive struc-
turing requires both physical and mental activity. In a
very basic sense, humans learn about the world and its
properties by interacting within it. As a second sup-
port we can cite embodied cognition. Theories and
research of embodied cognition regard bodily activ-
ity as being essential to understanding human cog-
nition (Pecher and Zwaan, 2005). While these the-
ories address cognition through action in physical en-
vironments, they also have important implications for
designing interactive systems. In fact, body engage-
ment with virtual environments constitutes an impor-
tant aspect of cognitive work. Because an action is
always an action-over-something, the kind of interac-
tion spaces and objects we create in a Social System
will influence which cognitive work the user will do
over the system.
3.1 The Overall Framework
We can think about each of the above described el-
ement as a line (an axis) that starts from the lack of
presence of the element to the fulfillment of its pres-
ence for a social serious game. For example, for
the concept of identity when totally missing repre-
sents anonymity while when fulfilled represents a so-
cial presence (with intermediate points such as per-
sonal identity construction). For the concept of space
when missing represents a topographical space while
when fulfilled represents social places (with interme-
diate points such as third places and personal places).
For the concept of time its totally lack is system ’am-
nesia’ while its fulfillment is memory (with interme-
diate points linked more or less to the concept of per-
sistence). Finally, for the concept of action its totally
lack is the obstruction of action (i.e., my user can only
look at my application) while its fulfillment is social
actions (with intermediate points such as public per-
sonal actions and the like). However, the total frame-
work is not simply a list of elements (i.e., its applica-
tion doesn’t mean to put one after the other the four
elements in your system) but it’s created through the
delicate balancing between them. This means that it
is up to the designer to choose which element of the
framework stress or not during the creation of a dy-
AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME
97
namic experience such as a serious game with social
integrated features.
4 A METHOD FOR EVALUATING
AN EXISTING GAME DESIGN
While the above described framework can also be
used in the phase of design of a social serious games,
for this paper we will focus on its application to the
phase of evaluation of the designed application. In
fact, usually a software development starts with a re-
quirement document that captures a set of require-
ments and features of the future application. This
initial requirement document is written by stakehold-
ers using natural language to express their needs. So,
during the design phase always arose the problem of
translating natural language requirements into soft-
ware requirements. In order to overcome this dif-
ficulty, we suggest a methodology, already used in
game developments, based on the concept of stripes.
Flynt defines a stripe as ’a set of functionality em-
bodied in a single component of the system.’(Flynt,
2004). More specifically, a stripe embodies a subset
of functionalities described in the requirements doc-
ument as a coherent succession of actions. The pro-
cess producing stripes is a top-down process starting
from the most general system features. For instance,
one can find the login stripe that describes a set of ac-
tions allowing the user to open a session. The level
of detail and complexity grows with each stripe, but
because the detail and complexity are layered, at no
point does complexity become overwhelming thanks
to the hierarchical organization of stripes. This hi-
erarchical organization of stripes does not mean that
all stripes are independent. In fact, stripes can over-
lap when they concern similar functionalities. For ex-
ample, both login and personalization stripes can in-
volve a stripe that loads the user’s avatar and profile.
However, extracting stripes from natural language de-
scription is quite difficult due to inherent ambiguity
and complexity of natural language. To overcome this
difficulty we inserted a phase of pre-design between
concept development and classical design phase. The
idea behind this insertion is to use the phase to help
the designer to translate the design documents into re-
quirements.
As first step the ’translator’ makes a very sim-
ple task: he takes the design and he highlights ac-
tions (i.e., verbs) in natural language. He next de-
fines for each action the elements that impact over
the framework filling a structured table. For exam-
ple he uses a field named ’Interaction space’ (that an-
swer the question ’the action impact over?’) to iden-
Figure 1: An example of evaluated stripe.
tify space/place, and answers to the ’who’ and ’when’
questions (’who’ is doing this action? Is a persistent
action? and the like) to identify identity and persis-
tence. The application of this method is linear (in fact
you analyze the document paragraph after paragraph).
The result of this first analysis is a set of very detailed
stripes that still have to be worked on in order to de-
rive features and components. However, this phase is
also the most interesting for early evaluation. In fact,
as we have said in classical design cycles, the eval-
uation of the system (in order to decide if or not it
is coherent with your expectation) can be done only
after the development phase (in reality, after the use
of the application). In our case we can evaluate the
system before starting the implementation.
In order to make such an evaluation we provide
weights, ranging from 0 to 1, for each stripe (i.e, we
analyze the above mentioned table for each stripe).
Next we ’fill’ another table like in Fig. 1 with weights.
In our example, the modification of the avatar impacts
over the identity but is also a persistent action. On the
contrary, it does not impact on the space of the appli-
cation. This kind of evaluation has two main advan-
tages. Firstly, it will show you if the resulting evalu-
ation is exactly what you expect from the application
(the profile you have wished for) before you start the
implementation. It may happen, for example, that you
want to create a design that is balanced over all the
four elements but the result of your evaluation shows
that you lack e.g., in identity features. Secondly, it
allows you to re-define your stripes in this phase in
order to avoid the problems highlighted by the evalua-
tion (or, on the contrary, to re-think your first design).
In next section we will describe an experiment
we held during the creation of a social serious game
where we asked our designers to apply the above de-
scribed method and to evaluate it.
5 AN EXPERIMENTATION OF
THE METHOD: SCHOOL
SOCIETY
5.1 Method
The objective of the study reported in this section
is twofold: (i) to check practicality of the proposed
framework when building an actual complex social
CSEDU 2010 - 2nd International Conference on Computer Supported Education
98
serious game; (ii) and to evaluate qualitatively the
benefits of such methodology. A textual description
of a social serious game named School Society has
been prepared by a team of researchers composed
mainly by pedagogues (2 persons) and game design-
ers (2 persons). Computer scientists have been al-
lowed to participate but were not authorized to ad-
dress technical issues. These meetings have produced
a game design document that describes, to some ex-
tent, the “culture” (Salen and Zimmerman, 2003) and
rules of the social serious game.
To summarize, the game can be described as fol-
low: (i) every student has an avatar within the system
and lives in a house situated in a private island,(ii) the
student can use a canoe to reach one of the follow-
ing destinations:school, market place, pub, friend’s is-
land,(iii) the student takes quizzes in the school in or-
der to have a certain amount of virtual money and ex-
perience,(iv) the student buy items to decorate his/her
house in the market place and these items are visible
to friends visiting the house,(v) finally, the pub is used
to chat with other players and make new friends It is
worth noting that this is a simple description of the
game and the actual game design document is about
20 pages (4000 words and 15 figures)
A computer science engineer, with an experience
of 3 years, has been specifically hired to produce the
serious game specifications starting from the game
design document. The methodology was introduced
to the engineer during a half-day training session con-
ducted by the authors. The engineer was not allowed
to question the methodology. However, he was al-
lowed to ask questions on the game design docu-
ment by explicitly raising issues using a standard is-
sue tracking system
2
. Authors of the game design
document were able to answer explicitly these issues
by providing more explanations or making choices to
clear ambiguities. The identities of the engineer and
game design authors were not revealed to prevent in-
direct interactions and all interactions have occurred
explicitly through the issue tracking system. The pro-
duction of the final serious game specification was
performed in two phases.
Phase 1: the engineer produced all stripes and de-
rived automatically the application profile. The
result of this phase was then transmitted to game
design authors that provide feedbacks in order to
correct what is missing and guide the next phase.
At this stage, they were allowed to make changes
to the original game design document. Fig.2
shows the first evaluated profile for the applica-
tion.
2
The issue management system Redmine has been used.
Figure 2: The first evaluated profile.
Figure 3: The final evaluated profile.
Phase 2: the engineer takes into account phase 1
feedbacks and makes revisions to produce a new
set of stripes and a new application profile. Again
these results were presented to game design au-
thors. Fig.3 shows the final evaluated profile for
the application. As you can see the second profile
is more balanced over the four axis.
It is worth to note that this balancing among all di-
mensions was decided by game designers for this par-
ticular application. Within other contexts, designers
may decide to emphasis on particular dimensions and
deliberately neglect some others. For instance, in
learning casual games, the persistence dimension can
be deliberately diminished.
The study was limited to two phases but one can
notice that this is an iterative process that can be iter-
ated more than twice.
Table 1: Duration and number of stripes produced in each
phase.
Phases Time N. of
Stripes
N. of mod.
Stripes
Phase 1 3 weeks 75 n-a
Phase 2 1 week 81 18
AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME
99
Table 1 presents duration of each phase. One can
notice that phase 1 is three times longer that phase
2. This is due mainly to the learning curve of the
methodology and also by the fact that most of stripes
have been produced during phase 1 and only few have
been modified and added from phase 1 and 2.
5.2 Results
The final evaluation was conducted using interviews
with the engineer and game design authors.
5.2.1 Engineer’s Feedback
At the end of the study an interview was conducted
with the engineer. During the interview the engineer
has addressed several points about his experience with
the presented methodology. Hereafter a summary of
the main points that have been discussed:
The engineer has found the methodology very
useful in identifying non-functional aspects of the
serious game. In fact, the culture of classical
software engineering does not highlight the im-
portance of non-functional aspects of a software
system that creates a specific atmosphere for an
application. In serious game non-functional as-
pects are crucial to capture game culture and at-
mosphere. Thanks to the methodology, all non-
functional aspects have been recorded as non-
functional stripes.
The engineer has already some experience in de-
veloping large software systems. So, he was
aware of the cost of implementation and revision
of software systems. Consequently, he pointed
out that the early evaluation performed between
phase 1 and phase 2 has helped to revise some
fundamental decisions without having to conduct
costly implementations.
The engineer has also pointed out that having ex-
tracted all stripes and grouping them into clusters
makes it very easy to build a detailed storyboard
of the serious game and facilitates the implemen-
tation phase. In fact, during the implementation
all stripes are translated to features that are imple-
mented by developers. This decomposition can
also help to adopt an iterative approach by decid-
ing what features to implement for each release.
5.2.2 Game Designers’ Feedback
Surprisingly, the application profile has been consid-
ered by game designers as an interesting communi-
cation medium to communicate with the engineer. In
fact, they were given a feedback that represents, to
some extent, the interpretation of the engineer. For
instance after phase 1, game designers have noticed
that several points concerning social interactions have
been missed by the engineer. In fact, phase 1 game
design mentioned, but succinctly, chat between play-
ers. Since this description was not that significant the
engineer did not create a specific stripe. When ana-
lyzing the early evaluation of phase 1, game designers
discovered this omission and have decided to add into
the game design a description of a pub to allow play-
ers socializing. This has generated specific stripes in
phase 2.
6 CONCLUSIONS AND FUTURE
WORKS
In this paper we started from the assumption that so-
cial learning and social features enabling social pres-
ence are essential when building a serious game. For
this reason we described a framework and an early
evaluation method that can be useful in order to an-
ticipate ’lacks’ in the design. The framework and the
early evaluation approach can be useful at different
levels. Firstly, the early evaluation of the game design
is performed when multiple experts (experts on learn-
ing, computer scientists and the like) have to work
together by fostering communication between them.
Secondly, an early evaluation of the designed appli-
cation can help designers to anticipate several prob-
lems that can arise when developing a serious game
with social aspects before starting the implementation
(thus returning on the game design phase to add miss-
ing elements and restraining development cost). In or-
der to support these two last statements we described
an experiment we held where a computer scientists
and pedagogues worked together in order to build a
virtual world for learning purposes. Results of this ex-
periment were in line with the idea that adding a layer
between user centered design, and user satisfaction
measurement can really help designers to build seri-
ous games in a better way. Moreover, it’s our inten-
tion to reproduce the experimentation of the method
over different serious games design, in order to bet-
ter support our findings. Besides, we also believe that
trough users’ feed-backs we could show empirical ev-
idence that the method and the framework can help to
build better serious games from the social point of
view. For this reason we are conducting a real world
experiment to relate designer’s impressions with the a
posteriori evaluation of end users.
Results of this experiment will be subject of future
publications to enrich serious game literature with ex-
perimental works to demonstrate advantages and lim-
CSEDU 2010 - 2nd International Conference on Computer Supported Education
100
itations of this approach.
REFERENCES
American’s Army (2002). The american’s army game web-
site. http://www.americasarmy.com.
Biocca, F. (1997). The cyborg’s dilemma: Progressive
embodiment in virtual environments [1]. Journal of
Computer-Mediated Communication, 3(2).
Burke, J., McNeill, M., Charles, D., Morrow, P., Crosbie, J.,
and McDonough, S. (2009). Optimising engagement
for stroke rehabilitation using serious games. The Vi-
sual Computer.
Carmona, M., Heath, T., and Tiesdell, S. (2003). Public
Places, Urban Spaces: The Dimensions of Urban De-
sign. Architectural Press, Burlington, MA.
Danet, B., Ruedenberg-Wright, L., and Rosenbaum-Tamari,
Y. (1997). Hmmm... where’s that smoke coming
from? writing, play and performance on internet relay
chat. In Journal of Computer-Mediated Communica-
tion, volume 2.
Flynt, J. P. (2004). Software Engineering for Game Devel-
opers. Software Engineering Series. Thomson.
Goffman, E. (1959). The presentation of self in everyday
life. Doubleday.
Gudmundsen, J. (2006). Movement aims to get serious
about games. USA Today, 5/19/2006.
Harré, R. and Langenhove, L. V. (1991). Varieties of posi-
tioning. Journal for the Theory of Social Behaviour,
21(4):393–407.
Joinson, A. N. (2003). Understanding the Psychology of
Internet Behaviour: Virtual Worlds, Real Lives. Pal-
grave Macmillan.
Kasvi, J. (2000). Not just fun and games – internet games as
a training medium. In Cosiga learning with comput-
erised simulation games, pages 23–34. P Kymäläinen
and L Seppänen.
Kiili, K. (2005). Educational game design: Experiential
gaming model revised. Technical Report Research re-
port 4, Tampere University of Technology.
Klemmer, S. R. and Hartmann, B. (2006). How bodies mat-
ter: Five themes for interaction design. In In Pro-
ceedings of Design of Interactive Systems, volume 74,
pages 140–149.
Lakoff, G. and Turner, M. (1988). Categories and Analo-
gies, volume 3. University Of Chicago Press.
Nicola, W. (2007). Motivation and computer game based
learning. Proceedings ascilite Singapore 2007.
Pecher, D. and Zwaan, R. A. (2005). Grounding Cognition:
The Role of Perception and Action in Memory, Lan-
guage, and Thinking. Cambridge University Press.
Piaget, J. (1952). The origins of intelligence in children.
International University Press.
Salen, K. and Zimmerman, E. (2003). Rules of Play: Game
Design Fundamentals. The MIT Press.
Serious Games Initiative (2002). The
serious games initiative websites.
http://www.seriousgames.org/index2.html.
Short, J., Williams, E., and Christie, B. (1976). The Social
Psychology of Telecommunications. John Wiley and
Sons Ltd.
Tu, C. H. (2000). On-line learning migration: from social
learning theory to social presence theory in a cmc en-
vironment.
Vredenburg, K., Isensee, S., and Righi, C. (2001). User-
Centered Design: An Integrated Approach. Prentice
Hall PTR.
Wenger, E. (2000). Communities of practice and social
learning systems. Organization, 7(2):225–246.
Wenger, E., Mcdermott, R., and Snyder, W. M. (2002). Cul-
tivating Communities of Practice. Harvard Business
School Press, Boston.
Witmer, B. G. and Singer, M. J. (1998). Measuring pres-
ence in virtual environments: A presence question-
naire. Presence, 7:225–240.
Xu, Z., John, D., and Boucouvalas, A. C. (2006). Social
factors and interface design guidelines. In Ghaoui, C.,
editor, Encyclopedia of Human Computer Interaction,
pages 523–532. Idea Group, London.
AN EARLY EVALUATION METHOD FOR SOCIAL PRESENCE IN SERIOUS GAME
101