Finite State Machine component. The game man-
ager is responsible for managing all the game me-
chanics, interfaces and player interactions. Finally,
the Unity3D kernel is responsible for managing the
graphical pipeline, graphic asset and scene graph.
4 EVALUATION
One important differentiating factor between serious
games and entertainment games is the importance
of assessment. Measuring, discussing and reasoning
about the game play effectiveness is very important in
this domain. This can lead to reflection and there-
fore improved learning (Lopes and Bidarra, 2011).
Chen and Michael have identified some of the main
challenges that assessment in serious games is facing,
namely effecting and improving player experience.
The authors suggest that log information and teach-
ers/instructors knowledge should be fully explored
and, in some way, incorporated back in the game, to
guide its course (Chen and Michael, 2005).
Pamela Kato (Kato, 2012) has also argued that the
few research studies published on the validity and ef-
ficacy of health games are often poorly designed and
that their conclusions cannot be considered valid evi-
dence to support or refute efficacy. In this respect, she
has suggested a set of guidelines for conducting high
quality efficacy studies on games for health. These
guidelines include grounding the game design in well
defined theories, conducting randomized trials that in-
cluded adequate control groups and number of partic-
ipants, and also incorporating standardized measures
to facilitate comparisons across studies.
The authors agree with the recommendations pro-
vided by the previous cited authors and they have
been considered both during the development of the
game as well as when designing and planning the
evaluation of game play effectiveness. During game
design, both experts and lay people were involved in
defined check-points to validate that both the peda-
gogical goals as well as the user interactions were ac-
cording to the expectations of the end-users. Also,
the inclusion of pedagogical content in the game
had into account previous studies of similar games
and approaches such as design patterns (Bj
¨
ork and
Holopainen, 2005) and finite state machine diagrams.
It was then important to evaluate if the game actu-
ally taught what it had been designed to. This was the
hypothesis of our study. In order to confirm or dis-
card this hypothesis, we designed a qualitative study
based on pre-experimental design research method
(Creswell, 2008), which includes three specific mo-
ments: briefing, where the player is introduced to the
pedagogical goals of the game; a pre-test, where the
user responds to the questionnaire; game experience,
where the user plays the different game modes (train-
ing mode and evaluation mode); and, a pos-test where
the player responds to the questionnaire. The pre-test
and pos-test questionnaire are one and the same, and
consist of a set of questions related to the ERC’s pro-
tocol and some demographic information related to
the player. In the reminder of this section, we detail
how the study was conducted and the achieved results.
4.1 Pre-test and Pos-test
Before each game experience each player was asked
to answer the questionnaire in order to have feed-
back about their knowledge of basic life support. The
main goal of the pre-test was to evaluate the previous
knowledge the user had about the different procedures
provided in ERC’s protocol.
The questionnaire was composed by ten ques-
tions. The first three questions concern demographic
information about the player. According to this infor-
mation, the participants are between 20-30 years; 23
are male and 8 are female, in a total of 31 participants,
without any specific previous knowledge about Basic
Life Support.
The remaining questions, were related to the ERC’s
protocol and they were validated by healthcare pro-
fessionals.
After playing, each player was asked to answer
the questionnaire again in order to evaluate the knowl-
edge the player had acquired during the game session.
The pos-test was exactly like the pre-test except for
the first three questions concerning demographic in-
formation. The main goal of this pos-test was to eval-
uate if the game teaches the sequence of procedures
described in the ERC’s protocol. In other words, vali-
dated the evolution of the user’s knowledge after play-
ing the serious game SeGTE.
After all the participants were evaluated, the data
collected were analysed and the results of the pre and
pos-test were compared.
5 DISCUSSION
Comparing the results of the pre-test with the pos-test,
we obtain better results in all of the questions in the
pos-test. This positive trend may show that the seri-
ous game has some impact in the user’s knowledge,
as presented in Figure 3.
These results are very positive which give us a certain
confidence that the game fulfils its educational goals.
Furthermore, the participants in general demonstrated
GRAPP2014-InternationalConferenceonComputerGraphicsTheoryandApplications
442