Evaluating the Emotional Effects of Role-playing Software on
Interactive Digital Storytelling from the Perspectives of User,
Storyteller, Teammate and Audience
Gloria Yi-Ming Kao and Xing-Yi Huang
Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology,
Keelung Rd., Sec. 4, Da’an Dist., Taipei City 10607, Taiwan (R.O.C.)
Keywords: Interactive Digital Storytelling, Emotions, Role-playing, Authoring Tools, User Experience.
Abstract: This paper discusses the applications of emotions in interactive digital storytelling. Emotion is one of the most
important elements in digital storytelling. Participants were asked to design a story collaboratively and present
it to the audience using the role-playing software, FaceRig. They were asked to fill out a questionnaire about
their attitude toward this activity from the perspective of an audience member, a storyteller, a group member,
and a software user. The findings indicated that most participants held a positive attitude toward this activity
whether as an audience member or as a storyteller. Most participants thought that listening to and telling
stories with FaceRig was a pleasant and appealing experience. They loved the idea of performing the facial
expressions of the characters during both the preparation process and the performing process. We concluded
that the application of emotions in storytelling is a way to improve the storytelling experience of both audience
and storytellers. To further enhance user experience, we suggest that designers of such role-playing software
could implement features such as voice changing functions and body movement detection.
1 INTRODUCTION
Studies of storytelling have suggested that
storytelling has had a great influence on human
culture, on education, and even on the medical field
(Barrett, 2006; Chandrasekar et al., 2018; Schmoelz,
2018; Psomos and Kordaki, 2012; Tsou et al., 2006).
In the field of education, many studies have proposed
that storytelling is an effective learning and teaching
tool, and that it increases students’ learning
engagement, learning outcomes, and their creativity
(Barrett, 2006; Mokhtar et al., 2011; Schmoelz, 2018;
Sandars and Murray, 2011; Tsou et al., 2006). Digital
storytelling, or storytelling with the help of
technology, has become a common field in the realm
of storytelling. The difference between traditional and
digital storytelling is that digital storytelling is
performed through different kinds of media, such as
computer-produced images, typed text, sound effects,
music, and video (Miller, 2009; Robin, 2008). Digital
storytelling could not only provide positive results in
learning, but the media of digital storytelling could
also catch students’ attention and increase their
motivation (Robin, 2008). Furthermore, Dörner,
Grimm, and Abawi (2002) argued that digital
storytelling allows learners to actively take part in the
creation of the story. Participation in the process of
creating the story facilitates some important skills,
such as communication skills, ability to evaluate
one’s own work, social learning, and emotional
intelligence (Robin, 2008).
Emotions are one of the essential elements of
digital storytelling (Center for Digital Storytelling,
2005). Emotional content can provide the audience
with connections to the story content, and can make
the audience feel that the discussed issues are more
related to themselves (Center for Digital Storytelling,
2005). Freedberg and Gallese (2007) also proposed
that when observing people’s emotional expressions
or pictures arousing strong emotions, observers tend
to have emotional circuits in the similar facial
muscles or have the same emotions. Moreover, Costa,
Brunete, Bae, and Mavridis (2018) suggested that the
emotions expressed by the storytellers would impact
on the emotional responses of the audience. The
emotions can even make the audience stand in the
characters’ shoes (Costa et al., 2018). Cavazza, Pizzi,
Charles, Vogt, and André (2009) also agreed that
300
Kao, G. and Huang, X.
Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller, Teammate and Audience.
DOI: 10.5220/0007709503000306
In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 300-306
ISBN: 978-989-758-367-4
Copyright
c
2019 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
emotions could increase the audience’s sense of
reality. With the impact of emotions, a story could be
more than just a simple story. The emotional elements
of traditional storytelling were presented by the
variations in intonation. However, human beings
receive emotional cues not only from vocal messages
but also from visual information, such as facial
expressions and body language.
In this study, we adopted the role-playing
software, FaceRig, which allow users to role-play
different 3D characters by performing facial
expressions. The integration of role-playing in
storytelling helps storytellers depict the plot in detail
by presenting the characters’ images and emotions.
Story listeners could engage in the stories more
deeply because they could refer to the characters’
images and characters’ facial expressions through the
role-playing software we adopted.
In this study, we aimed to know whether emotions
could arouse the interest, increase the motivations,
and improve the state of engagement of audience and
storytellers in storytelling through the software,
FaceRig.
2 RESEARCH TOOLS
2.1 FaceRig Role-playing Software
This research adopted FaceRig (Version build 1450),
a type of role-playing software, as the instrument of
storytelling. FaceRig is an open creation platform for
users to roleplay different characters with its real-time
facial animation feature. It provides users with
several characters, backgrounds and props to choose
from. The emotions of the characters will change
along with the facial expressions of the user, which
are captured by the computer camera. Moreover,
users can customize their own characters using
FaceRig, giving them more possibilities in role-
playing, and they can roleplay their own characters
while watching one another’s performance through
either one of the following two functions of FaceRig.
First, FaceRig can be linked to online chat rooms, for
example, Skype and Hangouts, so the players can
watch the others’ characters and their performance in
the chat room. Second, the real-time facial animation
function of FaceRig can detect more than one
person’s facial expressions and apply the emotions of
different users to different characters. That is,
FaceRig allows more than one person to use it at the
same time with the same computer and interact with
each other with different characters.
FaceRig used in this study provides 52 different
characters, including humans and animals. It has 32
scenes. For instance, users can role-play as animals
such as dogs, wolves, and dragons, or impersonate
other human beings of different genders or ages (see
Figure 1). They can also choose the role-playing
background as streets, forests, a hall, and so on (see
Figure 2).
Figure 1: Characters provided by FaceRig.
Figure 2: Scenes provided by FaceRig.
2.2 The Questionnaire of User
Experience Evaluation
The questionnaire used in this study was derived from
the IRIS evaluation toolkit (Klimmt et al., 2010). We
designed the questionnaire based on four
perspectives: audience, storytellers, interactions with
teammates, and use of FaceRig. Audience refers to
participants that watch the storytelling performance
of other participants. Storytellers are the ones who
perform the storytelling through FaceRig.
Interactions with teammates include the interactions
during their preparation and performance. Use of
FaceRig refers to the usability of the system. We
adopted some factors from IRIS, including usability,
effectiveness, satisfaction, and user experiences as a
reference. De Lima, Feijó, Barbosa, Furtado, Ciarlini,
and Pozzer (2014) also adopted these four factors in
their questionnaire. The questionnaire includes 22 5-
point Likert scale items, with 1 standing for strongly
Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller,
Teammate and Audience
301
disagree to 5 for strongly agree, for the quantitative
analysis, and four open-ended questions for the
qualitative analysis. Four open-ended questions were
designed to investigate the “use of FaceRig”
perspective.
3 METHODOLOGY
3.1 Participants
The participants were 13 graduate students from a
graduate school in Taipei, Taiwan. There were five
males and eight females. All the participants were
taking the course, Creative Multimedia Design. The
FaceRig storytelling activity is a part of the course.
Every participant knew that the results of the “The
user experience of telling story with FaceRig”
questionnaire would not influence their grade.
3.2 Procedure
The researchers held a storytelling activity with
FaceRig in order to learn the differences in
storytellers’ and audience’s reactions to role-playing
storytelling and conventional storytelling. FaceRig is
role-playing software, offering users various 3D
characters for role playing. It detects storytellers’
facial expressions from the camera and projects them
onto the 3D characters.
The procedure is as shown in Figure 3. We
introduced FaceRig’s functions and how to operate it
to all participants one month before the official
experiment. Also, pictures of available scenes and
characters were offered, with the agenda of the
activity.
Before the storytelling activity, participants
worked with their group members to come up with
the outline of the story with a concept map, story face
(Staal,
2000).
There
was
no
limitation
to
the
content
Figure 3: Procedure.
CSEDU 2019 - 11th International Conference on Computer Supported Education
302
Figure 4: The story face structure (Staal, 2000) and story face of The Frog and the Princess.
Figure 5: Photo of FaceRig storytelling.
of the story. Original and adapted stories were both
accepted. After completing the concept map,
participants needed to design the plots and dialogues
(2 characters). For example, one story topic is The
Frog and the Princess. The story face of their story is
shown in Figure 4. Figure 5 shows a photo of FaceRig
storytelling.
All participants were both the storytellers and
listeners. During the formal activity, participants
worked in groups of two or three (2 groups of 2
students and 3 groups of 3 students) to tell stories with
FaceRig in the same room. When they were not in
Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller,
Teammate and Audience
303
charge of presenting their stories, they listened to
others’ stories and observed the expressions and
interactions between the 3D characters. Two
computers were used for the FaceRig role-playing.
The role-playing performance from the two
computers was linked using the online meeting
software, Hangouts. A third computer was also linked
to the two computers for the projection. Thus, the
audience could easily observe the performance from
the big screen. The storytelling process was recorded
using the inbuilt recording function of both FaceRig
and Hangouts. After each group finished their
storytelling, the participants gave comments to each
group. Later, the participants were asked to complete
the questionnaire.
4 RESULTS
The overall results showed that participants presented
a positive attitude toward the different factors of the
role-playing storytelling activity with FaceRig from
four perspectives: audience, storyteller, team
member, and software user. The results are presented
based on these four perspectives.
4.1 Watching the FaceRig Storytelling
as an Audience Member Procedure
As audience members, most of the participants found
watching storytelling with FaceRig a good
experience. More than 90% of the audience thought
that listening to storytelling with FaceRig was
interesting. Besides, all the participants enjoyed the
idea of adding emotions into storytelling. Moreover,
85% of the participants found storytelling with
FaceRig more appealing than conventional
storytelling. For effectiveness and satisfaction, 85%
found that storytelling with FaceRig assisted them in
understanding the emotions of the characters and
connected them to the plot as if they were really
experiencing the story. See Table 1 for the items and
detailed results of the audience perspective of the
questionnaire.
4.2 Telling Stories with FaceRig as a
Storyteller
The storytellers’ attitudes toward FaceRig
storytelling were mostly positive. Most of them found
it a good experience. Every participant agreed that
telling stories with FaceRig and expressing the
emotions
at the same time was interesting. Every
Table 1: The results of questionnaire part 1: As an audience
member.
ID
Items
SA A N D SD
1
I think it's fun to listen to the stories performed by
FaceRig.
30.8% 61.5% 7.7% 0% 0%
2
I like to see performers add emotional expressions
when they tell stories.
76.9% 23.1% 0% 0% 0%
3
Listen to stories told with FaceRig will give me a
better understanding of the emotions of the
characters.
23.1% 61.5% 15.4% 0% 0%
4
FaceRig plus emotional expressions will help me
experience better the stories.
15.4% 69.2% 15.4% 0% 0%
5
Storytelling with FaceRig is more appealing to me
than the storytelling in a normal way.
38.5% 46.2% 15.4% 0% 0%
SA stands for strongly agree, A for agree, N for neutral, D
for disagree, and SD strongly disagree.
Table 2: The results of questionnaire part 2: As a storyteller.
ID
Items
SA A N D SD
1
I think it's fun that I can express emotions during
storytelling by using FaceRig.
38.5% 61.5% 0% 0% 0%
2
I think using FaceRig will help me tell better
stories.
7.7% 76.9% 15.4% 0% 0%
3
I think using FaceRig will help me express
completely well the emotions that I want to show.
7.7% 30.8% 46.2% 15.4% 0%
4
I can perform more vividly the personalities of the
characters when I tell stories with FaceRig.
23.1% 46.2% 30.8% 0% 0%
5
In my experience, I think telling stories with
FaceRig can engage me more than the way I used
to do storytelling.
30.8% 59.2% 0% 0% 0%
6
When I tell stories with FaceRig, I am very
concerned about the reaction of the audience.
30.8% 53.8% 7.7% 7.7% 0%
7
When I tell stories with FaceRig, I will modify my
interpretation of the character instantly or
improvise according to the audience's response.
30.8% 38.5% 15.4% 15.4% 0%
8
I am looking forward to telling stories with
FaceRig in the future.
7.7% 61.5% 30.8% 0% 0%
SA stands for strongly agree, A for agree, N for neutral, D
for disagree, and SD strongly disagree
participant also agreed that storytelling with FaceRig
helped them engage in the storytelling process. In all,
85% reported that they cared about the audience’s
CSEDU 2019 - 11th International Conference on Computer Supported Education
304
reactions and responses, 70% thought that FaceRig
allowed them to present the emotions and the
characteristics they expected. Furthermore, more than
90% are looking forward to the next chance to tell
stories with FaceRig. For effectiveness, more than
90% thought that FaceRig helped them tell a better
story. In addition, 77% thought that FaceRig could
thoroughly present the emotions they expected, while
70% reported that they would adjust their
performance according to the reaction of the
audience. The items and results of storyteller
perspective of the questionnaire are shown in Table
2.
4.3 Interacting with Teammates
through FaceRig
The overall opinion on the interaction with team
members in the research activity was positive. More
than 90% of participants engaged in the preparation
process of plot and character design after knowing
that they would use FaceRig as a storytelling
instrument. More than 90% agreed that they would
observe the facial expressions of their teammates
while telling stories with FaceRig. Around 70%
showed that the emotions expressed by their
teammates would influence their facial expressions,
77% declared that they would come up with some
new ideas and improvise because of their teammates’
emotional expressions, and 70% enjoyed the FaceRig
storytelling with their teammates. See Table 3 for
each item and result of the interaction perspective of
the questionnaire.
4.4 Use of FaceRig
The findings of this section are divided into two main
parts: the quantitative and the qualitative results. For
the quantitative analysis, most of the participants
found FaceRig easy, fun, and effective to use. About
80% strongly agreed or agreed that FaceRig is easy to
use. Most participants agreed that the process of
telling stories with FaceRig is pleasant. Most
participants felt satisfied with the human-machine
interaction by role-playing with FaceRig. With regard
to the fluency in telling stories using FaceRig, about
85% agreed that FaceRig could increase the fluency
of their storytelling. The items and results of FaceRig
user experience are shown in Table 4.
For the qualitative analysis, the participants
provided some opinions on their FaceRig using
experience. Most suggested that they were relaxed
when using FaceRig because what the audience
watched
was not their faces but the faces of the
Table 3: The results of questionnaire part 3: Interaction with
team members.
ID
Items
SA A N D SD
1
After knowing that we are going to tell stories with
FaceRig, the group members and I are involved in
the design of story plots and the emotional
expressions of the characters.
30.8% 61.5% 7.7% 0% 0%
2
While using FaceRig, I will observe the facial
expressions made by the other group members.
30.8% 61.5% 7.7% 0% 0%
3
While using FaceRig, the facial expressions of the
group members will affect the way I express the
emotions of the character when I tell stories.
38.5% 30.8% 30.8% 0% 0%
4
While using FaceRig, I will come up with some
new ideas and improvise according to the facial
expressions of the group members.
30.8% 46.2% 15.4% 7.7% 0%
5
I like very much to tell stories with FaceRig with
the group members.
15.4% 53.8% 23.1% 7.7% 0%
SA stands for strongly agree, A for agree, N for neutral, D
for disagree, and SD strongly disagree
Table 4: The results of questionnaire part 4: Use of FaceRig.
ID
Items
SA A N D SD
1
In general, I think it is easy to use FaceRig.
23.1% 53.8% 7.7% 15.4% 0%
2
Telling stories with FaceRig is pleasant.
23.1% 61.5% 7.7% 7.7% 0%
3
I am satisfied with the human-machine interaction
provided by the FaceRig role-playing software.
23.1% 61.5% 7.7% 7.7% 0%
4
FaceRig helps me to tell stories more fluently.
0% 61.5% 23.1% 7.7% 7.7%
SA stands for strongly agree, A for agree, N for neutral, D
for disagree, and SD strongly disagree
characters. “I was comfortable to make facial
expressions because the audience could not see my
face.” However, some participants still felt stressed
because they were worried about their performance.
Besides, they thought that conventional storytelling
required the imagination of the audience, but with the
visualization, the audience could easily engage in the
story. Moreover, they shared their opinions about the
quality of the real-time facial animation features.
Some people found that some characters, especially
animals, could not express some emotions. For
example, some characters were designed to be serious
people, and those characters could not express a
delightful smile. Last but not least, the participants
discussed the functions they expected role-playing
Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller,
Teammate and Audience
305
software to have. Some thought that the detection of
body movement would be a good idea, while others
thought that the software could provide inbuilt sound
effects and music. Some expected the role-playing
software to have voice changing functions.
5 CONCLUSIONS
In this paper, we have discussed the role of emotions
in storytelling using the role-playing software,
FaceRig. We argue that emotions could have positive
impacts on both the audience and the storytellers. As
audience members, the participants found that
emotions made them have a better experience of
storytelling and be more engaged in the plot. As
storytellers, they felt relaxed and engaged in the
storytelling process. Moreover, as teammates, they
enjoyed the collaborating processes through FaceRig
and found that teammates’ emotions would influence
the way they told stories, and they might come up
with new ideas or improvise due to this interactive
collaboration process. Finally, as software users, most
of the participants found FaceRig easy to use and
provided some suggestions for the features that could
be included. Those suggestions could be a reference
for the role-playing software developers.
There are some limitations in the study which can
be improved in the future study. The participants of
this study played the role as the storytellers and story
listener. They might be more familiar with the
software and the storytelling processes. These might
have some impacts on the results. The researcher
might recruit more participants as simple audience in
future study. Furthermore, every participant used the
same software to tell story in the same designing and
storytelling process in the current study. It is possible
that the researcher conducts a comparison between
the FaceRig role-playing software and other
storytelling approach without emotions and role-
playing in future study.
REFERENCES
Barrett, H., 2006, March. Researching and evaluating digital
storytelling as a deep learning tool. In Society for
information technology & teacher education
international conference, pages 647-654. Association for
the Advancement of Computing in Education (AACE).
Cavazza, M., Pizzi, D., Charles, F., Vogt, T., & André, E.,
2009, May. Emotional input for character-based
interactive storytelling. In Proceedings of The 8th
International Conference on Autonomous Agents and
Multiagent Systems-Volume 1, pages 313-320.
International Foundation for Autonomous Agents and
Multiagent Systems.
Center for Digital Storytelling, 2005. Center for Digital
Storytelling Web site. Retrieved May 12, 2007, from
http://www.storycenter.org/history.html
Chandrasekar, H., Harte, S., Sherman, J., Park, K. T., & Lee,
H. C., 2018. Gaining Perspectives on Patient and Family
Disease Experiences by Storytelling. Academic
pediatrics, 18(4): 475-476.
Costa, S., Brunete, A., Bae, B. C., & Mavridis, N., 2018.
Emotional Storytelling Using Virtual and Robotic Agents.
International Journal of Humanoid Robotics, 15(03):
1850006.
De Lima, E. S., Feijó, B., Barbosa, S. D., Furtado, A. L.,
Ciarlini, A. E., & Pozzer, C. T., 2014. Draw your own
story: Paper and pencil interactive storytelling.
Entertainment computing, 5(1): 33-41.
Dörner, R., Grimm, P., & Abawi, D. F., 2002. Synergies
between interactive training simulations and digital
storytelling: a component-based framework. Computers
& Graphics, 26(1): 45-55.
FaceRig, Version build 1450 [Computer software]. Retrieved
from https://store.steampowered.com/app/274920/
FaceRig/
Freedberg, D., & Gallese, V., 2007. Motion, emotion and
empathy in esthetic experience. Trends in cognitive
sciences, 11(5): 197-203.
Klimmt, C., Roth, C., Vermeulen, I., & Vorderer, P., 2010.
The empirical assessment of the user experience in
interactive storytelling: construct validation of candidate
evaluation measures. Technical Report.
Miller, E. A., 2009. Digital storytelling. University of
Northern Iowa.
Mokhtar, N. H., Halim, M. F. A., & Kamarulzaman, S. Z. S.,
2011. The effectiveness of storytelling in enhancing
communicative skills. Procedia-Social and Behavioral
Sciences, 18: 163-169.
Psomos, P., & Kordaki, M., 2012. Pedagogical analysis of
educational digital storytelling environments of the last
five years. Social and Behavioral Sciences, 46: 1213-
1218.
Robin, B. R., 2008. Digital storytelling: A powerful
technology tool for the 21st century classroom. Theory
into Practice, 47(3): 220-228.
Sandars, J., & Murray, C., 2011. Digital storytelling to
facilitate reflective learning in medical students. Medical
Education, 45(6): 649-649.
Schmoelz, A., 2018. Enabling co-creativity through digital
storytelling in education. Thinking Skills and Creativity,
28: 1-13.
Staal, L. A., 2000. The story face: An adaptation of story
mapping that incorporates visualization and discovery
learning to enhance reading and writing. The Reading
Teacher, 54(1): 26-31.
Tsou, W., Wang, W., & Tzeng, Y., 2006. Applying a
multimedia storytelling website in foreign language
learning. Computers & Education, 47(1): 17-28.
CSEDU 2019 - 11th International Conference on Computer Supported Education
306