Evaluating the Emotional Effects of Role-playing Software on

Interactive Digital Storytelling from the Perspectives of User,

Storyteller, Teammate and Audience

Gloria Yi-Ming Kao and Xing-Yi Huang

Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology,

Keelung Rd., Sec. 4, Da’an Dist., Taipei City 10607, Taiwan (R.O.C.)

Keywords: Interactive Digital Storytelling, Emotions, Role-playing, Authoring Tools, User Experience.

Abstract: This paper discusses the applications of emotions in interactive digital storytelling. Emotion is one of the most

important elements in digital storytelling. Participants were asked to design a story collaboratively and present

it to the audience using the role-playing software, FaceRig. They were asked to fill out a questionnaire about

their attitude toward this activity from the perspective of an audience member, a storyteller, a group member,

and a software user. The findings indicated that most participants held a positive attitude toward this activity

whether as an audience member or as a storyteller. Most participants thought that listening to and telling

stories with FaceRig was a pleasant and appealing experience. They loved the idea of performing the facial

expressions of the characters during both the preparation process and the performing process. We concluded

that the application of emotions in storytelling is a way to improve the storytelling experience of both audience

and storytellers. To further enhance user experience, we suggest that designers of such role-playing software

could implement features such as voice changing functions and body movement detection.

1 INTRODUCTION

Studies of storytelling have suggested that

storytelling has had a great influence on human

culture, on education, and even on the medical field

(Barrett, 2006; Chandrasekar et al., 2018; Schmoelz,

2018; Psomos and Kordaki, 2012; Tsou et al., 2006).

In the field of education, many studies have proposed

that storytelling is an effective learning and teaching

tool, and that it increases students’ learning

engagement, learning outcomes, and their creativity

(Barrett, 2006; Mokhtar et al., 2011; Schmoelz, 2018;

Sandars and Murray, 2011; Tsou et al., 2006). Digital

storytelling, or storytelling with the help of

technology, has become a common field in the realm

of storytelling. The difference between traditional and

digital storytelling is that digital storytelling is

performed through different kinds of media, such as

computer-produced images, typed text, sound effects,

music, and video (Miller, 2009; Robin, 2008). Digital

storytelling could not only provide positive results in

learning, but the media of digital storytelling could

also catch students’ attention and increase their

motivation (Robin, 2008). Furthermore, Dörner,

Grimm, and Abawi (2002) argued that digital

storytelling allows learners to actively take part in the

creation of the story. Participation in the process of

creating the story facilitates some important skills,

such as communication skills, ability to evaluate

one’s own work, social learning, and emotional

intelligence (Robin, 2008).

Emotions are one of the essential elements of

digital storytelling (Center for Digital Storytelling,

2005). Emotional content can provide the audience

with connections to the story content, and can make

the audience feel that the discussed issues are more

related to themselves (Center for Digital Storytelling,

2005). Freedberg and Gallese (2007) also proposed

that when observing people’s emotional expressions

or pictures arousing strong emotions, observers tend

to have emotional circuits in the similar facial

muscles or have the same emotions. Moreover, Costa,

Brunete, Bae, and Mavridis (2018) suggested that the

emotions expressed by the storytellers would impact

on the emotional responses of the audience. The

emotions can even make the audience stand in the

characters’ shoes (Costa et al., 2018). Cavazza, Pizzi,

Charles, Vogt, and André (2009) also agreed that

300

Kao, G. and Huang, X.

Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller, Teammate and Audience.

DOI: 10.5220/0007709503000306

In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 300-306

ISBN: 978-989-758-367-4

emotions could increase the audience’s sense of

reality. With the impact of emotions, a story could be

more than just a simple story. The emotional elements

of traditional storytelling were presented by the

variations in intonation. However, human beings

receive emotional cues not only from vocal messages

but also from visual information, such as facial

expressions and body language.

In this study, we adopted the role-playing

software, FaceRig, which allow users to role-play

different 3D characters by performing facial

expressions. The integration of role-playing in

storytelling helps storytellers depict the plot in detail

by presenting the characters’ images and emotions.

Story listeners could engage in the stories more

deeply because they could refer to the characters’

images and characters’ facial expressions through the

role-playing software we adopted.

In this study, we aimed to know whether emotions

could arouse the interest, increase the motivations,

and improve the state of engagement of audience and

storytellers in storytelling through the software,

FaceRig.

2 RESEARCH TOOLS

2.1 FaceRig Role-playing Software

This research adopted FaceRig (Version build 1450),

a type of role-playing software, as the instrument of

storytelling. FaceRig is an open creation platform for

users to roleplay different characters with its real-time

facial animation feature. It provides users with

several characters, backgrounds and props to choose

from. The emotions of the characters will change

along with the facial expressions of the user, which

are captured by the computer camera. Moreover,

users can customize their own characters using

FaceRig, giving them more possibilities in role-

playing, and they can roleplay their own characters

while watching one another’s performance through

either one of the following two functions of FaceRig.

First, FaceRig can be linked to online chat rooms, for

example, Skype and Hangouts, so the players can

watch the others’ characters and their performance in

the chat room. Second, the real-time facial animation

function of FaceRig can detect more than one

person’s facial expressions and apply the emotions of

different users to different characters. That is,

FaceRig allows more than one person to use it at the

same time with the same computer and interact with

each other with different characters.

FaceRig used in this study provides 52 different

characters, including humans and animals. It has 32

scenes. For instance, users can role-play as animals

such as dogs, wolves, and dragons, or impersonate

other human beings of different genders or ages (see

Figure 1). They can also choose the role-playing

background as streets, forests, a hall, and so on (see

Figure 2).

Figure 1: Characters provided by FaceRig.

Figure 2: Scenes provided by FaceRig.

2.2 The Questionnaire of User

Experience Evaluation

The questionnaire used in this study was derived from

the IRIS evaluation toolkit (Klimmt et al., 2010). We

designed the questionnaire based on four

perspectives: audience, storytellers, interactions with

teammates, and use of FaceRig. Audience refers to

participants that watch the storytelling performance

of other participants. Storytellers are the ones who

perform the storytelling through FaceRig.

Interactions with teammates include the interactions

during their preparation and performance. Use of

FaceRig refers to the usability of the system. We

adopted some factors from IRIS, including usability,

effectiveness, satisfaction, and user experiences as a

reference. De Lima, Feijó, Barbosa, Furtado, Ciarlini,

and Pozzer (2014) also adopted these four factors in

their questionnaire. The questionnaire includes 22 5-

point Likert scale items, with 1 standing for strongly

Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller,

Teammate and Audience

301

disagree to 5 for strongly agree, for the quantitative

analysis, and four open-ended questions for the

qualitative analysis. Four open-ended questions were

designed to investigate the “use of FaceRig”

perspective.

3 METHODOLOGY

3.1 Participants

The participants were 13 graduate students from a

graduate school in Taipei, Taiwan. There were five

males and eight females. All the participants were

taking the course, Creative Multimedia Design. The

FaceRig storytelling activity is a part of the course.

Every participant knew that the results of the “The

user experience of telling story with FaceRig”

questionnaire would not influence their grade.

3.2 Procedure

The researchers held a storytelling activity with

FaceRig in order to learn the differences in

storytellers’ and audience’s reactions to role-playing

storytelling and conventional storytelling. FaceRig is

role-playing software, offering users various 3D

characters for role playing. It detects storytellers’

facial expressions from the camera and projects them

onto the 3D characters.

The procedure is as shown in Figure 3. We

introduced FaceRig’s functions and how to operate it

to all participants one month before the official

experiment. Also, pictures of available scenes and

characters were offered, with the agenda of the

activity.

Before the storytelling activity, participants

worked with their group members to come up with

the outline of the story with a concept map, story face

(Staal,

2000).

There

was

limitation

the

content

Figure 3: Procedure.

CSEDU 2019 - 11th International Conference on Computer Supported Education

302

Figure 4: The story face structure (Staal, 2000) and story face of The Frog and the Princess.

Figure 5: Photo of FaceRig storytelling.

of the story. Original and adapted stories were both

accepted. After completing the concept map,

participants needed to design the plots and dialogues

(2 characters). For example, one story topic is The

Frog and the Princess. The story face of their story is

shown in Figure 4. Figure 5 shows a photo of FaceRig

storytelling.

All participants were both the storytellers and

listeners. During the formal activity, participants

worked in groups of two or three (2 groups of 2

students and 3 groups of 3 students) to tell stories with

FaceRig in the same room. When they were not in

Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller,

Teammate and Audience

303

charge of presenting their stories, they listened to

others’ stories and observed the expressions and

interactions between the 3D characters. Two

computers were used for the FaceRig role-playing.

The role-playing performance from the two

computers was linked using the online meeting

software, Hangouts. A third computer was also linked

to the two computers for the projection. Thus, the

audience could easily observe the performance from

the big screen. The storytelling process was recorded

using the inbuilt recording function of both FaceRig

and Hangouts. After each group finished their

storytelling, the participants gave comments to each

group. Later, the participants were asked to complete

the questionnaire.

4 RESULTS

The overall results showed that participants presented

a positive attitude toward the different factors of the

role-playing storytelling activity with FaceRig from

four perspectives: audience, storyteller, team

member, and software user. The results are presented

based on these four perspectives.

4.1 Watching the FaceRig Storytelling

as an Audience Member Procedure

As audience members, most of the participants found

watching storytelling with FaceRig a good

experience. More than 90% of the audience thought

that listening to storytelling with FaceRig was

interesting. Besides, all the participants enjoyed the

idea of adding emotions into storytelling. Moreover,

85% of the participants found storytelling with

FaceRig more appealing than conventional

storytelling. For effectiveness and satisfaction, 85%

found that storytelling with FaceRig assisted them in

understanding the emotions of the characters and

connected them to the plot as if they were really

experiencing the story. See Table 1 for the items and

detailed results of the audience perspective of the

questionnaire.

4.2 Telling Stories with FaceRig as a

Storyteller

The storytellers’ attitudes toward FaceRig

storytelling were mostly positive. Most of them found

it a good experience. Every participant agreed that

telling stories with FaceRig and expressing the

emotions

at the same time was interesting. Every

Table 1: The results of questionnaire part 1: As an audience

member.

Items

SA A N D SD

I think it's fun to listen to the stories performed by

FaceRig.

30.8% 61.5% 7.7% 0% 0%

I like to see performers add emotional expressions

when they tell stories.

76.9% 23.1% 0% 0% 0%

Listen to stories told with FaceRig will give me a

better understanding of the emotions of the

characters.

23.1% 61.5% 15.4% 0% 0%

FaceRig plus emotional expressions will help me

experience better the stories.

15.4% 69.2% 15.4% 0% 0%

Storytelling with FaceRig is more appealing to me

than the storytelling in a normal way.

38.5% 46.2% 15.4% 0% 0%

SA stands for strongly agree, A for agree, N for neutral, D

for disagree, and SD strongly disagree.

Table 2: The results of questionnaire part 2: As a storyteller.

Items

SA A N D SD

I think it's fun that I can express emotions during

storytelling by using FaceRig.

38.5% 61.5% 0% 0% 0%

I think using FaceRig will help me tell better

stories.

7.7% 76.9% 15.4% 0% 0%

I think using FaceRig will help me express

completely well the emotions that I want to show.

7.7% 30.8% 46.2% 15.4% 0%

I can perform more vividly the personalities of the

characters when I tell stories with FaceRig.

23.1% 46.2% 30.8% 0% 0%

In my experience, I think telling stories with

FaceRig can engage me more than the way I used

to do storytelling.

30.8% 59.2% 0% 0% 0%

When I tell stories with FaceRig, I am very

concerned about the reaction of the audience.

30.8% 53.8% 7.7% 7.7% 0%

When I tell stories with FaceRig, I will modify my

interpretation of the character instantly or

improvise according to the audience's response.

30.8% 38.5% 15.4% 15.4% 0%

I am looking forward to telling stories with

FaceRig in the future.

7.7% 61.5% 30.8% 0% 0%

SA stands for strongly agree, A for agree, N for neutral, D

for disagree, and SD strongly disagree

participant also agreed that storytelling with FaceRig

helped them engage in the storytelling process. In all,

85% reported that they cared about the audience’s

CSEDU 2019 - 11th International Conference on Computer Supported Education

304

reactions and responses, 70% thought that FaceRig

allowed them to present the emotions and the

characteristics they expected. Furthermore, more than

90% are looking forward to the next chance to tell

stories with FaceRig. For effectiveness, more than

90% thought that FaceRig helped them tell a better

story. In addition, 77% thought that FaceRig could

thoroughly present the emotions they expected, while

70% reported that they would adjust their

performance according to the reaction of the

audience. The items and results of storyteller

perspective of the questionnaire are shown in Table

4.3 Interacting with Teammates

through FaceRig

The overall opinion on the interaction with team

members in the research activity was positive. More

than 90% of participants engaged in the preparation

process of plot and character design after knowing

that they would use FaceRig as a storytelling

instrument. More than 90% agreed that they would

observe the facial expressions of their teammates

while telling stories with FaceRig. Around 70%

showed that the emotions expressed by their

teammates would influence their facial expressions,

77% declared that they would come up with some

new ideas and improvise because of their teammates’

emotional expressions, and 70% enjoyed the FaceRig

storytelling with their teammates. See Table 3 for

each item and result of the interaction perspective of

the questionnaire.

4.4 Use of FaceRig

The findings of this section are divided into two main

parts: the quantitative and the qualitative results. For

the quantitative analysis, most of the participants

found FaceRig easy, fun, and effective to use. About

80% strongly agreed or agreed that FaceRig is easy to

use. Most participants agreed that the process of

telling stories with FaceRig is pleasant. Most

participants felt satisfied with the human-machine

interaction by role-playing with FaceRig. With regard

to the fluency in telling stories using FaceRig, about

85% agreed that FaceRig could increase the fluency

of their storytelling. The items and results of FaceRig

user experience are shown in Table 4.

For the qualitative analysis, the participants

provided some opinions on their FaceRig using

experience. Most suggested that they were relaxed

when using FaceRig because what the audience

watched

was not their faces but the faces of the

Table 3: The results of questionnaire part 3: Interaction with

team members.

Items

SA A N D SD

After knowing that we are going to tell stories with

FaceRig, the group members and I are involved in

the design of story plots and the emotional

expressions of the characters.

30.8% 61.5% 7.7% 0% 0%

While using FaceRig, I will observe the facial

expressions made by the other group members.

30.8% 61.5% 7.7% 0% 0%

While using FaceRig, the facial expressions of the

group members will affect the way I express the

emotions of the character when I tell stories.

38.5% 30.8% 30.8% 0% 0%

While using FaceRig, I will come up with some

new ideas and improvise according to the facial

expressions of the group members.

30.8% 46.2% 15.4% 7.7% 0%

I like very much to tell stories with FaceRig with

the group members.

15.4% 53.8% 23.1% 7.7% 0%

SA stands for strongly agree, A for agree, N for neutral, D

for disagree, and SD strongly disagree

Table 4: The results of questionnaire part 4: Use of FaceRig.

Items

SA A N D SD

In general, I think it is easy to use FaceRig.

23.1% 53.8% 7.7% 15.4% 0%

Telling stories with FaceRig is pleasant.

23.1% 61.5% 7.7% 7.7% 0%

I am satisfied with the human-machine interaction

provided by the FaceRig role-playing software.

23.1% 61.5% 7.7% 7.7% 0%

FaceRig helps me to tell stories more fluently.

0% 61.5% 23.1% 7.7% 7.7%

SA stands for strongly agree, A for agree, N for neutral, D

for disagree, and SD strongly disagree

characters. “I was comfortable to make facial

expressions because the audience could not see my

face.” However, some participants still felt stressed

because they were worried about their performance.

Besides, they thought that conventional storytelling

required the imagination of the audience, but with the

visualization, the audience could easily engage in the

story. Moreover, they shared their opinions about the

quality of the real-time facial animation features.

Some people found that some characters, especially

animals, could not express some emotions. For

example, some characters were designed to be serious

people, and those characters could not express a

delightful smile. Last but not least, the participants

discussed the functions they expected role-playing

Evaluating the Emotional Effects of Role-playing Software on Interactive Digital Storytelling from the Perspectives of User, Storyteller,

Teammate and Audience

305

software to have. Some thought that the detection of

body movement would be a good idea, while others

thought that the software could provide inbuilt sound

effects and music. Some expected the role-playing

software to have voice changing functions.

5 CONCLUSIONS

In this paper, we have discussed the role of emotions

in storytelling using the role-playing software,

FaceRig. We argue that emotions could have positive

impacts on both the audience and the storytellers. As

audience members, the participants found that

emotions made them have a better experience of

storytelling and be more engaged in the plot. As

storytellers, they felt relaxed and engaged in the

storytelling process. Moreover, as teammates, they

enjoyed the collaborating processes through FaceRig

and found that teammates’ emotions would influence

the way they told stories, and they might come up

with new ideas or improvise due to this interactive

collaboration process. Finally, as software users, most

of the participants found FaceRig easy to use and

provided some suggestions for the features that could

be included. Those suggestions could be a reference

for the role-playing software developers.

There are some limitations in the study which can

be improved in the future study. The participants of

this study played the role as the storytellers and story

listener. They might be more familiar with the

software and the storytelling processes. These might

have some impacts on the results. The researcher

might recruit more participants as simple audience in

future study. Furthermore, every participant used the

same software to tell story in the same designing and

storytelling process in the current study. It is possible

that the researcher conducts a comparison between

the FaceRig role-playing software and other

storytelling approach without emotions and role-

playing in future study.

REFERENCES

Barrett, H., 2006, March. Researching and evaluating digital

storytelling as a deep learning tool. In Society for

information technology & teacher education

international conference, pages 647-654. Association for

the Advancement of Computing in Education (AACE).

Cavazza, M., Pizzi, D., Charles, F., Vogt, T., & André, E.,

2009, May. Emotional input for character-based

interactive storytelling. In Proceedings of The 8th

International Conference on Autonomous Agents and

Multiagent Systems-Volume 1, pages 313-320.

International Foundation for Autonomous Agents and

Multiagent Systems.

Center for Digital Storytelling, 2005. Center for Digital

Storytelling Web site. Retrieved May 12, 2007, from

http://www.storycenter.org/history.html

Chandrasekar, H., Harte, S., Sherman, J., Park, K. T., & Lee,

H. C., 2018. Gaining Perspectives on Patient and Family

Disease Experiences by Storytelling. Academic

pediatrics, 18(4): 475-476.

Costa, S., Brunete, A., Bae, B. C., & Mavridis, N., 2018.

Emotional Storytelling Using Virtual and Robotic Agents.

International Journal of Humanoid Robotics, 15(03):

1850006.

De Lima, E. S., Feijó, B., Barbosa, S. D., Furtado, A. L.,

Ciarlini, A. E., & Pozzer, C. T., 2014. Draw your own

story: Paper and pencil interactive storytelling.

Entertainment computing, 5(1): 33-41.

Dörner, R., Grimm, P., & Abawi, D. F., 2002. Synergies

between interactive training simulations and digital

storytelling: a component-based framework. Computers

& Graphics, 26(1): 45-55.

FaceRig, Version build 1450 [Computer software]. Retrieved

from https://store.steampowered.com/app/274920/

FaceRig/

Freedberg, D., & Gallese, V., 2007. Motion, emotion and

empathy in esthetic experience. Trends in cognitive

sciences, 11(5): 197-203.

Klimmt, C., Roth, C., Vermeulen, I., & Vorderer, P., 2010.

The empirical assessment of the user experience in

interactive storytelling: construct validation of candidate

evaluation measures. Technical Report.

Miller, E. A., 2009. Digital storytelling. University of

Northern Iowa.

Mokhtar, N. H., Halim, M. F. A., & Kamarulzaman, S. Z. S.,

2011. The effectiveness of storytelling in enhancing

communicative skills. Procedia-Social and Behavioral

Sciences, 18: 163-169.

Psomos, P., & Kordaki, M., 2012. Pedagogical analysis of

educational digital storytelling environments of the last

five years. Social and Behavioral Sciences, 46: 1213-

1218.

Robin, B. R., 2008. Digital storytelling: A powerful

technology tool for the 21st century classroom. Theory

into Practice, 47(3): 220-228.

Sandars, J., & Murray, C., 2011. Digital storytelling to

facilitate reflective learning in medical students. Medical

Education, 45(6): 649-649.

Schmoelz, A., 2018. Enabling co-creativity through digital

storytelling in education. Thinking Skills and Creativity,

28: 1-13.

Staal, L. A., 2000. The story face: An adaptation of story

mapping that incorporates visualization and discovery

learning to enhance reading and writing. The Reading

Teacher, 54(1): 26-31.

Tsou, W., Wang, W., & Tzeng, Y., 2006. Applying a

multimedia storytelling website in foreign language

learning. Computers & Education, 47(1): 17-28.

CSEDU 2019 - 11th International Conference on Computer Supported Education

306