OCC FOR EMOTION GENERATION IN e-LEARNING SYSTEMS

Efhymios Alepis and Maria Virvou

Department of Informatics, University of Piraeus, 80 Karaoli & Dimitriou St., 18534 Piraeus, Greece

Keywords: Affective interaction, Multimedia, e-Learning, Mobile devices, Intelligent tutoring systems, Pedagogical

personas, Multi-modal interfaces.

Abstract: This paper describes an educational authoring tool that incorporates the OCC cognitive theory of emotions

in order to help instructors create and author affective courses. The authoring tool provides an important

facility to instructors for the creation of their own tutoring characters for the user interface of the resulting

applications. In this way, the tutoring characters that are speaking, animated personas may represent the

teaching behaviour of the human instructor who is in charge of the remote lessons. Students, who are going

to use the resulting educational applications, will have a user interface that is more human-like and

affective. Thus they may feel less deprived of the human-human interaction between them and a human

teacher that would take place in the settings of a real classroom.

1 INTRODUCTION

Web-based education is particularly good for remote

teaching and learning at any time and place, away

from classrooms and without necessarily the

presence of a human instructor. However, this

independence from real teachers and classrooms

may cause emotional problems to students who may

feel deprived of the benefits of human-human

interaction. This may affect the educational process

in a negative way because as Goleman (Goleman,

1995) points out, how people feel may play an

important role on their cognitive processes as well.

In this paper we address this problem by providing

an authoring tool for educational applications that

constructs animated tutoring personas with emotion

generation capabilities, thus rendering web-based

human-computer interaction more human-like.

Instructors in general may use this authoring tool to

create their own educational characters that will

interact with their students in e-learning

environments. Personas may be parameterized in

many aspects, the way they speak, the pitch, speed

and volume of their voice, their body-language, their

facial expressions and the content of their messages.

Additionally, for educational purposes, personas

may express specific emotional states by the

incorporation of the OCC (Ortony et. al., 1990)

model. In view of the above our system incorporates

an affective authoring module that relies on the OCC

theory. The system uses the OCC cognitive theory of

emotions for modelling possible emotional states of

users-students as well as for proposing tactics to the

instructors for improving the interaction between the

persona and the student while using the educational

application. Through the incorporation of the OCC

model, the system may suggest that the tutoring

persona should express a specific emotional state to

the student for the purpose of motivating her/him

while s/he learns. Consequently, the persona may

become a more effective teacher, reflecting the

instructors’ vision of teaching behaviour.

In the last decade, education has benefited a lot

from the advances of Web-based technology.

Indeed, there have been many research efforts to

transfer the technology of ITSs and authoring tools

over the Internet. A recent review (Brusilovsky,

1999) has shown that all well-known technologies

from the areas of ITS have already been re-

implemented for the Web. Some important assets

include platform-independence and the practical

facility that is offered to instructors of authoring e-

learning courses at any time and any place. A

remedy for these problems may lie in rendering

human-computer interaction more human-like and

affective in educational software. To this end, the

incorporation of speaking, animated personas in the

user interface of the educational application can be

very important. Indeed, the presence of animated,

speaking personas has been considered beneficial for

educational software (Johnson et. al., 2000, Lester

et. al., 1997). Hence, there have been many

educational applications that incorporate animated

pedagogical personas in their user interfaces (Rist et.

423

Alepis E. and Virvou M.

OCC FOR EMOTION GENERATION IN e-LEARNING SYSTEMS.

DOI: 10.5220/0001840204230426

In Proceedings of the Fifth International Conference on Web Information Systems and Technologies (WEBIST 2009), page

ISBN: 978-989-8111-81-4

al., 1997).

However, as yet there are no authoring tools that

provide parameterization in user interface

components such as speech-driven, animated

personas. The present authoring tool provides the

facility to authors to develop tutoring systems that

incorporate speaking, animated personas who can be

parameterized by the authors-teachers in a way that

reflects their own vision of teaching behaviour in the

user interface of the resulting applications. In many

cases it would be extremely useful to have such

facilities in handheld devices, such as mobile phones

rather than desktop or portable computers so that

additional assets may be gained. Such assets include

device independence as well as more independence

with respect to time and place in comparison with

web-based education using standard PCs. The

proposed system deals with the problem of

facilitating instructors in the educational software

management. Additionally, the resulting educational

software incorporates sophisticated mechanisms for

accessional emotional interaction in the educational

process in order to assist instructors in authoring

user-friendlier, more affective, thus more effective,

educational courses.

2 OVERVIEW OF THE

TUTORING PERSONAS

In the educational applications that result from the

authoring process described in this paper, the

tutoring persona of the user interface is a cartoon-

doctor. The cartoon-doctor is an animated persona

who can move around the tutoring text and can show

parts of the theory that a student should read (Figure

1). It has also incorporated features of human

body-language. It may show patience while the

student reads the theory, boredom if the student is

not responding to the system, wonder if the student

makes an unexpected move, etc. The cartoon-

doctor’s behaviour is programmatically controlled

by an underlying mechanism that relies on the OCC

theory, described in the next section.

Instructors may choose from 27 available speech

engines that the system incorporates. These speech

engines are synthesisers that produce different

voices. The system also offers the facility of

parameterising these voices by changing the pitch,

speed and volume, as illustrated in figure 2. Thus,

the resulting tutoring system may use the voices

differently in different contexts to show enthusiasm,

when the student is doing particularly well, to

imitate whisper, when it judges that the student

needs help, or even to show anger when the student

is consistently careless and does not pay any

attention to the system.

Figure 1: The cartoon-doctor.

Figure 2: Setting parameters for the voice of the tutoring

persona.

As an example, by increasing the pitch and also

the speed and volume of the speech we have the

effect of a more “angry” tone of speaking. This may

also be achieved by selecting the appropriate speech

engine. We may have a special speech engine that

always synthesises speech with an angry tone.

The system incorporates built in tools, to which

only the instructors have access. These tools help the

instructors modify the behaviour of the characters

further, with the personas’ emotion generation

facility as the final objective. Not only can the

instructor command the assistant to say something

under certain circumstances, but s/he can also add

commands in the text that will be spoken, in a way

that the persona may seem to express a specific

emotional state. These commands are understood by

the system and are interpreted into changing speech

attributes, body movements, facial expressions, etc.

3 THE OCC COGNITIVE MODEL

OF EMOTIONS

The expected contribution of our system is to affect

positively the educational process in general. More

WEBIST 2009 - 5th International Conference on Web Information Systems and Technologies

424

specifically, the system should motivate the students

for the purpose of learning more efficiently and also

more enjoyable. In (Soldato and Du Boulay, 1995) it

is suggested that a tutoring system must react with

the purpose of motivating distracted, less confident

or discontented students, or sustaining the

disposition of already motivated students. However,

even if new multimodal capabilities like 3D-video

and speech synthesis have made pedagogical

personas more human-like, there is also a great need

in determining “how” (what exactly should the

pedagogical persona do) and “when” (in which

situation) a pedagogical persona should act/behave

in each part of the tutoring process.

Elliott and colleagues (Elliott et. al., 1999)

believe that pedagogical personas will be more

effective teachers if they display and understand

emotions. More specifically they point out that:

• A persona should appear to care about

students and their progress

• A persona should be sensitive to the student’s

emotions

• A persona should foster enthusiasm in the

student for a subject matter

• A persona may make learning more fun

In view of the above our system incorporates an

affective authoring module that relies on the OCC

theory (Ortony et. al., 1990). The system uses the

OCC cognitive theory of emotions basically for

modeling possible emotional states of users-students

as well as for proposing tactics to the instructors for

improving the interaction between the educational

persona and the student while using the educational

application. Through the incorporation of the OCC,

the system may suggest that the pedagogical persona

should express a specific emotional state to the

student for the purpose of motivating her/him while

s/he learns. Accordingly, the persona becomes a

more effective teacher.

In OCC theory, emotional states arise from

cognitive models that measure positive and negative

reactions of users to situations consisting of events,

agents and objects. Correspondingly, events match

user goals that are key elements in the OCC theory.

Table 1 illustrates intensity variables concerning

user input actions and application events that are

used by the system’s adapted OCC emotion model

in order to propose an apposite for each case

emotional state for the pedagogical persona. The

variables illustrated in table 1 have been specified in

our own implementation and adaptation of OCC in

our educational application.

Table 1: Variables for calculating the intensity of events

for the OCC theory.

Event Variables

•

a mistake (the user may receive an error

message by the application or navigate

wrongly)

• many consecutive mistakes

• absence of user action for a period of time

• action unrelated to the main application

• correct interaction

• many consecutive correct answers (related to

a specific test)

• many consecutive wrong answers (related to

a specific test)

• user aborts an exercise

• user aborts reading the whole theory

• user requests help from the persona

• user takes a difficult test

• user takes an easy test

• user takes a test concerning a new part of the

theory

• user takes a test from a well known part of

the theory

Figure 3: Events-Actions of the persona for the synthesis

of an emotional state.

The application’s user interface is multi-modal,

thus it is possible for the system to monitor and

record user actions such as speed of typing through

the keyboard as well as low voice volume through

the microphone etc. The proposed authoring system

integrates the OCC model by comprising a subset of

five basic emotional states, namely happiness,

sadness, anger, fear and surprise. Each one of the

above mentioned five emotional states can be

synthesized by the animated persona, as it is

illustrated in figure 3.

As an example we describe a situation where a

student is taking a multiple choice test after having

read the corresponding theory of that lesson. The

OCC FOR EMOTION GENERATION IN e-LEARNING SYSTEMS

425

“default” goal for each user is to succeed in

answering correctly the questions of each test. In our

example we assume that the difficulty level of the

test is high and the student has already answered a

couple of questions correctly. At this point in

accordance with the system’s incorporated OCC

model the student is pleased that s/he has answered

correctly the previous questions and is also

experiencing hope that s/he will continue answering

correctly. The corresponding intensity variables for

this event are illustrated in table 1 as “many

consecutive correct answers (related to a specific

test)” and “user takes a test concerning a new part of

the theory”. The second variable indicates that

succeeding in such a test is difficult, thus can invoke

admiration by the pedagogical persona. The user’s

hope of continuing to answer correctly may then be

encouraged by the pedagogical persona by

expressing admiration for the student’s success and

encouraging her/him to continue answering

successfully. In this case the student has a “goal” for

answering correctly. If the student continues her/his

successful course the pedagogical persona will

express happiness, by saying something in a “happy

voice” and/or by smiling or doing a positive gesture.

4 CONCLUSIONS

This paper has described an educational authoring

tool that incorporates the OCC theory in order to

create tutoring personas with emotion expression

facilities. Based on each student’s interaction, goals,

achievements and mistakes, the system proposes an

emotional state for the tutoring persona as a tactic in

supporting the educational process. In this way, the

tutoring personas that are speaking, animated

characters may represent the teaching behavior of

the human instructor who is in charge of the remote

lessons. Students, who are going to use the

educational applications, will have a user interface

that is more human-like and affective. Thus they

may feel less deprived of the human-human

interaction between them and a human teacher that

would take place in the settings of a real classroom

and get motivated for the purpose of learning more

efficiently and also more enjoyable.

It is among our future plans to evaluate the

system in order to examine the degree of usefulness

of the authoring tool for the instructors, as well as

the degree of usefulness and user-friendliness for the

students who are going to use the educational

system.

ACKNOWLEDGEMENTS

Support for this work was provided by the General

Secretariat of Research and Technology, Greece,

under the auspices of the PENED-2003 program.

REFERENCES

Brusilovsky, P., 1999. Adaptive and Intelligent

Technologies for Web-based Education. In C.

Rollinger and C. Peylo (eds.), Künstliche Intelligenz

(4), Special Issue on Intelligent Systems and

Teleteaching,19-25.

Elliott, C., Rickel, J., Lester, J., 1999. Lifelike pedagogical

agents and affective computing: An exploratory

synthesis, Lecture Notes in Computer Science, 1600,

pp. 195-212.

Goleman, D., 1995. Emotional Intelligence, Bantam

Books: New York.

Johnson, W. L, J. Rickel, and Lester, J., 2000. Animated

Pedagogical Agents: Face-to-Face Interaction in

Interactive Learning Environments. International

Journal of Artificial Intelligence in Education, vol. 11,

pp. 47-78.

Lester, J., Converse, S., Kahler, S., Barlow, S., Stone, B.,

and Bhogal, R. 1997. The Persona Effect: affective

impact of animated pedagogical agents. In Pemberton

S. (Ed.) Human Factors in Computing Systems, CHI’

97, Conference Proceedings, ACM Press, pp. 359-

366,.

Ortony, A., Clore, G., Collins, A. 1990. The cognitive

structure of emotions, Cambridge University Press.

Rist, T., André, E. and Müller, J. 1997. Adding Animated

Presentation Agents to the Interface. In Proceedings of

the 1997 International Conference on Intelligent User

Interfaces (eds. J. Moore, E. Edmonds & A. Puerta),

79-86. ACM Press, New York.

Soldato, D, Boulay, D., 1995. Implementation of

motivational tactics in tutoring systems. Journal of

Artificial Intelligence in Education 6 (4), pp. 337-378.

WEBIST 2009 - 5th International Conference on Web Information Systems and Technologies

426