Learning Support System for Paleontological Environment

Based on Body Experience and Sense of Immersion

Extinct Animals Move in Synchronization with Human Actions

Tomohiro Nakayama

, Kaori Izuishi

, Fusako Kusunoki

, Ryuichi Yoshida

, Takayuki Adachi

Takeki Ogitsu

, Hiroshi Takemura

, Hiroshi Mizoguchi

and Shigenori Inagaki

Department of Mechanical Engineering, Tokyo University of Science, 2641 Yamazaki, Noda-shi, Chiba-ken, Japan

Department of Computing, Tama Art University, Tokyo, Japan

Graduate School of Human Development and Environment, Kobe University, Hyogo, Japan

Keywords: Kinect Sensor, Body Experience, FLASH Animation, Learning Support System.

Abstract: This paper proposes a simulation-based learning support system that animates extinct animals in

synchronization with a human’s location and actions. The system provides learners with a near-real body

experience, and the sense that they have entered the paleontological environment. The aim of the proposed

system is for this experience to improve the learning effect. Obviously, it is impossible for learners to

experience the actual paleontological environment, and it is therefore difficult for them to understand the

link between this environment and its actuality. This difficulty can be decreased by the proposed system,

which provides learners with a real body experience and sense of immersion. At this point in time, the

authors are implementing a system based on the proposed concept. As a preliminary evaluation, we conduct

an experiment using the system, and interview the participants about the improvement in the learning effect.

The experimental results confirm that most learners feel a tangible improvement in the learning effect due to

the enjoyment of moving their body during the learning process.

1 INTRODUCTION

In recent years, environmental problems involving

the destruction of nature have worsened. To improve

our understanding of these problems, we must

understand life in past geological periods and the

biological environment of former generations.

However, learners cannot watch or experience

extinct animals and their environment. Thus, it is

difficult to consider the learning target in relation to

real things, which presents a difficulty in improving

the learning effect.

Thus, to overcome this issue, we focus on the

body experience and sense of immersion in learning.

We are developing a more experiential simulation-

based learning support system, in which the learner

can enter the virtual environment of extinct animals.

The system measures specific information, such as

the human’s location or actions. This allows the

virtual environment and extinct animals to move

across a large-scale screen. In this way, the system

can compensate for near-real body experiences and

immerse users in the environment. This experience

and immersion lead to an improvement in the

learning effect.

In this paper, as the first step toward realizing the

above system, we describe a learning support system

that moves in synchronization with the learner. After

the learners have used the system, we interview

them about the effectiveness of the proposed

approach.

2 LEARNING THROUGH BODY

EXPERIENCE

2.1 Introduction of Body Experience

into Learning

Worsening environmental problems mean that

education is becoming increasingly treasured as a

way of relating the linkage between man and the

environment. To develop an understanding of these

problems, we must understand the wider transition

of the biological environment from ancient to

252

Nakayama T., Izuishi K., Kusunoki F., Yoshida R., Adachi T., Ogitsu T., Takemura H., Mizoguchi H. and Inagaki S..

Learning Support System for Paleontological Environment Based on Body Experience and Sense of Immersion - Extinct Animals Move in Synchroniza-

tion with Human Actions.

DOI: 10.5220/0004942802520257

In Proceedings of the 6th International Conference on Computer Supported Education (CSEDU-2014), pages 252-257

ISBN: 978-989-758-021-5

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

modern times. However, although extinct animals

and their environment actually existed, it is

impossible for learners to watch and experience

them. Additionally, it is impossible for learners to

experience the wide-scale transition of the biological

environment from such periods. There is a difficulty

in learning about what one cannot actually

experience, and so learners struggle to link their

understanding of the animals and their environment

to reality. Thus, learners may not realize that what

they are being taught substantively existed, which

prevents an improvement in the learning effect.

In this regard, we consider that the difficulty in

immersing the user into the virtual environment is

caused by a lack of body experience in the learning.

The body experience of the learner is limited to

watching or hearing the virtual environment from

the outside, and so learners are not immersed in the

environment(Akiko Deguchi, 2010; Takayuki

Adachi 2012).

In recent years, it has become possible to

replicate various environments with increasing

degrees of reality using technologies such as

computer graphics. Hence, the number of learning

support systems that use these technologies in the

form of an animation or game is increasing (El

Rhalibi, 2011; Wernhuar Tarng, 2007; Michael

Tscholl, 2013). These systems improve learners’

motivation and the learning effect in a recognizable

manner. However, learners can still only watch or

learn from outside the virtual environment. Thus,

they cannot obtain a satisfactory body experience

and level of immersion, and it is difficult for them to

consider what they learn from the virtual

environment in relation to its substantive existence.

With the aim of improving the learning effect,

we are developing an effective learning support

system that permits a body experience (Figure 1). In

this system, extinct animals and their environment

are projected onto a large-scale display. Learners

move the animals or environment, and obtain

knowledge by moving their own body. To realize

Figure 1: Proposed experiential learning support system.

this, we use various sensors, and measure the

learners’ location, pose, and actions. Based on these

measurements, the system can be controlled in a

realistic manner. In this way, by changing the virtual

environment according to body actions, learners feel

more of a body experience and sense that they have

entered into the environment. From this body

experience, the system provides deeper immersion

than watching illustrations or videos. This

immersion can help to improve the learning effect.

Furthermore, moving the extinct animals in

synchronization with the learner makes it possible to

consider the extinct animal as being, in some sense,

real, rather than imaginary. Thus, the learner can

understand extinct animals more completely.

2.2 Current Implemented System

2.2.1 Overview

To implement the above learning support system, we

are developing an animation system incorporating

extinct animals. Figure 2 shows an overview of the

system. This animation moves in synchronization

with a human’s location and movement. By playing

with the content of this animation, the learner listens

to information about the animal and their

environment.

In this system, the extinct animal begins to walk

when the human walks, and stops when the human

stops. The background image scrolls according to

the human’s location. The learner sees the extinct

animal following the pace of his/her walking. In this

way, learners feel an affinity with the extinct animal,

and obtain the sense that they are walking in the

environment with the animal.

Figure 2: Overview of the implemented system.

LearningSupportSystemforPaleontologicalEnvironmentBasedonBodyExperienceandSenseofImmersion-Extinct

AnimalsMoveinSynchronizationwithHumanActions

253

We expect that this experience and sense will lead to

improvements in the learning effect.

2.2.2 Configuration of the System

The system must be able to control the animation in

synchronization with the human’s movement, which

requires real-time knowledge of the human’s

location and movement status (walking or stopped).

We utilize Microsoft’s Kinect sensor for this

purpose. Figure 3 shows the configuration of the

system.

The Kinect sensor is a range image sensor,

originally developed as a home video game device.

Although cheap, the sensor can record advanced

measurements of a user’s location. Additionally, this

sensor can recognize humans and the human

skeleton using a library such as OpenNI. It can

measure the location of human body parts, such as

hands and legs, and we can detect the human’s pose

or status (e.g., walking, standing, sitting) from this

function and location information(Jamie Shotton,

2011).

Figure 3: System configuration.

Figure 4: Measuring of human torso location.

We use these functions of the Kinect sensor and

OpenNI library to recognize humans and detect the

human skeleton. Then, we measure the location of

the human torso (Figure 4). From the transition of

pixel coordinates in the camera image of the human

torso location, we can estimate the velocity and

direction in which the human is walking.

The animation of extinct animals is created with

ActionScript, and moves according to input numeric

values. Information about the human’s location,

walking speed, or direction is sent to the PC that

controls the animation. At this point, socket

communication is used for the exchange of

information. The PC controls the FLASH animation

according to the information received, and the

animation is projected onto the screen.

3 EXPERIMENT

3.1 Motion Experiment

First, we conducted a motion experiment to confirm

that the animation system moves in synchronization

with human walking.

As shown in Figure 5, we installed a Kinect

sensor and a projector, and then asked a human to

move across the front of the Kinect sensor once or

twice, as shown in Figure 6. Thus, we can confirm

that the following functions have been realized.

Figure 5: Installation status of sensor and projector.

Figure 6: Movement of human in the experiment.

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation

254

Figure 7: Experimental conditions.

Figure 8: Evaluation experiment.

LearningSupportSystemforPaleontologicalEnvironmentBasedonBodyExperienceandSenseofImmersion-Extinct

AnimalsMoveinSynchronizationwithHumanActions

255

 Scrolling of background image linked with

human location.

 Change in behavior of extinct animal on the

screen according to human actions and direction

of movement.

 Animation stops when the human moves beyond

the measurable area of the Kinect sensor.

Figure 7 shows the experimental conditions. The

animal on the screen moves in same direction as the

human is walking, and stops when the human stops.

When the human changes direction, the animal

changes direction in synchronization with the human.

Additionally, when the human moves out of the

range of the Kinect sensor, the animal looks in the

direction that the human walked away, as if

watching after the human.

The background image scrolls in the opposite

direction to that in which the human is walking, as if

the human is moving forward in the environment.

The background image scrolls according to the

human’s location, moving faster when the human

walks faster.

From the above, we can confirm that we

succeeded in realizing the proposed system in which

animation contents move in accordance with a

walking human.

3.2 Questionnaire Survey

Second, we conducted an interview to assess the

improvement in the learning effect produced by the

proposed system. We allowed children to play and

learn with the system, and then conducted interviews

with them.

The experiment employed 29 6th-grade children.

As the children played with the proposed system, we

explained about the extinct animal and environment

on the screen (Figure 8). We then interviewed the

children using the following questionnaire.

< Questionnaire>

Which learning contents help to progress your

learning?

1. This learning support system (proposed system)

2. Conventional learning method (e.g., textbook,

classroom learning, model, movie, etc.)

3. Neither of the above

Figure 9 shows the results of this questionnaire.

We then asked the children who chose “1.

Proposed system” why they thought this was the

case. Some of the responses are given below.

 “It is effective for enjoying learning to move my

own body.”

 “It is interesting that the extinct animal moves

according to my actions.”

 “We can remember the learning content, because

it is fun.”

 “It is very interesting that the animal follows that

the animal follows my movements”

From these answers, we can confirm that the

participants enjoyed learning by moving their bodies.

By moving the animal and background according to

their body actions, they felt a degree of enjoyment,

and sensed that they were closer to the animal and

its environment. In this way, most of the learners felt

an improvement in the learning effect due to this

sense of immersion.

Additionally, the following opinions were

voiced:

 “It is interesting to learn the role and mechanism

of the sensors.”

 “I wonder how this system measures the human

torso location.”

Thus, the children took an interest in both the

learning content and the system and sensors. From

this, we believe the system led to an increased

interest in intelligent machines.

One of the children who chose “2. Conventional

learning method” stated, “In a museum, I feel the

learning targets ancient things more deeply. So, in

the case of studying old things such as extinct

animals, I think it is better to study in a museum

than with the proposed system.” In a museum, many

things are displayed that provide a sense of the

ancient. In contrast, our system uses various modern

devices, such as a PC and Kinect sensor. We think

that seeing these devices may prevent users from

‘feeling’ the ancient period. Thus, seeing the system

and its sensors may lead to an interest in intelligent

machines, but these devices should be hidden so that

learners cannot see them. Another opinion stated, “I

have watched more large-scale exhibits. They are

more dynamic and attractive than this system.”

Figure 9: Questionnaire results.

At this point, we are using only one Kinect

sensor and one projector. In future, we intend to

increase the number of sensors and projectors, and

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation

256

expand the scale of the system. Thus, we will make

the proposed system more experiential than

conventional learning methods or contents.

4 CONCLUSIONS

While learning about extinct animals and the

environment in which they lived, learners cannot

actually watch or experience the environment.

Hence, it is difficult for learners to understand

the linkages between real things and attain a greater

degree of learning. Thus, to compensate for this, we

have focused on the body experience component of

learning, and have proposed a learning support

system in which users learn by moving their body.

In this system, the learning content is controlled

according to the location, status, and actions of

thelearner. By moving their own bodies, learners can

obtain a near-real experience, as if they have entered

the environment, and this improves their

understanding of the learning target. In this paper,

towards the realization of such a system, we

developed a learning support system that moves in

synchronization with the learner’s location and

actions (walking or stopping). After confirming that

the system operated as intended, we conducted a

questionnaire survey. From the results of the

experiment, we have confirmed the potential for

improvement in the learning effect due to the

enjoyment of moving their bodies while learning. In

the future, we intend to develop the system so that

one can synchronize with not only walking also the

other action. And then we intend to conduct

evaluation test for more subjects.

ACKNOWLEDGEMENTS

I am particularly grateful for the illustration (Figure

1) provided by Ms Midori Aoki.

REFERENCES

El Rhalibi, A., Merabti, M., Ruwei Yun, Dan Liu, 2011,

Game Based Learning Framework for Virtual 3D

Dinosaurs Knowledge, Developments in E-systems

Engineering, pages 419-424.

Wernhuar Tarng, Hsin-Hun Liou, 2007, The Development

of a Virtual Dinosaur Museum, Journal of

Educational Technology Systems, Volume 35, Number

4, pages 385-409.

Michael Tscholl, Robb Linfgren, Emily Johnson, 2013,

Enacting Orbits: Refining the Design of a Full-Body

Learning Simulation, Interaction Design and Children

2013, pages 451-454.

Akiko Deguchi, Shigenori Inagaki, Fusako Kusunoki,

Etsuji Yamaguchi, Yoshiaki Takeda, Masanori

Sugimoto, 2010, “Vegetation interaction game: Digital

SUGOROKU of vegetation succession for children,”

ICEC2010, pages 493-495, 2010.

Takayuki Adachi, Masafumi Goseki, Keita Muratsu,

Hiroshi Mizoguchi, Miki Namatame, Masanori

Sugimoto, Fusako Kusunoki, Etsuji Yamaguchi,

Shigenori Imagaki, Yoshiaki Takeda, 2013, Human

SUGOROKU: Full-body Interaction System for

Students to Learn Vegetation Succession, Interaction

Design and Children 2013, pages 364-367.

Takayuki Adachi，Masafumi Goseki，Hiroshi Takemura，

Hiroshi Mizoguchi, Fusako Kusunoki, Masanori

Sugimoto, Etsuji Yamaguchi, Shigenori Inagaki,

Yoshiaki Takeda, 2012,”Study on Identification of

People and 3D Position Measurement with Ultrasonic

Sensors and Kinect Sensors”, Proc. NOLTA2012,

pages 247-250.

Jamie Shotton, Andrew Fitzgibbon, Mat Cook, Toby

Sharp, Mark Finocchio, Richard Moore, Alex Kipman,

and Andrew Blake, 2011,“ Real-Time Human

PoseRecognition in Parts from a Single Depth Image”,

2011 IEEE International Conf. CVPR, pages1297-

1304.

LearningSupportSystemforPaleontologicalEnvironmentBasedonBodyExperienceandSenseofImmersion-Extinct

AnimalsMoveinSynchronizationwithHumanActions

257