EVALUATION OF 3D INTERACTION TECHNIQUES FOR GOOGLE

EARTH EXPLORATION USING NINTENDO WII DEVICES

Takehiko Yamaguchi, Damien Chamaret and Paul Richard

Laboratoire d’Ing

enierie des Syst

emes Automatis

es (LISA), Universit

e d’Angers, Angers, France

Keywords:

3D interaction techniques, Navigation, Nintendo Wii, Google Earth, Operation workload.

Abstract:

We present a multi-modal virtual reality broadcasting system to support science education. This system is

based on a web platform using Adobe Flash and enables users to explore Google Earth using the Nintendo

Wii devices. The Google maps Flash API was used to control the user’s avatar. The Wii Remote was used for

zooming and steering and the Wii Balance Board was used for walking. We tested operation workload for 9

different threshold angle combinations. We found a most low workload threshold angle combination of 45

◦

(for zooming out) /-15

◦

(for zooming in) and of 30

◦

(for steering right) /-30

◦

(for steering left). Moreover, we

found overshoot range during actual operation.

1 INTRODUCTION

Virtual reality (VR) technology has become increas-

ingly popular and its applications span a wide va-

riety of different areas that include science educa-

tion (Johnson et al., 2001), medical training (Basdo-

gan et al., 2001), entertainment (Mignonneauand and

Sommerer, 2005).

The wide spread use of this technology has been

primarily limited because of the high cost of VR sys-

tems (Sines and Das, 2001). With the development

of hardware speed, rendering capabilities, and acces-

sible network bandwidth, network-based VR content

has become available at lower costs. Thus, most of

common web animation and game creating tools such

as Adobe Flash (Adobe, 1996), Unity (Unity, 2010)

have supported 3D capabilities. In addition, a number

of toolkits have been released as open-source software

for web applications (Polys, 2007), especially, with

the advent of the Nintendo Wii.

In section 2, present the related work. In section

3, we describe the proposed system. Then, we present

the experiment that was carried out to investigate the

operation workload during a navigation task. In Sec-

tion 5, the results of the experiment are presented and

discussed. Finally, we conclude and give some tracks

for future works.

2 RELATED WORK

Wii devices have been adopted by a number of re-

searchers for a wide variety of applications (Schlomer

et al., 2008). Generally, in case of using The Wii

Remote (WR) for navigation, rotation angles such as

pitch, yaw, and roll information are used. For ex-

ample, Duran (Duran et al., 2009) used the WR for

controlling wheelchair using pitch and yaw move-

ments. Fikkert (Fikkert et al., 2009) proposed inter-

action techniques using the WR and the Wii Balance

Board (WB). Both input devices were used for navi-

gating a maze application. If the direction is set using

rotation angles, threshold rotation angles are set up on

each axis as a trigger. Then, when rotation angle goes

over the threshold the direction is changed. In this

case, user needs to keep his/her angle position within

the threshold. Actually, it is important to consider the

threshold from the ergonomic viewpoint because of

operation workload of direction input. However, this

issue has not been really investigated so far. Before

the main experiment, we carried out a pilot study to

assess the available operation range (AOR) and ana-

lyze the range of zooming and steering operation.

334

Yamaguchi T., Chamaret D. and Richard P..

EVALUATION OF 3D INTERACTION TECHNIQUES FOR GOOGLE EARTH EXPLORATION USING NINTENDO WII DEVICES.

DOI: 10.5220/0003370103340338

In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP-2011), pages 334-338

ISBN: 978-989-8425-45-4

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

3 SYSTEM CONFIGURATION

We have been developing Flash based 3D Google

Earth navigation system which enables user to ex-

plore Google Earth. The navigation / exploration task

consists in controlling an avatar. We used Google

Maps API for Flash (Google, 2005) for the Google

Earth interface, and Nintendo WR and WB as input

devices.

3.1 Hardware Framework

Figure 1 shows the hardware conﬁguration of our sys-

tem. We used a desktop computer, the WR and WB,

and a front projected visual display. The WR and WB

are connected to the PC through a Bluetooth commu-

nication protocol. The WR is equipped with an accel-

eration sensor that allows to measure both pitch and

roll movements. The WB is equipped with 4 force

sensors (top left, top right, bottom left, and bottom

right).

Figure 1: Illustration of the experimental conﬁguration.

3.2 Software Framework

The main interface was developed using Adobe Flash.

We used Google Maps API to display Google Earth

interface and the Papervision3D library to display and

control the avatar. The 3D world developed using Pa-

pervision3D was updated at 40Hz. To control the WR

and WB, we used Wiiyourself! C++ library. Data

from the WR and WB are updated at about 1KHz. To

connect between the C++ native application and the

Flash application, we developed a C++ server appli-

cation (TCP/IP protocol).

4 INTERACTION TECHNIQUES

Three different interaction techniques have been pro-

posed to explore the Google Earth effectively. We de-

ﬁned a surﬁng mode (Fig. 2) to go straight and stop,

then we deﬁned a zooming mode (Fig. 3) to scale up

and down, and we deﬁned a steering mode (Fig. 4) to

turn left and right.

Figure 2: Illustration of the surﬁng interaction technique.

Figure 3: Illustration of the zooming interaction technique.

Figure 4: Illustration of the steering interaction technique.

EVALUATION OF 3D INTERACTION TECHNIQUES FOR GOOGLE EARTH EXPLORATION USING NINTENDO

WII DEVICES

335

5 EXPERIMENT

We conducted a pilot testing to determine available

operation range (AOR) of zooming and steering op-

eration before the main experiment. The AOR is a

operation range which a user can have while zooming

and steering without any physical demand. Six sub-

jects tested a simple moving task for 1 minute. We

found that zooming operation has an asymmetric pat-

tern ranging from -25

◦

to 68

◦

(Fig. 3), and steering op-

eration has a symmetric pattern ranging from -56

◦

(Fig. 4). Therefore, we decided to use nine thresh-

olds angle combinations for the experiment.

5.1 Design and Procedure

The aim of this experiment is to ﬁnd a best threshold

combinations for zooming and steering operations to

for exploring Google Earth. A total of 14 subjects (be-

tween 19 and 24 years old) participated in the exper-

iment. Each session included 9 trials that contained

3 simple subtasks deﬁned below. After each trial,

NASA-TLX questionnaires were given to each sub-

ject. Subject were instructed to ﬁnd 3 red goal ﬂags

(Fig. 5). When ﬁnding a yellow ﬂag on the way, sub-

jects were required to zoom in until the yellow ﬂag

become red. When the yellow ﬂag becomes red, the

subjects had to reach it.

Figure 5: Goal ﬂag status on Google Earth platform.

The subjects had to grab a WR with their dominant

hand and stand on the WB. Then they had to do a

trial to get acquainted with the zooming, s teering, and

surﬁng interaction techniques. Then the subjects were

instructed to check the threshold angle for zooming

and steering operation , and had to complete the task

which contains the 3 subtasks. They had to repeat

the task 9 times consecutively (9 threshold combina-

tions). Finally, the subjects had to ﬁll out the ques-

tionnaire.

6 RESULTS

6.1 Zooming Interaction Technique

The results of the NASA TLX questionnaires for

zooming interaction technique are shown in Fig. 6.

The ANOVA showed that there were no main effect

for zooming angle/steering angle and no interaction

between these interaction techniques. These results

show that participants were able to do zooming oper-

ation without effect of zooming/steering angle com-

binations.

Figure 6: Result of NASA-TLX for zooming interaction

technique.

6.2 Steering Interaction Technique

The results of the NASA TLX questionnaires for

steering interaction technique are shown in Fig. 7.

The ANOVA showed signiﬁcant main effect on steer-

ing angle for physical demand (F

(2,26)

= 5.22, p <

0.05), and for performance (F

(2,26)

= 3.74, p < 0.05).

LSD multiple comparison on Physical demand proved

that the effect of steering angle on RC1 was higher

than that of RC2/RC3, but there was no difference

signiﬁcantly between RC2 and RC3. As for the

LSD multiple comparison on performance, the ef-

fect of steering angle on RC1 was higher than that

of RC2, but there was no difference signiﬁcantly be-

tween RC2/RC3 and between RC1/RC3. These re-

sults suggest that participants were able to do steer-

ing operation without effect of zooming operation.

However, the steering angle combination affected the

steering operation.

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

336

Figure 7: Result of NASA-TLX for steering interaction

technique.

Figure 8: Completion time for each sub tasks.

6.3 Completion Time

Results relative to the completion time are illustrated

in Fig. 8 shows the results of the completion time for

each. The ANOVA showed no signiﬁcant main effect

on zooming angle, steering angle, and no interaction

between these interaction techniques. Since there is

signiﬁcant effect on the subjective results for steering

operation as showed in the previous subsection, we

expected the completion time supports the signiﬁcant

effect. However, ANOVA showed no signiﬁcant as

above. This result suggest that the subjective data are

affected by primitive operation workload more than

accomplishment of whole task so that actual operation

range during experimental task is considered in the

next subsection.

6.4 Overshoot of Operation Range

The results of the distribution of zooming and steer-

ing operation range are illustrated in Fig. 9. The mean

value was calculated to average over threshold oper-

ation angle which is operation angle over the set up

threshold angle. We expected that overshoot would

happen since it is difﬁcult to stop these operations on

the threshold so that the overshoot was happened on

overall tasks as shown in Fig. 9.

Figure 9: Distribution of operation range for each interac-

tion techniques.

Overshoot Range of Zooming Out Operation.

We observed a signiﬁcant effect on zooming angle

(2,24)

= 8.474, p < 0.01). LSD multiple compari-

son on zooming angle proved that the effect of zoom-

ing angle on PC1 was higher than that of PC2 and

PC3, and there was no difference signiﬁcantly be-

tween PC2 and PC3. The results show that the over-

shoot range has expanded as the set up threshold be-

comes small.

Overshoot Range of Zooming In Operation. The

ANOVA showed that there were no main effect for

zooming and steering angle and no interaction be-

tween these interaction technique. Since the set up

threshold was -15

◦

on all of angle combinations, there

was no signiﬁcant difference. According to the mini-

mum operation angle which we measured in the pilot

testing as mentioned in the Introduction, it was -25

◦

Therefore, the effect of overshoot range is limited in

about 10

◦

Overshoot Range of Steering Left Operation.

The ANOVA showed signiﬁcant main effect on steer-

ing angle (F

(2,24)

= 32.134, p < 0.01). LSD multiple

EVALUATION OF 3D INTERACTION TECHNIQUES FOR GOOGLE EARTH EXPLORATION USING NINTENDO

WII DEVICES

337

comparison on steering angle proved that the effect of

steering angle on RC1 was higher than that of RC2

and RC3. The effect on RC2 is higher than that on

RC3. The results show that the overshoot range has

expanded as the set up threshold becomes small since

the overshoot value was higher than that of the zoom-

ing operation. We estimate that the ratio of overshoot

affected subjective workload.

Overshoot Range of Steering Right Operation.

The ANOVA showed signiﬁcant main effect on steer-

ing angle (F

(2,24)

= 100.303, p < 0.01). LSD multiple

comparison on steering angle proved that the effect of

steering angle on RC1 was higher than that on RC2

and RC3, and the effct on RC2 is higher than that on

RC3. The results show that the overshoot range has

expanded as the set up threshold becomes small and

estimate that the ratio of overshoot affected subjective

workload as with the result of overshoot of steering

left operation since the steering operation has sym-

metric operation pattern.

Effect of Overshoot for Operation. Even if the

operation range includes the overshoot behavior, we

could expect that there are no signiﬁcant effect on

subjective and on objective data as long as the op-

eration range work within available operation range.

However actual task includes contiguous action of

each interaction technique. For example, when a user

action frequently shifts from zooming in operation to

zooming out operation, overshoot affects start time of

next action as a delay time so that this suggest that

the delay time affects subjective view and task perfor-

mance. Moreover, in case of that the user causes er-

ror behavior, the overshoot also affects reaching time

which is a time until user reaches target zooming level

or steering direction.

7 CONCLUSIONS

The goal of this study is to ﬁnd available operational

range (OOR) for low operation workload associated

with zooming and steering action using the Wii Re-

mote. We investigated actual operation pattern and

carried out and experiment which mixed zooming and

steering operation. We found no effect of zooming

angle combinations and steering angle combinations

as for the zooming operation. However, steering an-

gle combination affected the result of physical de-

mand and performance. We found no signiﬁcant ef-

fect on the completion time for all conditions. How-

ever, we observed overshoot values for steering oper-

ation higher than that of the zooming operation. We

believe that this result affect the physical demand and

performance workload. Moreover, we found that the

overshoot range has expanded as the set up threshold

becomes small. As a result from the subjective and

performance data, we deﬁned the OOR: for zooming

operation is between -15

◦

and 45

◦

, and for steering op-

eration is between -30

◦

and 30

◦

. As future work will

will try to improve the OOR by considering feedback

information.

REFERENCES

Adobe (1996). Adobe ﬂash. http://www.adobe.com/

products/ﬂash/.

Basdogan, C., Chih-hao, H., and Srinivasan, M. A. (2001).

Virtual environments for medical training:graphical

and haptic simulation of laparoscopic common bile

duct exploration. In IEEE/ASME Transactions on

mechatronics, Vol.6, No.3 .

Duran, L., Fernandez-Carmona, M., Urdiales, C., Peula,

J. M., and Sandoval, F. (2009). Conventional joy-

stick vs. wiimote for holonomic wheelchair control.

In IWANN ’09 Proceedings of the 10th International

Work-Conference on Artiﬁcial Neural Networks.

Fikkert, W., Hoeijmakers, N. van der Vet, P., and Nijholt, A.

(2009). Navigating a maze with balance board and-

wiimote. In 3rd International Conference on Intelli-

gent Technologies for Interactive Entertainment (IN-

TETAIN 09).

Google (2005). Google maps api for ﬂash. http://code.

google.com/intl/en/apis/maps/documentation/ﬂash/3d-

maps.html.

Johnson, A., Moher, T., Ohlsson, S., and Leigh, J.

(2001). Exploring multiple representations in ele-

mentary school science education. In Virtual Reality,

2001. Proceedings. IEEE.

Mignonneauand, L. and Sommerer, C. (2005). Designing

emotional, metaphoric, natural and intuitive interfaces

for interactive art, edutainment and mobile communi-

cations. In Computers & Graphics 29, p.p837851.

Polys, N. F. (2007). Parallel realities? the requirements of

web3d and immersive vr. In IEEE VR Workshop 2007:

Future Standards for Immersive VR.

Schlomer, T., Poppinga, B., Henze, N., and Boll, S. (2008).

Gesture recognition with a wii controller. In Proc.

of the 2nd International Conference on Tangible and

Embedded Interaction (TEIf 08).

Sines, P. and Das, B. (2001). Vrsemlab: A low cost virtual

reality system to illustrate complex concepts involving

spatial relationships. In Vol.5, No. 1, The International

Journal of Virtual Reality.

Unity (2010). Unity. http://www.unity3d.com.

GRAPP 2011 - International Conference on Computer Graphics Theory and Applications

338