Tactile and Tangible Interfaces in Handheld AR for Children

Santiago Gonz´alez-Gancedo

, M.-Carmen Juan

, Ignacio Segu´ı

and Francisco Abad

Instituto Universitario de Autom´atica e Inform´atica Industrial (ai2), Universidad Polit´ecnica de Valencia,

Camino de Vera s/n, 46022 Valencia, Spain

AIJU, Ibi, Alicante, Spain

Keywords:

Tangible User Interfaces, Tactile User Interfaces, Handheld Devices, Augmented Reality.

Abstract:

This paper presents a comparative study between tangible user interfaces (TUIs) and tactile user interfaces

(TacUIs) in handheld AR, with a contribution to the state of the art in HCI oriented to children. While TUIs

work with the manipulation of physical objects, TacUIs work with virtual representations of them. In our

evaluations to compare these two interactions with primary school children, we found that the TacUI was the

fastest for completing the task, what makes it better suited for educational purposes. The TacUI was found

easier to use by the children, although the TUI was found more solid and less slippery. Our conclusions should

be of interest not only to educational researchers, but also to the general HCI community working on tangible

and tactile interfaces.

1 INTRODUCTION

Augmented Reality (AR) is a ﬁeld with a great po-

tential of use with children, providing a very stimu-

lating environment that not only helps visualising 3D

objects, but also enhances motivation and enjoyabil-

ity highly. AR has evolved fast in the recent years,

specially concerning to the area of Human-Computer

Interaction (HCI), and designing user interfaces that

help the students visualise and explore AR environ-

ments.

Tangible User Interfaces (TUIs) deal with the ma-

nipulation physical objects. The hardware used to

work with TUIs and AR in education is very fre-

quently based on markers printed on books, such

as the well-known MagicBook (Billinghurst et al.,

2001), or on other objects, such as cards (Juan et al.,

2011). These systems have the great advantage of

working with a video see-through metaphor that pro-

vides a high sense of presence and immersion with a

direct view from the user perspective.

Tactile User Interfaces (TacUIs

), in opposition

to TUIs, do not deal with physical objects, but with

metaphors or pictures of them displayed on a sensi-

ble surface, and therefore the differences in the user

interaction and perception may be different. TacUIs

can be divided in touch screens (capacitive or resis-

tive) and projection based (Jones et al., 2010). Since

To our knowledge, there is not any standard term for

Tactile User Interfaces, so we introduce TacUIs.

Figure 1: Set used for the evaluation with the children con-

sisting on a TUI and a TacUI in the handheld device.

we use handheld devices in the research, we contex-

tualise this paper under touch screens. TacUIs play an

important role in handheld interaction contributing to

AR providing a new interaction channel.

The relation between TUIs and TacUIs in hand-

held AR has not been examined previously to our

knowledge. In this paper we present a novel study that

compares these two different forms of user interac-

tion. Based on related work studied, we developed an

experiment based on one of the most common uses of

AR in education and entertainment, which consists of

visualising 3D objects from different points of view.

We measured and analysed the differences from an

objective perspective (time spent by the user to inter-

act with the system) and from the user’s subjective

perspective (satisfaction with the system and the in-

393

González-Gancedo S., Juan M., Seguí I. and Abad F..

Tactile and Tangible Interfaces in Handheld AR for Children.

DOI: 10.5220/0004279503930396

In Proceedings of the International Conference on Computer Graphics Theory and Applications and International Conference on Information

Visualization Theory and Applications (GRAPP-2013), pages 393-396

ISBN: 978-989-8565-46-4

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

teraction).

This paper is structured as follows. Section 2 re-

views work related with our study. Section 3 explains

the proposed experiment and its apparatus. Section

4 describes the design of the study. Section 5 shows

the statistical analysis and the results of the study. Fi-

nally, section 6 discusses the conclusions of our re-

search and the future work.

2 RELATED WORK

There exists a large body of research in AR aimed to

children. For instance, Sayed, Zayed, and Sharawy

(2011) presented ARSC, a low cost visualisation tool

of 3D objects that reduced the learning time. The au-

thors used desktop computers with web cameras to

visualise objects on markers. They tested the system

with students between 10 and 17 years old, and 89%

of them were satisﬁed with it. The ARSC set de-

creased the expenses and increased the visualisation

ability (Sayed et al., 2011).

TUIs have also been studied with children. Juan

et al. (2010) combined AR with tangible cubes,

in which each side there was a marker. The users

wore a Head-Mounted Display (HMD) to visualise

the AR scenes, freely manipulating the cubes with

both hands. The authors compared the AR system

that displayed videos to traditional cubes with images

on the sides. They tested the system with children

from 7 to 12 years old, and concluded that despite the

HMD was uncomfortable, children enjoyed more the

AR system (Juan et al., 2010).

In our research, we have paid special attention to

previous work in TacUIs and handheld devices. Wag-

ner, Schmalstieg, and Billinghurst (2006) presented

a collaborative handheld AR game aimed to sort a

collection of artworks by date of creation using ei-

ther a paper, a PC, or a PDA. Users preferred paper

and PDA over the PC, and they also preferred the AR

PDA game over the paper game (Wagner et al., 2006).

Very recently, Gonz´alez-Gancedo et al. (2012) stud-

ied handheld devices with virtual reality (VR) and

AR to be used in the classroom. They compared

VR and AR for their use with handheld devices in

a game about the water cycle. Children from 8 to 10

years evaluated the system, and the authors concluded

that the combination of VR and AR improved moti-

vation end enhancement, although it did not improve

the learning outcomes in comparison to a simple VR

game (Gonz´alez-Gancedo et al., 2012).

This related work motivated our research, and

we attempt to study the relations between TUIs and

TacUIs in handheld devices in order to create refer-

ence for future developments. In this paper we try to

state the advantages and inconveniences of both inter-

actions given our experience.

3 EXPERIMENT

3.1 Description of the Experiment

In our game (Fig. 1) the handheld device was a tablet

PC with a 3MP camera and a capacitive screen. In or-

der to perform a fair comparative we decided to mod-

ify the traditional AR system. In terms of geometric

operations, the translation of the object is determined

by the AR system in the traditional way, but the rota-

tion around the upward axis relative to the marker is

determined dynamically.

The game can be played in two modes. In the tan-

gible mode the child only uses the handheld device

merely as a visualisation system, and she can control

a TUI device to rotate the marker. We built this in-

terface that allowed easily the rotation of the marker

but not the translation. In the tactile mode, the marker

is static, and the interaction is only through the tac-

tile screen. The user can touch and move the ﬁnger

horizontally in the screen, but vertical displacements

do not have any effect. The handheld device was in a

ﬁxed position using a stand to compare better the TUI

and the TacUI.

The AR system placed a model of the Taj Mahal

with several objects in different parts of the virtual

scene located in such a way that it was necessary to

rotate the model to see the objects hidden in the back.

The objective of the game was to count the number

of objects in the scene. Since children were studying

nature and water in other activities, the objects were

drops of water represented as characters.

4 EVALUATIONS DESIGN

4.1 Participants

A total number of 51 children from 8 to 10 years old –

with a mean age of 8.65±0.74– took part in the study.

The gender distribution was: 29 boys (57%), and 22

girls (43%). They were attending the Escola d’Estiu

at the Polytechnic University of Valencia.

4.2 Procedure

The children were divided in two groups depending

GRAPP2013-InternationalConferenceonComputerGraphicsTheoryandApplications

394

on the order in which they would play with both in-

terfaces: Group A (26 children) children who used the

TacUI ﬁrst and then the TUI, and Group B (25 chil-

dren) children who used the TUI ﬁrst and then the

TacUI.

Each child tested the two types of interaction in-

dividually. In each game the time to achieve the task

was measured, and the child had to ﬁll in a question-

naire afterwards. Consequently, we used four differ-

ent questionnaires (QTc1, QTc2, QTn1, and QTn2;

Tc stands for tactile and Tn for tangible), which con-

tained the proper questions to be asked after using

each type of interaction in ﬁrst or second place. How-

ever, each child had to ﬁll in only two questionnaires,

one for each interaction.

5 RESULTS

5.1 Time Analysis

The time to complete the task of counting the objects

in the scene was measured and studied as an indepen-

dent variable. We performed a mixed-design ANOVA

analysis to take into account the two interactions the

same child used, and to consider several factors si-

multaneously.

The mixed-design ANOVA reveals that there

were signiﬁcant differences in the interaction factor

(F[1,36] = 4.94, p = 0.03), and in the order factor

(F[1,36] = 33.13, p < 0.01). Since both factors have

two levels only, a simple post-hoc analysis compar-

ing the mean of each level revealed that the users took

less time to complete the task using the TacUI than

using the TUI with a signiﬁcant difference. Similarly,

we concluded that the users took more time to com-

plete the task the ﬁrst time they played than the sec-

ond time. No signiﬁcant differences were found for

the age, gender and group factors.

5.2 Satisfaction

We analysed the results of questions 1) degree

of enjoyability, 4) degree of ease of play and 6)

global game rate. All of these questions have very

high scores: #1 4.60±0.64, #4 4.58±0.66 and #6

4.64±0.49. Moreover, there are no signiﬁcant dif-

ferences in questions 4 and 6 for the studied factors.

The only arguable point is in question 1, where the

ANOVA test shows a relatively high p-value and a

small effect size in the order factor (F[1,36] = 2.96,

p = 0.09). A post-hoc analysis reveals that the possi-

ble difference would mean that the second game was

more enjoyed than the ﬁrst game. We believe that the

difference is meaningful and not only caused partially

by chance, but also by the order factor, as we can see

from the answers to the question 3. In this question,

93% of the children said they would like to play again

after the ﬁrst game, and we could see during the eval-

uations that they were very excited to play the second

game, what in our opinion has affected the results in

question 1.

The children appreciated the AR very much, as the

high scores to a question about its appreciation reveal

for the tactile game (mean 4.80±0.40) and for the tan-

gible game (mean 4.91±0.28). It is very interesting

to note that no signiﬁcant differences were found be-

tween both games (t[46] = 1.10, p = 0.28, Cohen’s d

= 0.32), which is an excellent result because it means

that the use of AR was very similarly enjoyed with

both interactions.

5.3 Interaction

The children were asked to choose the type of interac-

tion they thought it was easier to use. When the TacUI

was the last one to be used there was a 70% of chil-

dren who preferred it to the TUI, but when the TacUI

was the last one to be played, the TUI preferences

dropped to 48%. As we can see, there is the effect that

children chose more the last game they used, which is

a common tendency that we have experienced in pre-

vious studies with children (Gonz´alez-Gancedo et al.,

2012). When the children used the TacUI in second

place, there were signiﬁcant differences in the propor-

tions of TacUI and TUI (χ

= 5.57, p = 0.02, h = 0.80)

with a large effect size. This result evinces the pref-

erence of the TacUI. The difference is, however, no

longer signiﬁcant when the TUI was used last (χ

0.01, p = 1.00, h = 0.08). From the difference in the

signiﬁcances we can conclude that the tactile screen

caused a more powerful impact on the children.

The preference for the two types of interaction

was measured in another question, where the bias to-

wards the last game is also present. The most extreme

proportions are found in the tactile game played the

second time, where the tactile game was chosen by

the 78% of the children, and the tangible game by the

22%. In such case there are signiﬁcant differences

and a very large effect size (χ

= 12.52, p < 0.01, h =

1.20).

6 CONCLUSIONS

We designed an system to compare TacUIs and TUIs

under very similar circumstances. The handheld de-

vice was used as video see-through, taking some of

TactileandTangibleInterfacesinHandheldARforChildren

395

the advantages of HMDs, but resulting in a more com-

fortable way to the children. Our proposal is very well

suited for schools, since handheld devices are afford-

able and multipurpose, what can contribute to create

richer multimedia experiences for the students.

In the experiment we evaluated, the TUI consisted

on a rotatory base on top of which the AR marker

was placed. The children could rotate the marker to

see the 3D objects from different perspectives. This

interaction was compared to the TacUI, where the ro-

tation was calculated from the horizontal movements

of the ﬁnger on screen. The results indicate that it

took signiﬁcantly less time to the children to use the

TacUI than the TUI. Since both interactions were used

with the same game in very similar circumstances, we

can strongly afﬁrm that overall, children spent signif-

icantly less time dealing with the interface in the case

of the TacUI, allowing them to focus more on the task

to do rather than manipulating the device. From this

perspective, TacUIs are better suited for applications

where the main goal is to understand and solve a prob-

lem, like in the educational ﬁeld.

As for the children’s perspective, we could see

a very high level of satisfaction with the game us-

ing the two interfaces, and no signiﬁcative differences

were found. The children were very engaged with the

game. All of them wanted to play again after using the

TacUI, and most of them after using the TUI. How-

ever, there was no evidence that the interface used in-

ﬂuenced this engagement. The AR system appealed

the children very much without being affected by the

interaction method. Thus, our visualization metaphor

is similarly engaging for children using any of the in-

terfaces.

The children seemed to prefer the TacUI mainly

for being easier and faster to use, and some of them

also thought it was more comfortable. On the con-

trary, the TUI was preferred in some cases, surpris-

ingly for very similar reasons. These children found

the screen too fast to use and too slippery (some chil-

dren tended to touch with their ﬁngernails, which does

not work), and they preferred the more stable inter-

face that the TUI was. In general, more children pre-

ferred the TacUI, although this difference was not al-

ways statistically sustainable. Despite preferring the

TacUI, the appreciation of the TUI was very high, so

we can discard that the preference of TacUI is due

only to disliking the alternative. Finally, it is very

possible that in older ages we would ﬁnd a high in-

crease on preference of the TacUI, as the children’s

physical skills improve and do not have troubles with

the screen.

ACKNOWLEDGEMENTS

This work was funded by the Spanish APRENDRA

project (TIN2009-14319-C02). For their contribu-

tions, we would like to thank:

• The people who helped in the development and

validation.

• The Summer School of the UPV.

• The children who participated in this study.

• The ETSInf for letting us use its facilities during

the testing phase.

REFERENCES

Billinghurst, M., Kato, H., and Poupyrev, I. (2001). The

magicbook: a transitional ar interface. Computers &

Graphics, 25(5):745–753.

Gonz´alez-Gancedo, S., Juan, M.-C., Segu´ı, I., Rando, N.,

and Cano, J. (2012). Towards a mixed reality learn-

ing environment in the classroom. In International

Conference on Computer Graphics Theory and Appli-

cations, GRAPP 2012, pages 434–439, Rome, Italy.

SciTePress.

Jones, B., Sodhi, R., Campbell, R., Garnett, G., and Bailey,

B. (2010). Build your world and play in it: Interacting

with surface particles on complex objects. In Mixed

and Augmented Reality (ISMAR), 2010 9th IEEE In-

ternational Symposium on, pages 165–174. IEEE.

Juan, M., Carrizo, M., Gimenez, M., and Abad, F. (2011).

Using an augmented reality game to ﬁnd matching

pairs. In Proceedings of the 19 th International

Conference on Computer Graphics, Visualization and

Computer Vision (WSCG 2011), pages 59–66.

Juan, M. C., Toffetti, G., Abad, F., and Cano, J. (2010).

Tangible cubes used as the user interface in an aug-

mented reality game for edutainment. In Proceedings

of the 2010 10th IEEE International Conference on

Advanced Learning Technologies, ICALT ’10, pages

599–603, Washington, DC, USA. IEEE Computer So-

ciety.

Sayed, N. A. E., Zayed, H. H., and Sharawy, M. I. (2011).

ARSC: Augmented reality student card. Computers &

Education, 56(4):1045 – 1061.

Wagner, D., Schmalstieg, D., and Billinghurst, M. (2006).

Handheld AR for Collaborative Edutainment. Ad-

vances in Artiﬁcial Reality and TeleExistence, pages

85–96.

GRAPP2013-InternationalConferenceonComputerGraphicsTheoryandApplications

396