Virtual Interactive Tablet to Support Vocational Training in Immersive

Environment

Pierre Gac

1,2

, Paul Richard

, Yann Papouin

, S

ebastien George

and

Emmanuelle Richard

Universit

e d’Angers, LARIS-EA 7315, 49000, Angers, France

DEC Industrie, 3 rue du Champ du Verger, 72200, ALLONNES, France

Le Mans Universit

e, LIUM-EA 4023, 72085, Le Mans, Cedex 9, France

sebastien.george@univ-lemans.fr

Keywords:

Virtual Reality, Diegetic Interface, Vocational Training, Learning, Virtual Agent, Virtual Helper.

Abstract:

This paper presents a tool designed to assist a virtual reality learner in a vocational training context. The

beneﬁts of VR are spreading outside the video-games and research ﬁeld, leading the vocational education

institutions to consider using this technology for training purposes. By simulating emblematic professional

situations, teachers can train students in good conditions regarding safety, logistics and ﬁnancial resources.

In the French vocational training system, VR is just at its beginnings and the lack of experiments with this

training context highlighted the need for new tools allowing teachers to use VR with their students. The

work presented in this paper is part of a global project aiming to create, design and assess new VR tools and

methodologies for this speciﬁc context. In order to guide, inform and assist the user, we are presenting a

generic tool that can ease VR sessions by proposing embedded tools within easy reach of the immersed user,

such as a camera, an inventory, a printer or stock management. This tool is a virtual tablet that can be grabbed

by the immersed user allowing her/him to interact efﬁciently with the virtual environment.

1 INTRODUCTION

The emergence of commercial headsets such as HTC

Vive

, Oculus RIFT

or smartphone-based solu-

tions, highlights the possibilities of using VR tech-

nology for a large public. Nowadays, nearly everyone

has heard about VR. The original target of commer-

cial VR devices are video games, that is why when

people think about VR, they are instantly picturing

someone playing games with a headset and wireless

controllers. Since the arrival of the new line of com-

mercial headsets, the global perception of VR has

been evolving leading to new opportunities of VR

usage in other ﬁelds such as education. In our re-

search projects, we are using VR devices for voca-

tional training. Using modern VR on this ﬁeld is

something relatively new, both teachers and devel-

opers suffer from the lack of perspective about VR

applied for vocational training, which implies a new

development approach in order to create an effective

learning environment.

Vocational training is a very speciﬁc ﬁeld in edu-

cation with different ways and practical applications

across the world. That is why some studies about VR

and virtual learning cannot be applied to French voca-

tional education. Since a few years, the French gov-

ernment is pushing forward digital transition in edu-

cation, and VR is now considered as a new training

support. In the case of French vocational training, we

are facing different problems that VR can pretend to

ﬁx. In some sectors, there is a huge need of expen-

sive equipment and not all high schools can claim that

equipment. Some French academies have more ﬁnan-

cial and logistical support than others, which causes

great disparities in training offers. The use of VR

for these speciﬁc situations will make it possible to

propose virtual but realistic environments to provide

access to these important and expensive equipment

and thus, allow students to practice in an immersive

way. The other related problem is that in vocational

training, students can make mistakes that can have

negative consequences on equipment or endanger the

safety of people. However, these errors can be use-

ful for the learning process. VR can be a solution,

allowing students to try without consequences in the

real world. Furthermore, the teacher can use these

mistakes to give formative feedback to students dur-

ing debrieﬁng sessions. The other motivation of using

Gac, P., Richard, P., Papouin, Y., George, S. and Richard, É.

Virtual Interactive Tablet to Support Vocational Training in Immersive Environment.

DOI: 10.5220/0007456201450152

In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019), pages 145-152

ISBN: 978-989-758-354-4

145

VR in vocational training is the safety and autonomy

of students.

Vocational training covers various ﬁelds such as

electrical maintenance or sales. While developing im-

mersive training software, the designer is tempted to

setup realistic interactions in the same way as video-

games or marketing purposes. VR can show realistic

environments and interactions, but fully imitating the

reality in VR will question the relevance of this tech-

nology for training purposes, also it is sometimes in-

teresting to add unrealistic behavior to simplify tasks

or shorten unnecessary tasks. The aim of this speciﬁc

simpliﬁcation is to attract the student’s attention over

a precise goal. Choosing relevant tasks and interac-

tions drive us to design a versatile diegetic interface

that allows to use some tools intended to simplify the

virtual tasks. During the development, we have also

taken into account the diversity of vocational train-

ing situations. In this paper we will expose our tool

in the speciﬁc context of the French vocational train-

ing system. Our research projects are focused on VR

design and pedagogical relevance of VR applied to

vocational training in order to allow teachers to use

them easily in their learning programs.

The contribution of this paper is to propose a vir-

tual interactive tablet that is designed to help and as-

sist students to accomplish the virtual tasks. To aim

for this, we developed a virtual tablet that can be

grabbed by the user in the virtual world and use it

as a toolbox or as a helper. We will ﬁrst make a re-

view of the literature related to Pedagogical Virtual

Agents (PVA) and 3D user interfaces (UI). This re-

view of the literature is meant to explain the link be-

tween a PVA and UI. Then, we will add more details

about our proposal and the context of this work. We

have conducted some preliminary tests and the results

will be exposed in a speciﬁc section. Finally, we will

conclude with the ongoing experiments that we are

planning to test our tool as well as other possible iter-

ations of the virtual tablet.

2 RELATED WORK

2.1 Pedagogical Virtual Agents

In VR, we sometimes need to guide the user through

the environment and goals. To do so, several tech-

niques were created such as PVA. Those embodied

agents are virtual avatars which can have multiple

functions or representation. The ﬁrst iterations of

PVA highlighted the need of a helper that can guide

and also show manipulation examples by its interac-

tion capabilities to the user. The virtual agent will

react to user’s actions and provide speciﬁc assistance.

Such interaction is a good vector for student motiva-

tion thanks to customized advices. An agent persona

can be described using the four-factor model as de-

tailed in Ryu and Baylor (2005) (facilitating, engag-

ing, credible and human-like). In Jin (2010), students

found the VE with PVA more entertaining and edu-

cational than the one without it. Besides reacting to

interactions (object manipulation, user’s action or en-

vironment events), virtual agents can handle conver-

sational capabilities as described in the MAX frame-

work (Kopp et al., 2003; Lopez et al., 2014). Their

solutions can react to user’s questions or afﬁrmations.

This kind of conversational assistant is now present in

today’s life with assistants like Siri, Google Home or

Alexa. It is now natural for someone to give a speech

to a computer.

With the emergence of machine learning and AI,

PVA can now mimic a more realistic behavior and can

follow the student’s progression. To do so, agents

have to react to speciﬁc situations with intelligence

and adequate behavior as described by Mittal et al.

(2013) to have a believable behavior. To build an efﬁ-

cient and credible behavior of the agent, the agent sys-

tem needs to have access to all world/situation knowl-

edge (omni-data) but it has to limit its access to this

omni-data otherwise it will result in unrealistic behav-

ior (Mittal et al., 2013). It is also possible to make the

agent aware of the real world situation, thanks to new

sensory devices, given that, a real world-aware agent

can adapt its speech, gesture, or facial expression ac-

cording to the user in real-time (Wuttke et al., 2016).

In addition, decision making is a key element to the

realistic behavior of the PVA as stated Lopez et al.

(2014). VR is a conducive mean to collaboration,

this could be achieved with multiple humans or one

human and several agents. In their HUMANS suite,

Lourdeaux et al. (2017) have implemented both indi-

vidual and collaborative capabilities to their agents.

2.2 3D User Interface

In classic 2D games, the user interface (UI) area is

displayed on the camera view (overlay mode), while

in VR, we can setup a UI that is present in the 3D

world. Our work is focusing on those 3D menus that

use widgets and classic UI. Voice and gestures UI

are not compatible with our work context (vocational

training). For voice command UI, we faced three is-

sues that drove us to drop this approach. First, the

class organization around the VR system can gener-

ate noise, making the system unable to correctly inter-

pret oral commands. Secondly, we want to take into

account the speaking manners of some students (e.g.

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

146

strong accents, or language disorders). The third point

is about technical limitations such as programming

and logistics (e.g. inputs or Internet connection). For

the gesture UI, we decided to not consider this manner

of interaction because it will involve more develop-

ment time than the ”classic” UI as well as for the user

having to memorize the gesture patterns. Many types

of 3D UI selection metaphors have been developed for

VR applications. For instance, Bowman et al. (2001)

proposed a UI system that is controlled by the user’s

real hands, using gloves. Before commercial devices

like HTC Vive

, 3D UI interaction were made with

several unique devices. They also used a tablet-based

interaction that allows the user to select a virtual ﬂoat-

ing menu item by using a real tablet. Those interac-

tions are not efﬁcient with today’s tools, for example,

with the HTC Vive

, the user has an efﬁcient track-

ing of its hands in the virtual world allowing speciﬁc

interactions. This allows the user to easily interact

with 3D UI with some feedback such as the vibra-

tions of controllers, sounds or visuals.

In vocational VR training, we are looking for

pseudo realistic situations where the learner can be

projected in order to ﬁnd a link between the virtual

experience and the real one. Sometimes in VR we

need to display textual information, the problem for

the designer is to choose the appropriate metaphor

for displaying text. The diegesis theory is initially

a literature concept, that has been applied to video

games and thus, to VR by Galloway (2006). Finally,

the VR designer can use four kinds of UI: Diegetic,

non-diegetic, spatial or meta as described in Fager-

holt and Lorentzon (2009). The diegetic interfaces

are strongly integrated in the virtual world (Figure1).

They proposed the classiﬁcation regarding 2 condi-

tions: ”Is the representation visualized in the 3D game

space?” and ”Is the representation existing in the ﬁc-

tional game world?”. The answers are crossed into

an array. In a sci-ﬁ universe, a diegetic UI can be an

holographic representation of the menu. In FPS, it

can be the way ammo are displayed (Peacocke et al.,

2016) or health displays. The majority of studies

about the effectiveness of diegetic UI are empirical,

and have been tested by some videogame studios.

A positive aspect of diegetic interfaces, is that they

tend to improve feelings of immersion (Raffaele et al.,

2017). In their work, Raffaele et al. (2017), inves-

tigated the relations between immersion and virtual

interfaces through VR. Their results indicate that the

user will receive feedback from the UI without being

conscious about it and not noticing how feedback is

received. For 3D UI, not noticing the feedback source

is pointed out as the key to keeping the immersion

level high. Consequently, in our training context, it

Figure 1: Example of a diegetic interface in the game Fall-

out4.

can beneﬁt the learner. Therefore, the design of such

UI is meaningful in order to not confuse the ﬁnal user.

They conclude by saying that Spatial and Diegetic UI

are the most immersive types of UI. To achieve a cor-

rect immersion level, unless considering the quality of

the environment, VR designers have to favor Spatial

or Diegetic UI to keep immersion at the top most lev-

els. Feelings of immersion are sometimes disrupted

by the controllers complexity (Fragoso, 2014). In the

case of external UI, the feelings of immersion do not

vanish if the UI maintains the aesthetic perception

(Fragoso, 2014).

Llanos and Jørgensen (2011) suggests that a min-

imalist interface is easier to manipulate and so, pro-

vides better information as it is easier to access and

process. Participants said that minimalism is an ”el-

egant way of representing system information”. To

consolidate the importance of minimalist UI, Gerber

and Bechmann (2004) noticed that the average selec-

tion time grows with the number of items displayed.

It should point out that today’s standards of UI in

smartphones or computers are moving forward to the

minimalist approach as it is considered more ”user-

friendly”. Llanos and Jørgensen (2011) add in their

conclusion that users prefer to receive only relevant

information instead of a huge quantity of data. Ad-

ditional information that is not necessary will be tire-

some and thus, will somehow break the immersion

and force the learner to focus more on the UI. The

precision of the data displayed is a sensitive point for

VR designers.

3 CONTEXT OF THIS WORK

This work is part of a large-scale project that consists

of creating VR applications for vocational training.

The French Education Academy of Nantes is the pi-

lot academy in VR experiment and deployment. We

are working with teachers and inspectors on the next

iteration of digital training using VR. The skill-based

Virtual Interactive Tablet to Support Vocational Training in Immersive Environment

147

approach implies new design processes regarding sce-

nario writing, software development and evaluation.

There is a signiﬁcant need of ”useful” emblematic

training situations that can be used by the teacher and

professionals.

This project is divided in three main parts. The

ﬁrst part is the scenario design methodology and ”in

class” use of VR as well as sharing our results and

ﬁndings to the teachers in a way to improve the use

of VR in an Educational context. Existing method-

ologies presented in the VR literature cannot directly

be applied to our design approach. These methodolo-

gies are arduous to understand for teachers and pro-

fessionals. The second part relates to the VR develop-

ment, this is about creating a SDK that allows the user

to create a VR training scenario from scratch with

Unity3D without the need of consequent program-

ming skills. The targeted user can be a conﬁrmed

developer or a teacher. The third part deals with stu-

dents’ assessment by the VE and by the teacher. Voca-

tional training in France has its own speciﬁcities that

we need to take into account to build efﬁcient training

situations. Didactic knowledge is the key to achieve a

good implementation of VR training scenario.

4 VIRTUAL AGENTS FOR

VOCATIONAL TRAINING

In vocational training, users need to be guided while

practicing. Students require additional information

and help to achieve tasks they are not mastering.

While progressing and practicing, learners will gain

autonomy and will need less external help to com-

plete tasks. Using VR will not decrease this need of

help, instead, students will need more help to master

the tools and scenarios. That is why, when composing

a VR training application, designers should include a

virtual assistant system, embodied or not.

In our project, we ﬁrst considered using embod-

ied PVA as described in the literature for the beneﬁts

it provides (Kosinowski, 2009). Nevertheless, in vo-

cational training some situations are not compatible

with the presence of PVA. For electrical maintenance

or production line management, it is uncommon but

sometimes dangerous to have someone near the repair

task. The area of work can be too small, or standard

tasks do not allow more than one person in the work

area (e.g. scaffold inspection, electrical clearance).

In that case, the PVA should be outside the activity

space, which implies the user to navigate from and to

the work area to interact with the PVA. The type of

agent for vocational training purposes could be an in-

structor as described by Sklar (2003). The problem

with instructors is that over-guiding the user can ”get

in the way of the user’s learning”.

The other concern about PVA for us, is that vir-

tual agents will require huge development time and

resources to achieve realistic behavior (e.g. auton-

omy, interactions, intelligence) as described in the lit-

erature. Moreover, developing a PVA requires a sig-

niﬁcant amount of time for reﬁnement (Barthes et al.,

2018). The user need to trust the agent and to achieve

that reliability is not enough, attractiveness is just as

important as reliability (Yuksel et al., 2017). There-

fore, implementing trusting agent will cause more de-

sign work. A second-rate development like poor ani-

mations, poor facial expression or low-quality sounds

can lead to an awful user experience and poor attrac-

tiveness. We are using virtual agents only for simple

dialogs like ”Take this document” or ”You need to do

those tasks”. These are presented as a dialog popup

near the agent’s avatar and also played with sound

speech. This is a requirement for the students who

have difﬁculties at reading and hearing. The consid-

eration of disabilities such as hearing impairment or

difﬁculty of reading is not always done in VR soft-

ware. In addition of this easy use of PVA, we still

need a tool that improves the users’ autonomy and al-

lows them to get help or advice when practicing in

VE. The idea of an interface that encapsulates all of

these additional functionalities came through.

5 THE VIRTUAL TABLET

Figure 2: The home menu of the tablet.

The constraints encountered while creating the VR

training application drives us to consider new ways of

guiding and assisting the user. This idea came from

the observation of some virtual tasks. Some tasks

like printing a document have low pedagogical inter-

est and cost a huge amount of time. Indeed, the user

need to go to the printer, print the document and then

return to the right location to continue the task. A

ﬂoating 3D UI would appear above the printer. The

learner had to select the document to print and then

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

148

click on a ”print” button. After doing that, they had

to go back to the activity area. This time-consuming

process was inefﬁcient because if the student had for-

gotten something or printed the wrong document, s/he

had to navigate back to the printer area,while wasting

time traveling. By the end of the scenario, the student

will have been disinterested of navigating through the

VE. In vocational training, these kinds of errors are

very common, because students tend to forget or not

read the activity instructions.

Another problem encountered was the document

reviewing in VR. The naive approach was to copy the

reality into the virtual. Teachers initially thought that

if they copy reality inside the virtual world, the stu-

dent will be able to perform tasks the same way as in

the real world. They did not consider the interaction

process/tools, development complexity and the user’s

experience. They asked to have directions, plans, or

documents printed on virtual A4 paper displayed in-

side the virtual world. They were conﬁdent over the

results, but after testing this, they noticed that the doc-

uments were unreadable and required a lot of atten-

tion and focus to be fully understandable. This led

us to propose another way to visualize documents,

by removing the virtual A4 paper for the beneﬁt of

a minimalist 3D UI with only necessary information

displayed with better font and contrast.

VR in vocational training is not meant to mimic

the reality at 100%, because in that case, why would

VR be useful? The good exploitation of VR is a re-

quirement to build adequate training situations. To do

so, we need the following elements:

• Access to directions at anytime

• Simple interactions (to understand and to use)

• A solution that can be used in several training sit-

uations (generic-programming)

• Simplify real interactions by removing irrelevant

tasks/steps

• Give/Gather information from/to the user

• Display warning or directions at a speciﬁc time

when needed

• Guide the user through the scenario

With all of those elements, and considering the

case of PVA, we came up with the idea of the tablet

metaphor solution. The concept is to build a generic

tool that encapsulates all of those elements and more.

The interesting point of the tablet metaphor is the

equivalence with real emblematic situations. In a lot

of working ﬁelds, workers are using tablets, smart-

phones or PDA in their tasks (e.g. sales, electrical

maintenance, logistics, building workers). That is

why using a virtual tablet as a model for the interface

is relevant because of the similarities of the virtual

device to the real world use of devices even when our

virtual content does not exactly match the real use of

those tools.

This device is held by the user using one of her/his

hands and is considered by our SDK as a passive de-

vice. It means that the virtual hand containing the

tablet can not interact with the world by raycasting

scene objects. The student can switch the hand hold-

ing the tablet by simply clicking on the tablet with the

other hand. The aim of locking the tablet on one hand

is to allow the student to access it anytime and avoid

losing the tool somewhere in the VE. The purpose of

VR training situation is not to search for a tool or to

wander in the VR. While testing our VR solutions,

we noticed that wandering can lead the student to not

fully understand what teachers expect from her/him

or can feel that they are lost in the scenario. With the

tablet, we can mostly eliminate all potential wander-

ing situations.

For the document printing process, we achieved to

eliminate the unnecessary navigation part, the print-

ing station interactions and reduced possible errors

that can be made by the user because she/he can print

documents near the activity area. To achieve all that,

we put this process inside the tablet (see Figure4). An

application called ”Printer” was created, in which the

user can select a document from a category and print

it. As Fragoso (2014) established, clarity and func-

tionality are more important than integration in the

gameworld, so we decided to make a simple inter-

face for this tablet application. The user can select

a document by clicking on an image and then click on

”print”. The printed document will appear at the top

of the tablet and the student just have to select it like

she/he would do for any virtual object. This part is not

realistic, but it does not negatively impact the training

process, as the methodology behind document print-

ing is the same regardless of the interaction. All of

this process is achieved without any unnecessary nav-

igation in the VR world. We use the same approach

for other interactions such as the document reading

process, the inventory or directions.

We implemented a ”home” screen (Figure2)

where the user can start an application by simply

clicking on the corresponding icon. We also chose

to include applications that can be used anytime in

any training situation, such as the camera that allows

the user to take pictures of the game world. The im-

ages taken can be retrieved by the user after the vir-

tual sessions. This ”simple” tablet application has a

huge training interest as teachers can ask students to

capture things that they judge ”interesting”. The cap-

tured images can be used during a debrieﬁng step or

Virtual Interactive Tablet to Support Vocational Training in Immersive Environment

149

in other courses like written expression. In the case

of a shop safety inspection, teachers ask the students

to explain why they took this photo by writing down

their analysis process.

One advantage of VR is data gathering (traces).

We are currently using the Bowman et al. (2001) clas-

siﬁcation to sort out the interactions made by the user:

manipulation, selection, navigation and system con-

trol. The tablet is too advanced to be considered as a

simple manipulation or selection. The system con-

trol is the category that is mostly compatible with

the tablet. Bowman deﬁnes the system control as ”a

task in which a command is applied to change either

the state of the system or the mode of interaction”.

The tablet behaves in a similar way as described by

Bowman, but some applications cannot be included

in the system control. For example, the camera ap-

plication does not change the mode of interaction or

the state of the system. It is just a selection in this

case, on the contrary to the inventory application that

will change the mode of interaction when selecting an

object. Thus, the tablet includes all types of interac-

tions, and a deeper investigation is needed to ﬁnd a

correct categorization of the device. A potential cat-

egory may be ”Functionality selection” as the tablet

allows the user to select several features.

Figure 3: The camera application.

6 PRELIMINARY EXPERIMENTS

To prepare our system for the planned experiments,

we have conducted two preliminary tests using the

HTC VIVE

. The chosen activity was an order

preparation in a virtual shop designed for vocational

training. The task completion is between 15-30 min-

utes. This scenario include fetching the products,

storing them inside boxes and prepare the command

for departure. This emblematic situation involves

many different interactions, especially on the virtual

tablet.

6.1 Premilinary First Experiment

The ﬁrst experiment, involved high-school students (6

participants) in sales training. During those sessions,

teachers and us gathered subjective feelings of stu-

dents regarding the virtual situation and the tablet. In

addition to our research purposes, teachers wanted to

assess if VR can be an interesting tool for several stu-

dent’s proﬁles. One noticeable observation we made

is that users quickly master the tablet interface and

the purpose of each application. According to stu-

dents, it is thanks to the ”close design” of the tablet

to real devices like Android systems. Navigation on

the tablet seems to be ”natural” thanks to explicit im-

ages, concise labels and good UI contrasts. We also

asked the question ”Is the tablet useful in this sce-

nario ?”, to what participants answered ”the tablet is

useful because we can quickly access stuff, like the

printer or the stock screen, without loosing time do-

ing like in the real world”. ”With the tablet, we are

still focused to what we are doing and not tempted

to wander and play like in a video game”. One stu-

dent also added that if the scenario had transposed the

real world into the virtual (without the tablet), then

”the situation would be annoying and so does VR for

training”. Users agreed that the tablet is an efﬁcient

assistant and grant them autonomy in the virtual sce-

nario. Besides those positive feedbacks, we noticed

that users sometimes strangely handle the tablet, by

keeping it in front of their view instead of lowering the

device to clear the view. They did not notice this until

we say ”You do not need to tablet now, you can lower

your arm if you want”. We have also observed issues

with some UI design regarding the size of some but-

tons and labels. Two users reported that handling the

tablet on one hand can be exhausting over the time,

that is why we may need to consider new interaction

methods, different than the actual raycasting, for long

VR activity (more than 20min). We observed that the

raycasting is sometimes hard to achieve especially on

the top panel, because this panel is too small. Accord-

ing to teachers, the virtual tablet is interesting because

”It is similar to some work situations where workers

use technology to ease their jobs”. This interest is

also present in the student point of view.

6.2 Preliminary Second Experiment

The second experiment was conducted with adults (7

participants). The main purpose was to gather data

from the application (completion time, amount of cor-

rect/miss clicks, name of clicked objects, current ap-

plication) as well as with a questionnaire. We used

a French translated version of the System Usability

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

150

(a) Document selection (b) Waiting for the user to take the printed document

Figure 4: Document printing process with the virtual tablet.

Scale (SUS) (Brooke and Weerdmeester, 1996) and

replaced the word ”system” by ”tablet”. The mean

score is 79 (σ=13). From traces gathered, we noticed

that users perform a small amount of miss clicks on

the tablet. A miss click occurs when the user is click-

ing on a non interactive component like label or im-

age instead of a button. For an average of 111 (σ=32)

clicks on the tablet per session, users performed 2.8%

miss clicks (σ=3%). For this second test, we have

changed some fonts on the tablet, because we no-

ticed in the ﬁrst test sessions that users with difﬁ-

culty of reading had trouble to read instructions on

the tablet because of the font we initially choose (Os-

wald Regular). We tried a dyslexic friendly font

(OpenDyslexic), resulting in poor results in the CAVE

and headsets, mainly because of the resolution. After-

wards, we chose the Verdana font for the second test

and users do not pointed out any text issues. Data

showed that the tablet is not a source of difﬁculty, and

user’s proﬁle does not impact tablet usage. Further-

more, objective data retrieved from the tablet cannot

be used alone to deﬁne user’s proﬁle and categorize

it.

7 DISCUSSION

The usability of the tool relies on the UI clarity and

design, we chose a minimalist approach with only rel-

evant information, to not overload the student with

much many information. The two ﬁrst iterations faced

major design issues as well as generic programming

problems. We are still facing a major issue regarding

the degree of realism to include in course of the ped-

agogical virtual scenario. Indeed, we noticed that a

minimalist design is more welcomed by students from

initial formation than by teachers or older students.

Those young students appear to be more comfortable

in the VE than teachers. Teachers want to display as

much information as possible without considering the

readability problems in VR. We tried this approach

and the result is that students were lost because of the

overwhelming quantity of information. With the min-

imalist UI, we need to train and convince the teachers

that we can not display a large amount of information

because it will have a negative impact over students’

performance. The initial wish of teachers is to take

all the information from the real world and transpose

it into the virtual. As explained earlier, this is a bad

design choice and will cause bad results.

The tablet is held by one of the user’s hand, so the

hand containing the device can not be used to perform

other interactions. The user will be able to interact

with only one hand. In some training situations, it is

an issue. For example, a voltage tester will require

two hands to operate. So, the user has to place down

the tablet somewhere, perform the action and then

grab the tablet again. We are investigating another

metaphor, instead of having the tablet on one hand,

the device will be placed on the arm of the user, al-

lowing her/him to interact with their two hands. This

new approach is also facing some issues. The ﬁrst

issue is the size of the screen, and for it to be credi-

ble we need to reduce its size. Downsizing can have

a negative impact over the readability of the device

and thus, its usability. Another issue is the interaction

metaphor used to interact with the arm-based tablet.

We are actually using a raycast from the hand that is

not containing the tablet to do the selection process.

With an armed-based solution, performing a raycast is

practically complicated and requires the user to con-

tort itself to select an item on the arm-based tablet.

For this case, other metaphors are considered, like a

hybrid grab: the interaction is active when the interac-

tive object is within a certain range around the virtual

hand.

Virtual Interactive Tablet to Support Vocational Training in Immersive Environment

151

8 CONCLUSION AND FUTURE

WORK

This paper presents the design of a training-oriented

tool that allows the user to manage several interac-

tions with only one tool and without involving a com-

plex interaction process to achieve the desired actions.

To assess our design choices, we are planning ex-

perimentations with teachers and students. We are

interested in student’s performances with or without

the tablet as well as their feelings in the training en-

vironment. Some non-realistic interactions from the

tablet will have to be implemented in a realistic way

in the VE (like the printing station described earlier).

In addition to the immersive data gathered we will

ask teachers to assess student performances with the

data from the tablet and with data from a situation

without the tablet. We also want to experiment other

metaphors such as the arm-based solution previously

described.

REFERENCES

Barthes, J., Wanderley, G., Lacaze-Labadie, R., and Lour-

deaux, D. (2018). Designing Training Virtual Envi-

ronments Supported by Cognitive Agents. IEEE 22nd

International Conference on Computer Supported Co-

operative Work in Design, pages 307–312.

Bowman, D. A., Kruijff, E., LaViola, J. J., and Poupyrev,

I. (2001). An introduction to 3-D user interface de-

sign. Presence: Teleoperators and Virtual Environ-

ments, 10(1):96–108.

Brooke, J. and Weerdmeester, A. (1996). SUS-A quick

and dirty usability scale. Usability evaluation in indus-

try, B. Thomas and. Usability evaluation in industry,

pages 189–194 SRC – GoogleScholar FG – 0.

Fagerholt, E. and Lorentzon, M. (2009). Beyond the HUD.

User Interfaces for Increased Player Immersion in FPS

Games. Chalmers University, page 124.

Fragoso, S. (2014). Interface design strategies and disrup-

tions of gameplay: Notes from a qualitative study with

ﬁrst-person gamers. In Lecture Notes in Computer

Science (including subseries Lecture Notes in Artiﬁ-

cial Intelligence and Lecture Notes in Bioinformatics),

volume 8512 LNCS, pages 593–603.

Galloway, A. R. (2006). Gaming: Essays on algorithmic

culture, volume 18. U of Minnesota Press.

Gerber, D. and Bechmann, D. (2004). Design and evalua-

tion of the ring menu in virtual environments. Immer-

sive projection technologies.

Jin, S. A. (2010). The effects of incorporating a virtual agent

in a computer-aided test designed for stress manage-

ment education: The mediating role of enjoyment.

Computers in Human Behavior, 26(3):443–451.

Kopp, S., Jung, B., Leßmann, N., and Wachsmuth, I.

(2003). Max A Multimodal Assistant in Virtual Re-

ality Construction. K

unstliche Intelligenz, 4:11–17.

Kosinowski, H. (2009). Pedagogical Virtual Agents.

Llanos, S. and Jørgensen, K. (2011). Do Players Prefer Inte-

grated User Interfaces? A Qualitative Study of Game

UI Design Issues. DiGRA 2011 Conference: Think

Design Play, pages 1–12.

Lopez, T., Chevaillier, P., Evrard, P., Barange, M., Berth-

elot, R. B., Arnaldi, B., Lopez, T., Chevaillier, P.,

Evrard, P., and Nouviale, F. (2014). Communicative

Autonomous Agents To cite this version : Collabora-

tive Virtual Training with Physical and Communica-

tive Autonomous Agents .

Lourdeaux, D., Benabbou, A., Huguet, L., and Lacaze-

Labadie, R. (2017). HUMANS : suite logicielle

pour la sc

enarisation d ’ environnements virtuels pour

la formation

a des situations socio-techniques com-

plexes. Conf

erence Nationale sur les Applications

Pratiques de l’Intelligence Artiﬁcielle.

Mittal, S., Doyle, M. J., and Watz, E. (2013). Detecting in-

telligent agent behavior with environment abstraction

in complex air combat systems. SysCon 2013 - 7th

Annual IEEE International Systems Conference, Pro-

ceedings, (February):662–670.

Peacocke, M., Teather, R. J., Carette, J., and MacKenzie,

I. S. (2016). Evaluating the effectiveness of HUDs and

diegetic ammo displays in ﬁrst-person shooter games.

2015 IEEE Games Entertainment Media Conference,

GEM 2015, (October).

Raffaele, R. C., de Carvalho, B. J. A., and Silva, F. G. M.

(2017). Evaluation of immersive user interfaces in vir-

tual reality ﬁrst person games. Proceedings of EPCGI

2017, (October):123—-126.

Ryu, J. and Baylor, A. L. (2005). The Psychometric Struc-

ture of Pedagogical Agent Persona. Technology In-

struction Cognition and Learning, 2(4):291–314.

Sklar, E. (2003). Agents for Education: When too Much

Intelligence is a Bad Thing. Proceedings of the Inter-

antional Conference on Autonomous Agents, 2:1118–

1119.

Wuttke, M., Heidt, M., Rosenthal, P., Ohler, P., and M

uller,

N. H. (2016). Proactive functions of a pedagogical

agent-steps for implementing a social catalyst func-

tion. Lecture Notes in Computer Science (including

subseries Lecture Notes in Artiﬁcial Intelligence and

Lecture Notes in Bioinformatics), 9753:573–580.

Yuksel, B. F., Collisson, P., and Czerwinski, M. (2017).

Brains or Beauty: How to Engender Trust in User-

Agent Interactions. ACM Transactions on Internet

Technology (TOIT), 17(1):1–20.

HUCAPP 2019 - 3rd International Conference on Human Computer Interaction Theory and Applications

152