Augmented Reality Interface Design for Autonomous Driving

Raissa Pokam Meguia

, Christine Chauvin

and Serge Debernard

IRT SystemX, Renault, Paris, France

Lab-STICC UMR CNRS 6285, Université Bretagne Sud, Morbihan, France

LAMIH UMR CNRS 8201, Université de Valenciennes, Famars, France

1 RESEARCH PROBLEM

During the last decades, the driving paradigm has

been changed due to the introduction of the

automation. Indeed, from a configuration where the

driver was the sole responsible of the driving, we

move to a bilateral configuration where the human

agent and the technical agent share the driving task.

This bilateral configuration can be observed in

vehicles called autonomous ones. Considering the

number of tasks delegated to the technical agent, there

are several levels of automation (NHTSA, 2013). The

“full automation” level means that the technical agent

is the sole responsible of the driving. It will be

observable in the horizon 2030. Many vehicles

manufacturers and laboratories work currently on the

intermediate levels of automation.

The LRA (French Acronym for Localization and

Augmented Reality) project is a collaborative French

project where 10 academics and industrials work

together to deliverer an Augmented Reality (AR)

Human Machine Interface (HMI) of an autonomous

vehicle of level 3, in the National Highway Traffic

and Safety Administration (NHTSA) taxonomy. This

level supposes that the vehicle is able to drive alone

in some particular conditions. During this period

where the technical agent is in charge of the driving

(also called free time), the human agent can do some

secondary or tertiary tasks such as reading, writing,

etc. The human agent is, in these periods,

disconnected from the road environment and the

primary task of driving. He is out-of-the-loop, and it

is necessary to take into account two main situations.

In the first one, at particular moments, he may want

to verify how the technical agent works in order to

know what the technical is doing and if its behavior

is accurate. In the second one, at a precise time,

handover will be required by the system.

Consequently, the human agent has to be reengaged

physically and cognitively in the driving task. He has

to build a mental representation of what is going on

around him in the road environment and what the

technical agent is doing, in order to have a whole

situation understanding of the situation. This

task-specific understanding of the situation refers to

situation awareness (Endsley, 1985). Through

interface design, situation awareness has to be

enhanced. In order to achieve this goal, we decided to

use in information shaping, Augmented Reality, an

innovative technology.

AR typically describes interfaces that overlay

images of virtual objects on images of the real world.

We can talk also about Augmented Reality for haptic,

auditory or vestibular cues. In this research, we have

decided to focus on visual cues.

There are many definitions of Augmented Reality

but we have decided to choose Azuma (1997) one. He

defined Augmented Reality as any interface that has

these three characteristics:

1. combines the real and the virtual.

2. is interactive in real time.

3. is registered in 3d.

Consequently, considering the interface design in

autonomous mode, we have identified three

fundamental questions to design the interface:

a. In autonomous mode and in handover processing,

which sufficient representation should the drivers

maintain or establish? According to the Situation

Awareness model defined by Endsley (1995), this

question may be subdivided into three sub-questions:

(i) What should the drivers perceive?

(ii) What should they understand?

(iii) Which projection of the external environment

and the system should they perform?

b. How should we design the displays?

(ii) What should be displayed?

(iii) How should that information be displayed?

(iv) When should it be displayed?

(v) With which prioritization?

c. What is the added value of Augmented Reality in

the displays?

Pokam Meguia R., Chauvin C. and Debernard S..

Augmented Reality Interface Design for Autonomous Driving.

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

2 OUTLINE OF OBJECTIVES

The general objective is the design of rules for

Human-Machine Interface. Through the development

of a specific methodology correlated to an innovative

technology, Augmented Reality, we will design an

adaptive interface which will ensure a safe and a

comfortable driving in an automated vehicle of level

In order to meet the challenges outlined above, the

LRA project integrates multidisciplinary expertise

from French research institutions and vehicle

industrials. The involvement of the industrials creates

the potentiality of a vulgarisation of the finished

work. We have defined two main goals:

1) Application of a cognitive method to derive

information requirements for the driver and

hierarchize them through strong rules.

2) Conveyance all these information in an

appropriate shape by considering displays

capabilities.

To achieve these goals, we have realized a literature

review at the beginning.

3 STATE OF THE ART

This chapter contains the main theoretical concepts

that underlie our research work: driving task,

automation, situation awareness, transparency, and

Augmented Reality.

3.1 Driving Task

Michon (1985) has defined a hierarchical control

structure of the driving task. This structure divided

the driving task into three levels of control: strategic

level, tactical level and operational level. See Figure

1. Each level has its importance in the driving

task.

In the strategic level, we deal with the route that

the ego vehicle intends to follow. The route is planned

and the general objectives are fixed. In the tactical

level, we talk about the maneuvers: passing a vehicle,

entering highway, exiting highway, lane changing,

overtaking. This level concerns all the maneuvers

Figure 1: Three levels of control in driving.

which allow to achieve short-terms maneuvers. In the last

level, the longitudinal and the lateral controls are

concerned.

3.2 Automation

When the driver is responsible of the driving task, he

has to lead his vehicle in each of the aforementioned

levels of control. More often, we observe a delegation

of one or more of functions realized by the human to

one or several technical agents. Considering the

number of functions delegated to the technical agent,

we have many levels of automation (Sheridan, 1978).

There are many taxonomies that try to classify those

levels.

The most famous are: National Highway Traffic

and Safety Administration (NHTSA, 2013)

taxonomy, Society of Automotive Engineers (SAE)

taxonomy, Sheridan and Verplanck (1978)

taxonomy, Endsley and Kaber (1999) taxonomy,

Gasser and Westhoff (2012) taxonomy and Riley

(1989) taxonomy. As a LRA project constraint,

NHTSA taxonomy was chosen but it has the

disadvantage to not clearly define the functions

attributed to each agent. As mentioned above, we

work on an automated vehicle of level 3. As NHTSA

specify, vehicles at this level of automation enable the

driver to cede full control of all safety-critical

functions under certain traffic or environmental

conditions and in those conditions to rely heavily on

the vehicle to monitor for changes in those conditions

requiring transition back to driver control. The driver

is expected to be available for occasional control, but

with sufficiently comfortable transition time. The

vehicle is designed to ensure safe operation during

the automated driving mode. An example would be an

automated or self-driving car that can determine

when the system is no longer able to support

automation, such as from an oncoming construction

area, and then signals to the driver to reengage in the

driving task, providing the driver with an appropriate

amount of transition time to safely regain manual

control (Marinik Bishop, Fitchett, Morgan, J. F.,

Trimble & Blanco, 2014).

This kind of interaction and others ones introduce

Situation Awareness concept.

3.3 Situation Awareness

Situation Awareness is a term derived initially from

the aviation domain. In this domain, it plays a crucial

role in the design of military interface. There are

numerous definitions of Situation Awareness but the

most used and the widely accepted is the one from

AugmentedRealityInterfaceDesignforAutonomousDriving

Endsley (1995) defining it as the perception of

elements in the environment within a volume of time

and space, the comprehension of their meaning, and

the projection of their status in the near future.

The important information has to be conveyed to

the driver appropriately and accurately. We assume

that not all the information has to be conveyed to the

driver. This introduces the transparency term.

3.4 Transparency

The concept of transparency can be seen in two points

of view. Transparency in a form aspect, refers to the

level of opacity of one particular component. There is

also an aspect that deals with the quantity of

information. Considering human-automation,

transparency cannot, practically speaking, means that

the human knows everything about what the

automation is doing (Miller, 2014).This definition is

a naïve one. Chen et al. suggested another definition

for automation transparency: … the descriptive

quality of an interface pertaining to its abilities to

afford an operator’s comprehension about an

intelligent agent’s intent, performance, future plans

and reasoning process.

Transparency could take advantage of innovative

technology, such as Augmented Reality.

3.5 Augmented Reality

Many modern cars (e.g. Audi Q7, BMW M3 Berline)

are equipped with Head-Up Display (HUD)

technology. This technology enables Augmented

Reality (AR) implementation (Tonnis, Sandor,

Lange, & Bubb, 2005). Usually, AR is defined as a

continuum from real to Virtual Reality (Milgram,

1994). Generally, AR in cars deals with “the problem

of directing a user’s attention to a point of interest

(Tonnis et al., 2005). AR can “alert drivers and guide

their attention to dangerous situations” (Tonnis et al.,

2005). We thus assume that AR can enhance global

awareness and local guidance by conveying the right

information at the right moment.

4 METHODOLOGY

The figure 2 presents, step by step, the general

approach employed for the design and development

work we conduct. This work involves the

specification and design of the interface.

4.1 First Step of the Methodology:

Cognitive Work Analysis

This step involves modeling tasks and extracting

information requirements for drivers. We decided to

use a method that considers both technical and human

aspects together: Cognitive Work Analysis (CWA)

(Rasmussen, 1990). It is an integrated framework that

defines the work demands of complex sociotechnical

systems in terms of the constraints on actors

(Rasmussen, 1986; Rasmussen, Pejtersen, &

Goodstein, 1994; Vicente, 1999; Naikar, 2013).

CWA is an Ecological Interface Design (EID) -based

approach. In EID approach, the constraints of the

system are enhanced in order to allow drivers to take

effective actions and to know the impact on their

actions in their goals achievement (Burns and

Hajdukiewicz, 2004 cited by Salmon, Regan, Lenné,

Stanton and Young, 2006).

CWA also provides information regarding the

different possibilities for actions inside the system.

Figure 2: HMI design methodology.

ICINCO2015-DoctoralConsortium

This method consists of several phases of analysis:

work domain analysis, task analysis, strategies

analysis, organizational analysis, and skills analysis.

4.1.1 Work Domain Analysis (WDA)

It is the first phase of Cognitive Work Analysis. It

focuses on analyzing the boundary conditions or

constraints of a work system. Through an Abstraction

Hierarchy or an Abstraction Decomposition System,

the structural functions of the system are determined.

4.1.2 Control Task Analysis (ConTA)

The control task analysis identifies the activity that is

required in a work domain (Naikar, 2013). After the

definition of the constraints associated with the

environment, this dimension of the CWA focuses on

determining the constraints associated with what

needs to be done in the system (Naikar, 2013). In this

phase, two tools are usually used: the contextual

activity template for work situations and functions

modeling, and decision ladder template for control

tasks modeling (Naikar, 2013).

4.1.3 Strategies Analysis (StrA)

This phase identifies how the activity can be carried

out (Naikar, 2013). Strategies analysis deals with the

constraints associated with all the possible ways to

realize an activity. In fact, it is possible to have many

strategies for a single activity. To represent

graphically a strategy, an information flow map can

be used.

4.1.4 Social Organization and Cooperation

Analysis (SOCA)

This phase identifies who can do the work and how it

can be shared. (Naikar, 2013). That means, social

organization and cooperation analysis is concerned

with the constraints that the allocation, distribution,

and coordination of work impose on actors (Naikar,

2013). Work can be organized in many ways in a

particular system.

4.1.5 Worker Competencies Analysis

(WCA)

This phase identifies the perceptual and cognitive

capabilities of workers that are required for

performing the work described in the previous phases

(Naikar, 2013).

Remark. To determine the information requirements

of the driver, we need a task knowledge and the driver

behaviour model towards it. That it is the reason why

we focused on the first two phases of CWA. These

information will help us to precisely define which

data to communicate in autonomous moments of

driving. Then, we assume that the rules based on these

information, will help the driver to better understand

the road situation, the Human-Machine Interface and

the automated system. The rules that will be created,

will be derived in a specific algorithm.

4.2 Second Step of the Methodology:

Algorithm Building

In the second step, we suggest defining general rules

to deal with the complexity and dynamics of the

driving system. CWA is quite theoretical, although a

method is proposed to translate the results obtained in

terms of interface specifications (Burns &

Hajdukiewicz, 2013). This is why we suggest

building a strong structure of prioritized rules of

information. Parasuraman (2000) identified four

classes of functions that can be automated at different

levels: information acquisition, information analysis,

decision and action selection, and action

implementation. Those levels, combined with the

extracted CWA information, will allow us to build a

matrix of salient information regarding driving

management and the interaction between driver and

automated system. This matrix will lead to a set of

prioritized AR rules for each detailed use case, which

will provide the drivers with means to deal with

unanticipated and unforeseen events. These rules will

help to cope with the information capabilities of the

displays. Consequently, information presentation will

be optimized and unnecessary information will be

minimized.

4.3 Third Step of the Methodology:

Interface Specification

The third step consists in specifying the interface

practically according to the two use cases we have

selected. We will define clusters of information and

their modality on the interface. There, we will provide

a description of how the information is required to the

driver, when the vehicle is driving in autonomous

mode. By giving some parameters, these information

will be described. Miller (1999) has identified five

parameters: scope, resolution, bandwidth, importance

and control.

AugmentedRealityInterfaceDesignforAutonomousDriving

4.4 Fourth Step of the Methodology:

Users’ Tests

In the final step, we will evaluate the interface design

through user testing on a simulator. All the data and

specifications have to be tested. Indeed, a design can

be conceptually good but practically not suitable. For

that reason, we have planned three major tests to

improve the whole design:

a) In the first test, we will test the interface in

a full virtual windshield HUD, a pseudo HUD and

other devices.

b) In the second test, the virtual HUD will have

dimensions larger than the ones of a

conventional HUD.

c) In the third test, the virtual HUD has the

dimensions of a conventional HUD Virtual

HUD is created for AR information HUD is

for classic information like speed.

5 EXPECTED OUTCOME

At the end of our work, we expect an interface that

adapts itself with the current situation of driving. We

assume that this interface will enhance driver’s

understanding of the interface and the road

environment. This interface will be driven by strong

rules that will permit to capture pertinent information

in the appropriate shape. We assume that this

interface will be suitable for lane change in

autonomous mode, for transition from autonomous to

manual mode, and for night cruising in manual mode.

6 STAGE OF THE RESEARCH

For now, we have already realized the necessary

phases of CWA in the situation of lane change, one of

our use case apart from manual driving in night and

transition from autonomous mode to manual mode.

6.1 Work Domain Analysis Application

to Lane Change

There is a sparse literature on Work Domain Analysis

on Road Driving. Some authors have realized this

analysis. Stoner, Wiese and Lee (2003) have applied

the Abstraction Hierarchy analysis to the driving

domain. They identified information requirements for

drivers. Salmon, Regan, Lenné, Stanton and Young

(2006) have realized Work Domain Analysis of the

road transport system in Victoria, Australia. These

particular works present a general application to the

whole domain of the driving, including the total

system, the subsystems and the components.

There are several steps in WDA methodology.

Step 1: WDA purposes

There we consider the problem definition and the

approach to address the problem. For this thesis, the

purpose of WDA is related to the information

requirements of driver whilst driving in an

autonomous vehicle of level 3. Particularly, we pay

attention to lane changing in autonomous mode,

“night driving” in manual mode and transition from

autonomous to manual mode.

Step 2: Project constraints identification

LRA project has many constraints that do not allow

to go deeply into detail. These constraints include: the

time constraint, the expertise-related constraints.

In fact, there are few experts on the project who do

not have enough time to invest in this analysis. These

constraints “forced” us to focus on the subsystem

(driver-vehicle-road system) rather than to consider

the whole system (road transportation) and each

component.

Step 3: WDA boundaries

If boundaries are not clearly determined, WDA can

become very large, complex and not understandable.

That is why we consider highways and motorways as

roads where the vehicle can drive. This choice helps

to not consider potential obstacles like pedestrians.

Step 4: Constraints nature identification

Naikar and al. (2005) have identified 5 categories of

work systems within a causal-intentional continuum

where the focus system falls. In our work, our focus

system can be from the first category “Automated

systems governed by laws of nature” in autonomous

mode, or between the third category “Systems

governed by actors’ intentions” and the fourth

category “Systems governed by actors’ personal

objectives” in manual mode. This classification let us

conclude to focus on causal constraints (Salmon et al.,

2006).

Step 5: Information sources identification

To realize WDA, we mainly use documents sources

of information: articles dealing with autonomous

vehicles, articles dealing withlane changing,

handover process and so on. Brainstorming with

some experts and legislation documentation were also

identified as sources of information.

Step 6: Abstraction Decomposition first construction

ICINCO2015-DoctoralConsortium

With our first reading, we construct our first

Abstraction Decomposition Space (ADS). Ordinarily,

ADS is a matrix composed of the Abstraction

Hierarchy and the Decomposition Space. We just

realized an Abstraction Hierarchy which is

decomposed in 5 levels: functional purposes, values

and priority measures, Purpose-related functions,

object-related functions and physical objects.

Decomposition Space is formed of total system,

subsystems and different components. First, we

construct a macroscopic WDA for the autonomous

driving. Then, we construct a detailed WDA related

to the lane changing considered as a unique system. It

is this WDA that we will explained into detail in the

following paragraphs.

Step 7: Abstraction Decomposition second iteration

After many reviews, we modified some elements to

obtain the final WDA for the lane change.

Figure 3 presents the WDA of the autonomous

vehicle. We present an overview of its structure

because it is beyond the scope of this paper to present

the complete ADS for the Autonomous Vehicle.

Figure 3: Autonomous vehicle Work Domain Analysis.

Figure 4: Work Domain Analysis of a “human” lane change

system.

We insist on one particular point related to the figure

3. In the purpose related-functions, there is “Vehicles

physical dynamics”. Within these vehicle physical

dynamics, there are speed, velocity, maneuverability,

etc. Lane changing is a maneuver that autonomous

vehicle can realize in any mode. Because lane

changing is one of our use case, we have decided to

construct a WDA for the lane change system which

does not exist now. We assumed that this WDA will

also help to design lane change system.

Figure 4 presents the LC WDA in manual mode.

For each level of the Abstraction Hierarchy, we

give a definition and present its components.

6.1.1 Functional Purposes

Functional purposes are the reason to be of the

system. Contrarily to goals, functional purposes are

more stable over time (Burns, Vicente, 2001). Naikar

suggests some questions to find them such as “Why

does the system exist?”, “Why is the system

necessary?”, “Which purposes should the system

achieve?” (Naikar, 2013).

A lane change has been defined as a deliberate and

substantial shift in the lateral position of a vehicle

(Chovan et al., 1994). That means that the reason of a

lane change is a shift, a movement from one lane to

another.

6.1.2 Values and Priority Measures

Values and priority measures are the criteria that help

to evaluate the system progression towards its

functional purposes (Naikar, 2013). Those criteria

help for system evaluation. They also allow to

prioritize the elements of the below level, the

purpose-related functions. To determine values and

priority measures, Naikar suggests some questions

like “What criteria can be used for evaluating how

well the system is fulfilling its functional purposes?”,

“What fundamental laws, principles, or values must

be respected by the system?” (Naikar, 2013).

Considering a lane change system, we have found

three main values and priority measures:

a) Optimize lane change duration: A lane change

longs between 3.5 and 8.5s with a mean of 5.8 s

on the highways (Tijerna, 1997). Hetrick studies

have estimated lane change duration between 3.4

and 13.6 s with of 6 s (Hetrick, 1997). We assume

that a lane change system should realize the

maneuver in a time interval [3.4 s; 7 s] to ensure a

certain level of safety. Considering the dynamic

characteristic of the road environment, a

hypothesis is that more the lane change will last,

more the maneuver will be dangerous.

AugmentedRealityInterfaceDesignforAutonomousDriving

b) Minimize maneuver risk and severity of

potential accidents: This measure completes the

first one. In fact, the danger of a lane change is not

just related to its duration but also by the severity

of crashes that can occur because a lack of

adjustment in the maneuver. Then, a good

criterion is to evaluate the number of accidents

caused by the change system with an expectation

of a zero accident observation.

c) Optimize ego vehicle driver comfort: The safety

is the main aspect to consider in a lane change.

When we consider also comfort aspect, it is better.

In this configuration, a cognitive calculation is

made in driver’s head to respect the security and

performance constraints. He will want to do the

maneuver and be comfortable at the same time.

We have already projected to evaluate comfort

through a qualitative analysis of users’ tests that

will be carried out on the simulator.

6.1.3 Purpose-related Functions

Located in the middle of the hierarchy, the purpose-

related functions refers to functions that a system

must accomplish to achieve the functional purposes.

They can be seen as the “uses” that physical objects

and their object-related processes “put to” in a system

(Miller and Vicente, 1998, p.15 cited by Naikar,

2013, p72). Those functions can also influence values

and priority measures (Stanton, 2014). for a lane

change system, we have identified the following

functions:

a) Scan the environment

b) Detect near objects around the ego vehicle

c) Monitor near objects around the ego

vehicle

d) Understand near objects intentions around

the ego vehicle

e) Evaluate gaps between closest vehicles and

the ego vehicle (speed differential, distance

differential, etc.)

f) Adapt style maneuver

6.1.4 Object-related Functions

The object-related processes, which are highly

dependent on the properties of physical objects, serve

the system to achieve its purpose-related functions

(Naikar, 2013). To find those processes, we can

answer to some questions suggested by Naikar:

“What can physical objects of relevance to the system

do or afford?”,”What functional purposes or

functional capabilities of physical objects are

necessary for the system to achieve its

purpose-related functions?” (Naikar, 2013, p.182).

We have extracted some object-related functions

for a naturalistic lane change:

a) Collect and store road characteristics: Road

characteristics refer to access type (freeway,

highway, arterial, ramp, secondary road), road

type (weaving, rural) and number of lanes.

b) Collect and store the ambient characteristics:

Ambient characteristics; refer to the weather (rain,

sun, ice snow, cold), the visibility (sun, dust, rain

fog) and he time of day.

c) Collect and store the road traffic signs and

signals: In the traffic signs, we include, speed

limitation traffic signs, direction traffic sign,

traffic markings and stationary cameras/police

cars.

d) Collect and store the location of the ego vehicle

and the others vehicles.

e) Collect and store the speed of the ego vehicle

and those of others vehicles.

f) Identify the type of the near vehicles (truck,

bus, etc.).

g) Realize the maneuver according to the

acceleration and braking maximal capabilities

of the ego vehicle.

h) Adjust lateral control and longitudinal control

of the ego vehicle.

i) Detect turn signals activation of others

vehicles.

6.1.5 Physical Objects

It is the level the most concrete of the hierarchy. In

this level, all the physical objects present in the

system are concerned. They have functional

capabilities that allow the system to accomplish the

object-related processes.

For our use case, we did not go deeply into the

detail because the hard “part” is out the scope of our

work. We have listed the main elements.

a) Global Positioning System (GPS)

b) Pedals

c) Steering Wheel

d) Turn signal

e) Others sensors and actuators: radar,

LIDAR, Adaptive Cruise Control, Lane

Departure Warning,

6.2 Control Task Analysis Application

to Lane Change

Control Task Analysis can be seen as an analysis of

ICINCO2015-DoctoralConsortium

Figure 5: Decision ladder template applied to lane change system.

the activity in terms of decision making. It does not

answer the question related to how the activity should

be realized nor the question of who is accomplishing

the activity. Like we have said before, there are many

activities that can occurred in a particular work

domain. There, we will analyze the lane change

activity from the beginning to the end.

6.2.1 The Decision Ladder Template

The decision ladder is comprised with annotations

that come from Rasmussen and Vicente papers

(Rasmussen, 1976; Vicente, 1999). It is composed of

rectangles and ovals. Rectangles are the boxes of

information processing. Ovals are knowledge states

which can be either the inputs either the outputs of the

rectangles. The writings near the oval are formulated

like questions to show actors reasoning (Elix and

Naikar, 2008). The ladder must not to be linearly

read. The result of the lane change decision ladder is

represented in figure 5.

6.2.1.1 Goals

At the top of the ladder, the first state of knowledge

is the one of goals. This state consists in the system

goals definition. It is slightly similar to the first level

of Abstraction Hierarchy, the functional purposes.

Literature advises the following formulation for this

circle “Goal that begins with a verb + constraints”

(Elix and Naikar, 2008; Jenkins, Stanton, Walker and

Rafferty, 2009).

For the lane change system, we formulate the goal

as move from one original lane to a destination

lane safely and efficiently, by taking into account

the time windows and the navigational

constraints.

6.2.1.2 Alert

After the goals definition, we consider the circle at the

bottom of the left branch of the ladder: the alert. The

alert refers to all the elements that can trigger an

event.

In our case, it refers to all the causes that can

trigger a lane change. According to Lee, Olsen,

Wierwille and Naranjo, the following question can be

asked to determine the alert (Lee, Olsen, Wierwille,

2004; Naranjo, 2008; Olsen, 2003).

A1: Should I enter in a highway?

A2: Should I prepare to exit the highway or

should I exit now?

A3: Should I anticipate a vehicle merging?

AugmentedRealityInterfaceDesignforAutonomousDriving

A4: Should I avoid obstacles like works on the

road or crash situations?

A5: Should I anticipate a vehicle coming fast

behind?

A6: Should I return to a preferred lane?

A7: Is there a lanes number decrease?

A8: Is there a lanes number increase?

A9: Is there a slow vehicle in front of me?

A10: Is the distance differential between the

preceding vehicle and I increasing?

6.2.1.3 Information

There we find all the information necessary to

evaluate the alert. For example, Stanton and Bossell

(2014), while studying how a submarine returns to

periscope depth, enumerated in the information level,

the surface constraints, weather constraints, etc.

We have formulated questions to grasp the

necessary information.

I1: What is the lateral position of the ego vehicle

in its lane?

I2: What is its speed?

I3: What are the others vehicles location (in one

lane, between two lanes with the lateral

displacement) in the ego lane and in the adjacent

lanes?

I4: What are their speeds?

I5: What are the others vehicles kinds (truck, city

cars, bus, ambulance, etc.)?

I6: What are the distances between those vehicles

and the ego vehicle (distance proximity)?

I8: What are the infrastructure constraints (speed

limitation, markings, distance to the next exit,

etc.)?

I9: What is the road configuration (curve, straight,

etc.)?

I10: What are the weather conditions?

I11: Are the turn signals of the near vehicle are

activated?

6.2.1.4 System States

Analysis and fusion of information allow knowing the

state in which the system is (Jenkins et al., 2009).

Because of the diversity of information in terms of

nature, quantity and understanding, there are several

system states.

Lane change can be realized or not, depending of

the relations between the subject vehicle also called

ego vehicle and the others.

S1: What are the intentions of the drivers around

me?

S2: What are the trajectories of the vehicles or

their current actions?

S3: Are the safety gaps respected between the

ego vehicle and the others ones?

S4: Are the speed differential and the distance

differential sufficiently safe to permit me to do a

lane change?

S5: Do I have to take an exit in a few seconds

(Spatial constraints)?

S6: Is it possible to realize a lane change while

respecting speed limitation?

S7: Do the weather conditions influence the lane

change (visibility)?

6.2.1.5 Options

Towards a system state, many actions can be carried

out to achieve the purpose of the system. The number

of actions to realize are deeply related to the system

state. We assume that less complex a system will be,

less action will need to be realized. Jenkins suggests

to formulate the elements at this level by: “Is it

possible…?” (Jenkins, 2009).

O1: Is it possible to realize a quick lane change

(because of a weak maneuver margin)?

O2: Is it possible to realize a safe lane change?

O3: Is it possible to urgently realize a lane change

(because of an imminent crash)?

O4: Is it possible to pursue a cruising in the same

lane?

O5: Is it possible to realize an overtaking?

6.2.1.6 Chosen Goal

As we mentioned before, a system can have one or

many goals. But at a particular time, just one goal can

be elicited because of the environment constraints

(Elix, Naikar, 2008 cited by Jenkins et al., 2009).

In our case, the chosen goal is the same that the

goal because there is only one goal: “move from one

original lane to a destination lane safely and

efficiently, by taking into account the time windows

and the navigational constraints.”

6.2.1.7 Target State

When an option is chosen, it becomes the target state.

The target state can be formulated by “Is (option) can

be adopted?” (Jenkins et al., 2009).

T1: Lane change is fast.

T2: Lane change is safe.

T3: Lane change must be urgently realized.

T4: Lane change can be realized now (temporary

cruising).

T5: Lane change cannot be realized

(Undetermined cruising).

ICINCO2015-DoctoralConsortium

6.2.1.8 Task

It is the task and a group of tasks to realize in order to

achieve the goal(s).

In the lane change system, the main task is to

define the trajectory and follow it.

6.2.1.9 Procedure

It is the procedure that has to be conducted in order to

realize the tasks.

P1: Monitor the road environment.

P2: Activate turn signal to notify the lane change,

beginning.

P3: Come near the line that separates the original

lane and the destination lane.

Accelerate to precede the vehicle that is front if

necessary.

P4: Deactivate the turn signal.

P5: Stabilize the speed and the ego vehicle

position in the destination lane.

P6: Follow the trajectory.

This procedure can be reiterated if necessary.

6.3 Information Requirements for

Interface Design

The information that is presented by an interface

should be the appropriate information in the

appropriate shape. Then the question of form of the

information and its quantity are critical for drivers’

situation awareness and workload.

The WDA and the CTA help us to extract

information requirements independently of the

autonomous system. The categories of information

based on those analyses are listed below:

 For the ego vehicle

o Vehicle condition: fuel level

o Vehicle component conditions: if

they work well or not

o Current location and desired end

point

o Own dynamics: speed, velocity,

lateral displacement

 For the others vehicle

o heir type

o Their location

o Their relative proximity with the

ego vehicle in terms of speed,

distance or time

o Their intentions

o Their actions

 For the infrastructure

o Road signage presentation

o Road type

o Infrastructure related Warning

o Lane width

There are also information requirements related to the

automated system and the ego vehicle driver.

Considering that autonomous mode should be

enjoyable time where the driver will have access to

the information he wants to have access to, driver

should know for example, how long he has to plan its

leisure activity. It is also necessary to check driver

vigilance especially when the transition from

autonomous to manual mode will be required. Other

information should be conveyed:

 For the automated system

o Its intentions

o Its current action

o Its comprehension of road rules

o Its road perception

o Remaining time in autonomous

mode

 For the ego vehicle driver

o Its distraction level

o Its protection information:

information related to the seatbelt

lock or unlock, hands on steering

wheel or not, feet on pedals or not.

Some Intelligent Transport Systems provide those

information but for some, not in their entirety.

Consequently, the next step will be to determine

which information will be design in Augmented

Reality.

Comments. At this stage of the work, we have

realized the most critical part of the work. We have

spent much time to do it but it was really important to

have a convenient work. Now we are ready to go into

the prioritization and rules levels for a first users’ test

in December.

7 CONCLUSIONS

To design an interface, precise and adequate

information are timely needed, especially in

autonomous mode for driving. In this paper, we have

described our methodology for interface design. We

have finished the first step of Cognitive Work

Analysis to capture information needs in lane change

and associate them with our use cases.

We are thinking on information representation

and the level of transparency of the interface. This

methodology, derived from a cognitive approach, will

lead to a set of rigorous rules. Those rules will allow

AugmentedRealityInterfaceDesignforAutonomousDriving

at specific time specific components to appear either

in Augmented Reality form or not.

Acknowledgements. The thoughts expressed here are

the work of the authors, but related work has been

supported by French government in conformance

with the PIA (French acronym for Program of

FUTURE Investments) within IRT (French acronym

for Technologic Research Institute) SystemX.

ACKNOWLEDGEMENTS

The thoughts expressed here are the work of the

authors, but related work has been supported by

French government in conformance with the PIA

(French acronym for Program of FUTURE

Investments) within IRT (French acronym for

Technologic Research Institute) SystemX.

REFERENCES

Azuma, R. T. (1997). A survey of augmented reality.

Presence, 6(4), 355-385.

Burns, C. M., & Hajdukiewicz, J. (2013). Ecological.

interface design. CRC Press.

Chen, J. Y. C, Boyce, Wright, J., Procci, K., Barnes, M. (in.

prep.). SA-based Agent Transparency. ARLTechnical

Report.

Chovan, J. D., Tijerina, L., Alexander, G., &

Hendricks, D. L. (1994). Examination of lane change

crashes and potential IVHS countermeasures (DOT HS

808 071) Washington, DC: National Highway Traffic

Safety Administration. Available online at:

http://www.itsdocs.fhwa.dot.gov/\JPODOCS\REPTS_

TE/61B01!.PDF.

Elix, B., & Naikar, N. E. E. L. A. M. (2008) Designing safe

and effective future systems: A new approach for

modelling decisions in future systems with Cognitive

Work Analysis. In Proceedings of the 8

international

symposium of the Australian Aviation Psychology

Association.

Endsley, M. R. (1995). Toward a theory of situation

awareness in dynamic systems. Human Factors: The

Journal of the Human Factors and Ergonomics Society,

37(1), 32-64.

Endsley, M.R., & Kaber, D.B; (1999). Level of Automation

effects on performance, situation awareness and

workload in dynamic control task.

Jenkins, D. P., Stanton, N. A., Salmon, P. M., Walker, G.

H., & Rafferty, L. (2010). Using the decision-ladder to

add a formative element to naturalistic decision-making

research. Intl. Journal of Human Computer Interaction,

26(2-3), 132-146.

Gasser, T., & Westhoff, D. (2012, July). BASt-study:

Definitions of automation and legal issues in Germany.

In Presentation at the Road Vehicle Automation

Workshop.

Habenicht, S., Winner, H., Bone, S., Sasse, F., &

Korzenietz, P. (2011, June). A maneuver-based lane

change assistance system. In Intelligent Vehicles

Symposium (IV), 2011 IEEE (pp. 375-380). IEEE.

Hetrick, S. (1997). Examination of driver lane change

behavior and the potential effectiveness of warning

onset rules for lane change or "side" crash avoidance

systems. Unpublished master's thesis, Virginia

Polytechnic Institute and State University, Blacksburg,

Virginia. Available on-line at:

http://scholar.lib.vt.edu/theses/available/etd

382915749731261/unrestricted/etd.pdf.

Lee, S. E., Olsen, E. C., & Wierwille, W. W. (2004). A

comprehensive examination of naturalistic lane

changes (No. HS-809 702,).

Marinik, A., Bishop, R., Fitchett, V., Morgan, J. F.,

Trimble, T. E., & Blanco, M. (2014, July). Human

factors evaluation of level 2 and level 3 automated

driving concepts: Concepts of operation. (Report No.

DOT HS 812 044). Washington, DC: National

Highway Traffic Safety Administration.

Michon, J. A. (1985). A critical view of driver behavior

models: what do we know, what should we do?. In

Human behavior and traffic safety (pp. 485-524).

Springer US.

Naikar, N. (2013). Work domain analysis: Concepts,

guidelines, and cases. CRC Press. National Highway

Traffic Safety Administration. (2013). Preliminary

statement of policy concerning automated vehicles.

Washington, DC.

Olsen Erik (2003). Modeling Slow Lead Vehicle.

Changing. Dissertation submitted to the faculty of the

Virginia Polytechnic Institute and State University in

partial fulfillment of the requirements for the degree of

doctor of philosophy in Industrial and Systems

Engineering. 22 September 2003, Blacksburg, Virginia.

Available at: http://scholar.lib.vt.edu/theses/available/

etd12032003152916/unrestricted/olsen_dissertation.pdf.

Riley, V. (1989, October). A general model of mixed.

initiative human-machine systems. In Proceedings of

the Human Factors and Ergonomics Society Annual

Meeting (Vol. 33, No. 2, pp. 124-128). SAGE

Publications.

Sheridan, T. B., & Verplank, W. (1978). Human and

Computer Control of Undersea Teleoperators.

Cambridge, MA: MA: Man-Machine Systems

Laboratory, Department of Mechanical Engineering,

MIT.

Sheridan, T. B., & Parasuraman, R. (2005). Human

automation interaction.Reviews of human factors and

ergonomics, 1(1), 89-129.

Shinar, D. (1978). Psychology on the Road: The Human

Factor in Traffic Safety. New York: Wiley.

Smith, J., 1998. The book, The publishing company

London, 2nd edition.

Tijerina, L., Garrott, R, W., Glecker, M., Stoltzfus, D., &

Parmer, E. (1997, November 3). Van and passenger car

driver eye glance behavior during lane change decision

ICINCO2015-DoctoralConsortium

phase (Revised DRAFT: INTERIM REPORT).

Transportation Research Center, Inc. and National

Highway Transportation Safety Administration,

Vehicle Research and Test Center.

Van Winsum, W. (1999). The human element in car

following models. Transportation research part F:

traffic psychology and behaviour, 2(4), 207-211.

AugmentedRealityInterfaceDesignforAutonomousDriving