Augmented Reality Interface Design for Autonomous Driving
Raissa Pokam Meguia
1
, Christine Chauvin
2
and Serge Debernard
3
1
IRT SystemX, Renault, Paris, France
2
Lab-STICC UMR CNRS 6285, Université Bretagne Sud, Morbihan, France
3
LAMIH UMR CNRS 8201, Université de Valenciennes, Famars, France
1 RESEARCH PROBLEM
During the last decades, the driving paradigm has
been changed due to the introduction of the
automation. Indeed, from a configuration where the
driver was the sole responsible of the driving, we
move to a bilateral configuration where the human
agent and the technical agent share the driving task.
This bilateral configuration can be observed in
vehicles called autonomous ones. Considering the
number of tasks delegated to the technical agent, there
are several levels of automation (NHTSA, 2013). The
“full automation” level means that the technical agent
is the sole responsible of the driving. It will be
observable in the horizon 2030. Many vehicles
manufacturers and laboratories work currently on the
intermediate levels of automation.
The LRA (French Acronym for Localization and
Augmented Reality) project is a collaborative French
project where 10 academics and industrials work
together to deliverer an Augmented Reality (AR)
Human Machine Interface (HMI) of an autonomous
vehicle of level 3, in the National Highway Traffic
and Safety Administration (NHTSA) taxonomy. This
level supposes that the vehicle is able to drive alone
in some particular conditions. During this period
where the technical agent is in charge of the driving
(also called free time), the human agent can do some
secondary or tertiary tasks such as reading, writing,
etc. The human agent is, in these periods,
disconnected from the road environment and the
primary task of driving. He is out-of-the-loop, and it
is necessary to take into account two main situations.
In the first one, at particular moments, he may want
to verify how the technical agent works in order to
know what the technical is doing and if its behavior
is accurate. In the second one, at a precise time,
handover will be required by the system.
Consequently, the human agent has to be reengaged
physically and cognitively in the driving task. He has
to build a mental representation of what is going on
around him in the road environment and what the
technical agent is doing, in order to have a whole
situation understanding of the situation. This
task-specific understanding of the situation refers to
situation awareness (Endsley, 1985). Through
interface design, situation awareness has to be
enhanced. In order to achieve this goal, we decided to
use in information shaping, Augmented Reality, an
innovative technology.
AR typically describes interfaces that overlay
images of virtual objects on images of the real world.
We can talk also about Augmented Reality for haptic,
auditory or vestibular cues. In this research, we have
decided to focus on visual cues.
There are many definitions of Augmented Reality
but we have decided to choose Azuma (1997) one. He
defined Augmented Reality as any interface that has
these three characteristics:
1. combines the real and the virtual.
2. is interactive in real time.
3. is registered in 3d.
Consequently, considering the interface design in
autonomous mode, we have identified three
fundamental questions to design the interface:
a. In autonomous mode and in handover processing,
which sufficient representation should the drivers
maintain or establish? According to the Situation
Awareness model defined by Endsley (1995), this
question may be subdivided into three sub-questions:
(i) What should the drivers perceive?
(ii) What should they understand?
(iii) Which projection of the external environment
and the system should they perform?
b. How should we design the displays?
(ii) What should be displayed?
(iii) How should that information be displayed?
(iv) When should it be displayed?
(v) With which prioritization?
c. What is the added value of Augmented Reality in
the displays?
22
Pokam Meguia R., Chauvin C. and Debernard S..
Augmented Reality Interface Design for Autonomous Driving.
Copyright
c
2015 SCITEPRESS (Science and Technology Publications, Lda.)
2 OUTLINE OF OBJECTIVES
The general objective is the design of rules for
Human-Machine Interface. Through the development
of a specific methodology correlated to an innovative
technology, Augmented Reality, we will design an
adaptive interface which will ensure a safe and a
comfortable driving in an automated vehicle of level
3.
In order to meet the challenges outlined above, the
LRA project integrates multidisciplinary expertise
from French research institutions and vehicle
industrials. The involvement of the industrials creates
the potentiality of a vulgarisation of the finished
work. We have defined two main goals:
1) Application of a cognitive method to derive
information requirements for the driver and
hierarchize them through strong rules.
2) Conveyance all these information in an
appropriate shape by considering displays
capabilities.
To achieve these goals, we have realized a literature
review at the beginning.
3 STATE OF THE ART
This chapter contains the main theoretical concepts
that underlie our research work: driving task,
automation, situation awareness, transparency, and
Augmented Reality.
3.1 Driving Task
Michon (1985) has defined a hierarchical control
structure of the driving task. This structure divided
the driving task into three levels of control: strategic
level, tactical level and operational level. See Figure
1. Each level has its importance in the driving
task.
In the strategic level, we deal with the route that
the ego vehicle intends to follow. The route is planned
and the general objectives are fixed. In the tactical
level, we talk about the maneuvers: passing a vehicle,
entering highway, exiting highway, lane changing,
overtaking. This level concerns all the maneuvers
Figure 1: Three levels of control in driving.
which allow to achieve short-terms maneuvers. In the last
level, the longitudinal and the lateral controls are
concerned.
3.2 Automation
When the driver is responsible of the driving task, he
has to lead his vehicle in each of the aforementioned
levels of control. More often, we observe a delegation
of one or more of functions realized by the human to
one or several technical agents. Considering the
number of functions delegated to the technical agent,
we have many levels of automation (Sheridan, 1978).
There are many taxonomies that try to classify those
levels.
The most famous are: National Highway Traffic
and Safety Administration (NHTSA, 2013)
taxonomy, Society of Automotive Engineers (SAE)
taxonomy, Sheridan and Verplanck (1978)
taxonomy, Endsley and Kaber (1999) taxonomy,
Gasser and Westhoff (2012) taxonomy and Riley
(1989) taxonomy. As a LRA project constraint,
NHTSA taxonomy was chosen but it has the
disadvantage to not clearly define the functions
attributed to each agent. As mentioned above, we
work on an automated vehicle of level 3. As NHTSA
specify, vehicles at this level of automation enable the
driver to cede full control of all safety-critical
functions under certain traffic or environmental
conditions and in those conditions to rely heavily on
the vehicle to monitor for changes in those conditions
requiring transition back to driver control. The driver
is expected to be available for occasional control, but
with sufficiently comfortable transition time. The
vehicle is designed to ensure safe operation during
the automated driving mode. An example would be an
automated or self-driving car that can determine
when the system is no longer able to support
automation, such as from an oncoming construction
area, and then signals to the driver to reengage in the
driving task, providing the driver with an appropriate
amount of transition time to safely regain manual
control (Marinik Bishop, Fitchett, Morgan, J. F.,
Trimble & Blanco, 2014).
This kind of interaction and others ones introduce
Situation Awareness concept.
3.3 Situation Awareness
Situation Awareness is a term derived initially from
the aviation domain. In this domain, it plays a crucial
role in the design of military interface. There are
numerous definitions of Situation Awareness but the
most used and the widely accepted is the one from
AugmentedRealityInterfaceDesignforAutonomousDriving
23
Endsley (1995) defining it as the perception of
elements in the environment within a volume of time
and space, the comprehension of their meaning, and
the projection of their status in the near future.
The important information has to be conveyed to
the driver appropriately and accurately. We assume
that not all the information has to be conveyed to the
driver. This introduces the transparency term.
3.4 Transparency
The concept of transparency can be seen in two points
of view. Transparency in a form aspect, refers to the
level of opacity of one particular component. There is
also an aspect that deals with the quantity of
information. Considering human-automation,
transparency cannot, practically speaking, means that
the human knows everything about what the
automation is doing (Miller, 2014).This definition is
a naïve one. Chen et al. suggested another definition
for automation transparency: … the descriptive
quality of an interface pertaining to its abilities to
afford an operator’s comprehension about an
intelligent agent’s intent, performance, future plans
and reasoning process.
Transparency could take advantage of innovative
technology, such as Augmented Reality.
3.5 Augmented Reality
Many modern cars (e.g. Audi Q7, BMW M3 Berline)
are equipped with Head-Up Display (HUD)
technology. This technology enables Augmented
Reality (AR) implementation (Tonnis, Sandor,
Lange, & Bubb, 2005). Usually, AR is defined as a
continuum from real to Virtual Reality (Milgram,
1994). Generally, AR in cars deals with “the problem
of directing a user’s attention to a point of interest
(Tonnis et al., 2005). AR can “alert drivers and guide
their attention to dangerous situations” (Tonnis et al.,
2005). We thus assume that AR can enhance global
awareness and local guidance by conveying the right
information at the right moment.
4 METHODOLOGY
The figure 2 presents, step by step, the general
approach employed for the design and development
work we conduct. This work involves the
specification and design of the interface.
4.1 First Step of the Methodology:
Cognitive Work Analysis
This step involves modeling tasks and extracting
information requirements for drivers. We decided to
use a method that considers both technical and human
aspects together: Cognitive Work Analysis (CWA)
(Rasmussen, 1990). It is an integrated framework that
defines the work demands of complex sociotechnical
systems in terms of the constraints on actors
(Rasmussen, 1986; Rasmussen, Pejtersen, &
Goodstein, 1994; Vicente, 1999; Naikar, 2013).
CWA is an Ecological Interface Design (EID) -based
approach. In EID approach, the constraints of the
system are enhanced in order to allow drivers to take
effective actions and to know the impact on their
actions in their goals achievement (Burns and
Hajdukiewicz, 2004 cited by Salmon, Regan, Lenné,
Stanton and Young, 2006).
CWA also provides information regarding the
different possibilities for actions inside the system.
Figure 2: HMI design methodology.
ICINCO2015-DoctoralConsortium
24
This method consists of several phases of analysis:
work domain analysis, task analysis, strategies
analysis, organizational analysis, and skills analysis.
4.1.1 Work Domain Analysis (WDA)
It is the first phase of Cognitive Work Analysis. It
focuses on analyzing the boundary conditions or
constraints of a work system. Through an Abstraction
Hierarchy or an Abstraction Decomposition System,
the structural functions of the system are determined.
4.1.2 Control Task Analysis (ConTA)
The control task analysis identifies the activity that is
required in a work domain (Naikar, 2013). After the
definition of the constraints associated with the
environment, this dimension of the CWA focuses on
determining the constraints associated with what
needs to be done in the system (Naikar, 2013). In this
phase, two tools are usually used: the contextual
activity template for work situations and functions
modeling, and decision ladder template for control
tasks modeling (Naikar, 2013).
4.1.3 Strategies Analysis (StrA)
This phase identifies how the activity can be carried
out (Naikar, 2013). Strategies analysis deals with the
constraints associated with all the possible ways to
realize an activity. In fact, it is possible to have many
strategies for a single activity. To represent
graphically a strategy, an information flow map can
be used.
4.1.4 Social Organization and Cooperation
Analysis (SOCA)
This phase identifies who can do the work and how it
can be shared. (Naikar, 2013). That means, social
organization and cooperation analysis is concerned
with the constraints that the allocation, distribution,
and coordination of work impose on actors (Naikar,
2013). Work can be organized in many ways in a
particular system.
4.1.5 Worker Competencies Analysis
(WCA)
This phase identifies the perceptual and cognitive
capabilities of workers that are required for
performing the work described in the previous phases
(Naikar, 2013).
Remark. To determine the information requirements
of the driver, we need a task knowledge and the driver
behaviour model towards it. That it is the reason why
we focused on the first two phases of CWA. These
information will help us to precisely define which
data to communicate in autonomous moments of
driving. Then, we assume that the rules based on these
information, will help the driver to better understand
the road situation, the Human-Machine Interface and
the automated system. The rules that will be created,
will be derived in a specific algorithm.
4.2 Second Step of the Methodology:
Algorithm Building
In the second step, we suggest defining general rules
to deal with the complexity and dynamics of the
driving system. CWA is quite theoretical, although a
method is proposed to translate the results obtained in
terms of interface specifications (Burns &
Hajdukiewicz, 2013). This is why we suggest
building a strong structure of prioritized rules of
information. Parasuraman (2000) identified four
classes of functions that can be automated at different
levels: information acquisition, information analysis,
decision and action selection, and action
implementation. Those levels, combined with the
extracted CWA information, will allow us to build a
matrix of salient information regarding driving
management and the interaction between driver and
automated system. This matrix will lead to a set of
prioritized AR rules for each detailed use case, which
will provide the drivers with means to deal with
unanticipated and unforeseen events. These rules will
help to cope with the information capabilities of the
displays. Consequently, information presentation will
be optimized and unnecessary information will be
minimized.
4.3 Third Step of the Methodology:
Interface Specification
The third step consists in specifying the interface
practically according to the two use cases we have
selected. We will define clusters of information and
their modality on the interface. There, we will provide
a description of how the information is required to the
driver, when the vehicle is driving in autonomous
mode. By giving some parameters, these information
will be described. Miller (1999) has identified five
parameters: scope, resolution, bandwidth, importance
and control.
AugmentedRealityInterfaceDesignforAutonomousDriving
25
4.4 Fourth Step of the Methodology:
Users’ Tests
In the final step, we will evaluate the interface design
through user testing on a simulator. All the data and
specifications have to be tested. Indeed, a design can
be conceptually good but practically not suitable. For
that reason, we have planned three major tests to
improve the whole design:
a) In the first test, we will test the interface in
a full virtual windshield HUD, a pseudo HUD and
other devices.
b) In the second test, the virtual HUD will have
dimensions larger than the ones of a
conventional HUD.
c) In the third test, the virtual HUD has the
dimensions of a conventional HUD Virtual
HUD is created for AR information HUD is
for classic information like speed.
5 EXPECTED OUTCOME
At the end of our work, we expect an interface that
adapts itself with the current situation of driving. We
assume that this interface will enhance driver’s
understanding of the interface and the road
environment. This interface will be driven by strong
rules that will permit to capture pertinent information
in the appropriate shape. We assume that this
interface will be suitable for lane change in
autonomous mode, for transition from autonomous to
manual mode, and for night cruising in manual mode.
6 STAGE OF THE RESEARCH
For now, we have already realized the necessary
phases of CWA in the situation of lane change, one of
our use case apart from manual driving in night and
transition from autonomous mode to manual mode.
6.1 Work Domain Analysis Application
to Lane Change
There is a sparse literature on Work Domain Analysis
on Road Driving. Some authors have realized this
analysis. Stoner, Wiese and Lee (2003) have applied
the Abstraction Hierarchy analysis to the driving
domain. They identified information requirements for
drivers. Salmon, Regan, Lenné, Stanton and Young
(2006) have realized Work Domain Analysis of the
road transport system in Victoria, Australia. These
particular works present a general application to the
whole domain of the driving, including the total
system, the subsystems and the components.
There are several steps in WDA methodology.
Step 1: WDA purposes
There we consider the problem definition and the
approach to address the problem. For this thesis, the
purpose of WDA is related to the information
requirements of driver whilst driving in an
autonomous vehicle of level 3. Particularly, we pay
attention to lane changing in autonomous mode,
“night driving” in manual mode and transition from
autonomous to manual mode.
Step 2: Project constraints identification
LRA project has many constraints that do not allow
to go deeply into detail. These constraints include: the
time constraint, the expertise-related constraints.
In fact, there are few experts on the project who do
not have enough time to invest in this analysis. These
constraints “forced” us to focus on the subsystem
(driver-vehicle-road system) rather than to consider
the whole system (road transportation) and each
component.
Step 3: WDA boundaries
If boundaries are not clearly determined, WDA can
become very large, complex and not understandable.
That is why we consider highways and motorways as
roads where the vehicle can drive. This choice helps
to not consider potential obstacles like pedestrians.
Step 4: Constraints nature identification
Naikar and al. (2005) have identified 5 categories of
work systems within a causal-intentional continuum
where the focus system falls. In our work, our focus
system can be from the first category “Automated
systems governed by laws of nature” in autonomous
mode, or between the third category “Systems
governed by actors’ intentions” and the fourth
category “Systems governed by actors’ personal
objectives” in manual mode. This classification let us
conclude to focus on causal constraints (Salmon et al.,
2006).
Step 5: Information sources identification
To realize WDA, we mainly use documents sources
of information: articles dealing with autonomous
vehicles, articles dealing withlane changing,
handover process and so on. Brainstorming with
some experts and legislation documentation were also
identified as sources of information.
Step 6: Abstraction Decomposition first construction
ICINCO2015-DoctoralConsortium
26
With our first reading, we construct our first
Abstraction Decomposition Space (ADS). Ordinarily,
ADS is a matrix composed of the Abstraction
Hierarchy and the Decomposition Space. We just
realized an Abstraction Hierarchy which is
decomposed in 5 levels: functional purposes, values
and priority measures, Purpose-related functions,
object-related functions and physical objects.
Decomposition Space is formed of total system,
subsystems and different components. First, we
construct a macroscopic WDA for the autonomous
driving. Then, we construct a detailed WDA related
to the lane changing considered as a unique system. It
is this WDA that we will explained into detail in the
following paragraphs.
Step 7: Abstraction Decomposition second iteration
After many reviews, we modified some elements to
obtain the final WDA for the lane change.
Figure 3 presents the WDA of the autonomous
vehicle. We present an overview of its structure
because it is beyond the scope of this paper to present
the complete ADS for the Autonomous Vehicle.
Figure 3: Autonomous vehicle Work Domain Analysis.
Figure 4: Work Domain Analysis of a “human” lane change
system.
We insist on one particular point related to the figure
3. In the purpose related-functions, there is “Vehicles
physical dynamics”. Within these vehicle physical
dynamics, there are speed, velocity, maneuverability,
etc. Lane changing is a maneuver that autonomous
vehicle can realize in any mode. Because lane
changing is one of our use case, we have decided to
construct a WDA for the lane change system which
does not exist now. We assumed that this WDA will
also help to design lane change system.
Figure 4 presents the LC WDA in manual mode.
For each level of the Abstraction Hierarchy, we
give a definition and present its components.
6.1.1 Functional Purposes
Functional purposes are the reason to be of the
system. Contrarily to goals, functional purposes are
more stable over time (Burns, Vicente, 2001). Naikar
suggests some questions to find them such as “Why
does the system exist?”, “Why is the system
necessary?”, “Which purposes should the system
achieve?” (Naikar, 2013).
A lane change has been defined as a deliberate and
substantial shift in the lateral position of a vehicle
(Chovan et al., 1994). That means that the reason of a
lane change is a shift, a movement from one lane to
another.
6.1.2 Values and Priority Measures
Values and priority measures are the criteria that help
to evaluate the system progression towards its
functional purposes (Naikar, 2013). Those criteria
help for system evaluation. They also allow to
prioritize the elements of the below level, the
purpose-related functions. To determine values and
priority measures, Naikar suggests some questions
like “What criteria can be used for evaluating how
well the system is fulfilling its functional purposes?”,
“What fundamental laws, principles, or values must
be respected by the system?” (Naikar, 2013).
Considering a lane change system, we have found
three main values and priority measures:
a) Optimize lane change duration: A lane change
longs between 3.5 and 8.5s with a mean of 5.8 s
on the highways (Tijerna, 1997). Hetrick studies
have estimated lane change duration between 3.4
and 13.6 s with of 6 s (Hetrick, 1997). We assume
that a lane change system should realize the
maneuver in a time interval [3.4 s; 7 s] to ensure a
certain level of safety. Considering the dynamic
characteristic of the road environment, a
hypothesis is that more the lane change will last,
more the maneuver will be dangerous.
AugmentedRealityInterfaceDesignforAutonomousDriving
27
b) Minimize maneuver risk and severity of
potential accidents: This measure completes the
first one. In fact, the danger of a lane change is not
just related to its duration but also by the severity
of crashes that can occur because a lack of
adjustment in the maneuver. Then, a good
criterion is to evaluate the number of accidents
caused by the change system with an expectation
of a zero accident observation.
c) Optimize ego vehicle driver comfort: The safety
is the main aspect to consider in a lane change.
When we consider also comfort aspect, it is better.
In this configuration, a cognitive calculation is
made in driver’s head to respect the security and
performance constraints. He will want to do the
maneuver and be comfortable at the same time.
We have already projected to evaluate comfort
through a qualitative analysis of users’ tests that
will be carried out on the simulator.
6.1.3 Purpose-related Functions
Located in the middle of the hierarchy, the purpose-
related functions refers to functions that a system
must accomplish to achieve the functional purposes.
They can be seen as the “uses” that physical objects
and their object-related processes “put to” in a system
(Miller and Vicente, 1998, p.15 cited by Naikar,
2013, p72). Those functions can also influence values
and priority measures (Stanton, 2014). for a lane
change system, we have identified the following
functions:
a) Scan the environment
b) Detect near objects around the ego vehicle
c) Monitor near objects around the ego
vehicle
d) Understand near objects intentions around
the ego vehicle
e) Evaluate gaps between closest vehicles and
the ego vehicle (speed differential, distance
differential, etc.)
f) Adapt style maneuver
6.1.4 Object-related Functions
The object-related processes, which are highly
dependent on the properties of physical objects, serve
the system to achieve its purpose-related functions
(Naikar, 2013). To find those processes, we can
answer to some questions suggested by Naikar:
“What can physical objects of relevance to the system
do or afford?”,”What functional purposes or
functional capabilities of physical objects are
necessary for the system to achieve its
purpose-related functions?” (Naikar, 2013, p.182).
We have extracted some object-related functions
for a naturalistic lane change:
a) Collect and store road characteristics: Road
characteristics refer to access type (freeway,
highway, arterial, ramp, secondary road), road
type (weaving, rural) and number of lanes.
b) Collect and store the ambient characteristics:
Ambient characteristics; refer to the weather (rain,
sun, ice snow, cold), the visibility (sun, dust, rain
fog) and he time of day.
c) Collect and store the road traffic signs and
signals: In the traffic signs, we include, speed
limitation traffic signs, direction traffic sign,
traffic markings and stationary cameras/police
cars.
d) Collect and store the location of the ego vehicle
and the others vehicles.
e) Collect and store the speed of the ego vehicle
and those of others vehicles.
f) Identify the type of the near vehicles (truck,
bus, etc.).
g) Realize the maneuver according to the
acceleration and braking maximal capabilities
of the ego vehicle.
h) Adjust lateral control and longitudinal control
of the ego vehicle.
i) Detect turn signals activation of others
vehicles.
6.1.5 Physical Objects
It is the level the most concrete of the hierarchy. In
this level, all the physical objects present in the
system are concerned. They have functional
capabilities that allow the system to accomplish the
object-related processes.
For our use case, we did not go deeply into the
detail because the hard “part” is out the scope of our
work. We have listed the main elements.
a) Global Positioning System (GPS)
b) Pedals
c) Steering Wheel
d) Turn signal
e) Others sensors and actuators: radar,
LIDAR, Adaptive Cruise Control, Lane
Departure Warning,
6.2 Control Task Analysis Application
to Lane Change
Control Task Analysis can be seen as an analysis of
ICINCO2015-DoctoralConsortium
28
Figure 5: Decision ladder template applied to lane change system.
the activity in terms of decision making. It does not
answer the question related to how the activity should
be realized nor the question of who is accomplishing
the activity. Like we have said before, there are many
activities that can occurred in a particular work
domain. There, we will analyze the lane change
activity from the beginning to the end.
6.2.1 The Decision Ladder Template
The decision ladder is comprised with annotations
that come from Rasmussen and Vicente papers
(Rasmussen, 1976; Vicente, 1999). It is composed of
rectangles and ovals. Rectangles are the boxes of
information processing. Ovals are knowledge states
which can be either the inputs either the outputs of the
rectangles. The writings near the oval are formulated
like questions to show actors reasoning (Elix and
Naikar, 2008). The ladder must not to be linearly
read. The result of the lane change decision ladder is
represented in figure 5.
6.2.1.1 Goals
At the top of the ladder, the first state of knowledge
is the one of goals. This state consists in the system
goals definition. It is slightly similar to the first level
of Abstraction Hierarchy, the functional purposes.
Literature advises the following formulation for this
circle “Goal that begins with a verb + constraints”
(Elix and Naikar, 2008; Jenkins, Stanton, Walker and
Rafferty, 2009).
For the lane change system, we formulate the goal
as move from one original lane to a destination
lane safely and efficiently, by taking into account
the time windows and the navigational
constraints.
6.2.1.2 Alert
After the goals definition, we consider the circle at the
bottom of the left branch of the ladder: the alert. The
alert refers to all the elements that can trigger an
event.
In our case, it refers to all the causes that can
trigger a lane change. According to Lee, Olsen,
Wierwille and Naranjo, the following question can be
asked to determine the alert (Lee, Olsen, Wierwille,
2004; Naranjo, 2008; Olsen, 2003).
A1: Should I enter in a highway?
A2: Should I prepare to exit the highway or
should I exit now?
A3: Should I anticipate a vehicle merging?
AugmentedRealityInterfaceDesignforAutonomousDriving
29
A4: Should I avoid obstacles like works on the
road or crash situations?
A5: Should I anticipate a vehicle coming fast
behind?
A6: Should I return to a preferred lane?
A7: Is there a lanes number decrease?
A8: Is there a lanes number increase?
A9: Is there a slow vehicle in front of me?
A10: Is the distance differential between the
preceding vehicle and I increasing?
6.2.1.3 Information
There we find all the information necessary to
evaluate the alert. For example, Stanton and Bossell
(2014), while studying how a submarine returns to
periscope depth, enumerated in the information level,
the surface constraints, weather constraints, etc.
We have formulated questions to grasp the
necessary information.
I1: What is the lateral position of the ego vehicle
in its lane?
I2: What is its speed?
I3: What are the others vehicles location (in one
lane, between two lanes with the lateral
displacement) in the ego lane and in the adjacent
lanes?
I4: What are their speeds?
I5: What are the others vehicles kinds (truck, city
cars, bus, ambulance, etc.)?
I6: What are the distances between those vehicles
and the ego vehicle (distance proximity)?
I8: What are the infrastructure constraints (speed
limitation, markings, distance to the next exit,
etc.)?
I9: What is the road configuration (curve, straight,
etc.)?
I10: What are the weather conditions?
I11: Are the turn signals of the near vehicle are
activated?
6.2.1.4 System States
Analysis and fusion of information allow knowing the
state in which the system is (Jenkins et al., 2009).
Because of the diversity of information in terms of
nature, quantity and understanding, there are several
system states.
Lane change can be realized or not, depending of
the relations between the subject vehicle also called
ego vehicle and the others.
S1: What are the intentions of the drivers around
me?
S2: What are the trajectories of the vehicles or
their current actions?
S3: Are the safety gaps respected between the
ego vehicle and the others ones?
S4: Are the speed differential and the distance
differential sufficiently safe to permit me to do a
lane change?
S5: Do I have to take an exit in a few seconds
(Spatial constraints)?
S6: Is it possible to realize a lane change while
respecting speed limitation?
S7: Do the weather conditions influence the lane
change (visibility)?
6.2.1.5 Options
Towards a system state, many actions can be carried
out to achieve the purpose of the system. The number
of actions to realize are deeply related to the system
state. We assume that less complex a system will be,
less action will need to be realized. Jenkins suggests
to formulate the elements at this level by: “Is it
possible…?” (Jenkins, 2009).
O1: Is it possible to realize a quick lane change
(because of a weak maneuver margin)?
O2: Is it possible to realize a safe lane change?
O3: Is it possible to urgently realize a lane change
(because of an imminent crash)?
O4: Is it possible to pursue a cruising in the same
lane?
O5: Is it possible to realize an overtaking?
6.2.1.6 Chosen Goal
As we mentioned before, a system can have one or
many goals. But at a particular time, just one goal can
be elicited because of the environment constraints
(Elix, Naikar, 2008 cited by Jenkins et al., 2009).
In our case, the chosen goal is the same that the
goal because there is only one goal: “move from one
original lane to a destination lane safely and
efficiently, by taking into account the time windows
and the navigational constraints.”
6.2.1.7 Target State
When an option is chosen, it becomes the target state.
The target state can be formulated by “Is (option) can
be adopted?” (Jenkins et al., 2009).
T1: Lane change is fast.
T2: Lane change is safe.
T3: Lane change must be urgently realized.
T4: Lane change can be realized now (temporary
cruising).
T5: Lane change cannot be realized
(Undetermined cruising).
ICINCO2015-DoctoralConsortium
30
6.2.1.8 Task
It is the task and a group of tasks to realize in order to
achieve the goal(s).
In the lane change system, the main task is to
define the trajectory and follow it.
6.2.1.9 Procedure
It is the procedure that has to be conducted in order to
realize the tasks.
P1: Monitor the road environment.
P2: Activate turn signal to notify the lane change,
beginning.
P3: Come near the line that separates the original
lane and the destination lane.
Accelerate to precede the vehicle that is front if
necessary.
P4: Deactivate the turn signal.
P5: Stabilize the speed and the ego vehicle
position in the destination lane.
P6: Follow the trajectory.
This procedure can be reiterated if necessary.
6.3 Information Requirements for
Interface Design
The information that is presented by an interface
should be the appropriate information in the
appropriate shape. Then the question of form of the
information and its quantity are critical for drivers’
situation awareness and workload.
The WDA and the CTA help us to extract
information requirements independently of the
autonomous system. The categories of information
based on those analyses are listed below:
For the ego vehicle
o Vehicle condition: fuel level
o Vehicle component conditions: if
they work well or not
o Current location and desired end
point
o Own dynamics: speed, velocity,
lateral displacement
For the others vehicle
o heir type
o Their location
o Their relative proximity with the
ego vehicle in terms of speed,
distance or time
o Their intentions
o Their actions
For the infrastructure
o Road signage presentation
o Road type
o Infrastructure related Warning
o Lane width
There are also information requirements related to the
automated system and the ego vehicle driver.
Considering that autonomous mode should be
enjoyable time where the driver will have access to
the information he wants to have access to, driver
should know for example, how long he has to plan its
leisure activity. It is also necessary to check driver
vigilance especially when the transition from
autonomous to manual mode will be required. Other
information should be conveyed:
For the automated system
o Its intentions
o Its current action
o Its comprehension of road rules
o Its road perception
o Remaining time in autonomous
mode
For the ego vehicle driver
o Its distraction level
o Its protection information:
information related to the seatbelt
lock or unlock, hands on steering
wheel or not, feet on pedals or not.
Some Intelligent Transport Systems provide those
information but for some, not in their entirety.
Consequently, the next step will be to determine
which information will be design in Augmented
Reality.
Comments. At this stage of the work, we have
realized the most critical part of the work. We have
spent much time to do it but it was really important to
have a convenient work. Now we are ready to go into
the prioritization and rules levels for a first users’ test
in December.
7 CONCLUSIONS
To design an interface, precise and adequate
information are timely needed, especially in
autonomous mode for driving. In this paper, we have
described our methodology for interface design. We
have finished the first step of Cognitive Work
Analysis to capture information needs in lane change
and associate them with our use cases.
We are thinking on information representation
and the level of transparency of the interface. This
methodology, derived from a cognitive approach, will
lead to a set of rigorous rules. Those rules will allow
AugmentedRealityInterfaceDesignforAutonomousDriving
31
at specific time specific components to appear either
in Augmented Reality form or not.
Acknowledgements. The thoughts expressed here are
the work of the authors, but related work has been
supported by French government in conformance
with the PIA (French acronym for Program of
FUTURE Investments) within IRT (French acronym
for Technologic Research Institute) SystemX.
ACKNOWLEDGEMENTS
The thoughts expressed here are the work of the
authors, but related work has been supported by
French government in conformance with the PIA
(French acronym for Program of FUTURE
Investments) within IRT (French acronym for
Technologic Research Institute) SystemX.
REFERENCES
Azuma, R. T. (1997). A survey of augmented reality.
Presence, 6(4), 355-385.
Burns, C. M., & Hajdukiewicz, J. (2013). Ecological.
interface design. CRC Press.
Chen, J. Y. C, Boyce, Wright, J., Procci, K., Barnes, M. (in.
prep.). SA-based Agent Transparency. ARLTechnical
Report.
Chovan, J. D., Tijerina, L., Alexander, G., &
Hendricks, D. L. (1994). Examination of lane change
crashes and potential IVHS countermeasures (DOT HS
808 071) Washington, DC: National Highway Traffic
Safety Administration. Available online at:
http://www.itsdocs.fhwa.dot.gov/\JPODOCS\REPTS_
TE/61B01!.PDF.
Elix, B., & Naikar, N. E. E. L. A. M. (2008) Designing safe
and effective future systems: A new approach for
modelling decisions in future systems with Cognitive
Work Analysis. In Proceedings of the 8
th
international
symposium of the Australian Aviation Psychology
Association.
Endsley, M. R. (1995). Toward a theory of situation
awareness in dynamic systems. Human Factors: The
Journal of the Human Factors and Ergonomics Society,
37(1), 32-64.
Endsley, M.R., & Kaber, D.B; (1999). Level of Automation
effects on performance, situation awareness and
workload in dynamic control task.
Jenkins, D. P., Stanton, N. A., Salmon, P. M., Walker, G.
H., & Rafferty, L. (2010). Using the decision-ladder to
add a formative element to naturalistic decision-making
research. Intl. Journal of Human Computer Interaction,
26(2-3), 132-146.
Gasser, T., & Westhoff, D. (2012, July). BASt-study:
Definitions of automation and legal issues in Germany.
In Presentation at the Road Vehicle Automation
Workshop.
Habenicht, S., Winner, H., Bone, S., Sasse, F., &
Korzenietz, P. (2011, June). A maneuver-based lane
change assistance system. In Intelligent Vehicles
Symposium (IV), 2011 IEEE (pp. 375-380). IEEE.
Hetrick, S. (1997). Examination of driver lane change
behavior and the potential effectiveness of warning
onset rules for lane change or "side" crash avoidance
systems. Unpublished master's thesis, Virginia
Polytechnic Institute and State University, Blacksburg,
Virginia. Available on-line at:
http://scholar.lib.vt.edu/theses/available/etd
382915749731261/unrestricted/etd.pdf.
Lee, S. E., Olsen, E. C., & Wierwille, W. W. (2004). A
comprehensive examination of naturalistic lane
changes (No. HS-809 702,).
Marinik, A., Bishop, R., Fitchett, V., Morgan, J. F.,
Trimble, T. E., & Blanco, M. (2014, July). Human
factors evaluation of level 2 and level 3 automated
driving concepts: Concepts of operation. (Report No.
DOT HS 812 044). Washington, DC: National
Highway Traffic Safety Administration.
Michon, J. A. (1985). A critical view of driver behavior
models: what do we know, what should we do?. In
Human behavior and traffic safety (pp. 485-524).
Springer US.
Naikar, N. (2013). Work domain analysis: Concepts,
guidelines, and cases. CRC Press. National Highway
Traffic Safety Administration. (2013). Preliminary
statement of policy concerning automated vehicles.
Washington, DC.
Olsen Erik (2003). Modeling Slow Lead Vehicle.
Changing. Dissertation submitted to the faculty of the
Virginia Polytechnic Institute and State University in
partial fulfillment of the requirements for the degree of
doctor of philosophy in Industrial and Systems
Engineering. 22 September 2003, Blacksburg, Virginia.
Available at: http://scholar.lib.vt.edu/theses/available/
etd12032003152916/unrestricted/olsen_dissertation.pdf.
Riley, V. (1989, October). A general model of mixed.
initiative human-machine systems. In Proceedings of
the Human Factors and Ergonomics Society Annual
Meeting (Vol. 33, No. 2, pp. 124-128). SAGE
Publications.
Sheridan, T. B., & Verplank, W. (1978). Human and
Computer Control of Undersea Teleoperators.
Cambridge, MA: MA: Man-Machine Systems
Laboratory, Department of Mechanical Engineering,
MIT.
Sheridan, T. B., & Parasuraman, R. (2005). Human
automation interaction.Reviews of human factors and
ergonomics, 1(1), 89-129.
Shinar, D. (1978). Psychology on the Road: The Human
Factor in Traffic Safety. New York: Wiley.
Smith, J., 1998. The book, The publishing company
London, 2nd edition.
Tijerina, L., Garrott, R, W., Glecker, M., Stoltzfus, D., &
Parmer, E. (1997, November 3). Van and passenger car
driver eye glance behavior during lane change decision
ICINCO2015-DoctoralConsortium
32
phase (Revised DRAFT: INTERIM REPORT).
Transportation Research Center, Inc. and National
Highway Transportation Safety Administration,
Vehicle Research and Test Center.
Van Winsum, W. (1999). The human element in car
following models. Transportation research part F:
traffic psychology and behaviour, 2(4), 207-211.
AugmentedRealityInterfaceDesignforAutonomousDriving
33