Evaluation of a New System in Future L4 Vehicles: Use Cases and

Methodology for the SUaaVE European Project

Davide Salanitri, James Jackson and Cristina Periago

APPLUS+ IDIADA Automotive Technology SA, L’albornar, Santa Oliva, Spain

Keywords: Human Factors, Automotive, Automated Vehicles, Use Cases, L4 Vehicles.

Abstract: The current era is experiencing an epochal technological advancement in the automotive industry, with

electrification, automation and connectivity driving research and innovation. Various disciplines involved in

the design and development of vehicles are affected by this progress and new aspects, such as acceptance,

trust, ethics and emotions which will become fundamental components of the market. All these aspects will

be investigated in the project SUaaVE. SUaaVE is a European project aiming at improving the users’

acceptance, trust and comfort of future Level 4 automated vehicles by involving the user in the development

of the concept of a new system called ALFRED. This paper will describe the methodology developed for the

evaluation of ALFRED, by representation of future vehicle use cases to be used as a means for assessment.

1 INTRODUCTION

The introduction of autonomous vehicles is set to

disrupt the automotive industry and market (Diels et

al., 2017). Without the driver, future research in

almost all the fields concerning automotive will have

to change its point of view (Diels et al., 2017). New

aspects of, for example, comfort, security, ethics and

design must be introduced. This paper aims at

expanding the knowledge on future automated

vehicles. More in details, it describes the

methodology for the validation of a new conceptual

system for a L4 automated car. This system, called

ALFRED (Automation Level Four+ Reliable

Empathic Driver) is being theorised in the European

project SUaaVE (SUpporting acceptance of

automated VEhicle). The system aims to be a user

centred technology able to adapt to the user state

considering concepts like acceptance and ethics. This

work is focused on the construction of the use cases

and the methodology underlying the evaluation of

ALFRED concept.

This paper refers to L4 vehicles. L4 vehicles have the

capability of being full automated and do not require

a driver to be present. However, the vehicle has

limitation in its ability to drive autonomously in

aspects like type of roads, type of environment or

general conditions. L4 cars are able to detect the

limitations and safely manage any situation where it

is not possible to drive autonomously. The conditions

when L4 vehicles drive autonomously is known as an

Operational Design Domain (ODD).

1.1 The SUaaVE Project

SUaaVE aims at solving the gap between

technological advancement and public acceptance by

adopting a Human-Driven Design (HDD) approach,

enhancing synergies in social science, human factors

research and automotive market by means of an

iterative process of assessment, co-design and

prototyping. The main outcomes will be:

1. A new paradigm of automation: ALFRED, that

aims at “colouring the decision-making processes of

the CAV with human emotions” with: (a) an

EMpathY Unit to understand the emotional and

cognitive state of the passenger and (b) an Adaptive,

Cognitive and Emotional Interface with a set of

services (vehicle dynamics, ambient and postural

comfort) to enhance passenger experience.

2. An immersive Virtual Human Centred Design (V-

HCD) platform, allowing the simulation of CAV.

3. Guidelines to support Public Authorities,

representing a breakthrough in the public acceptance

of future CAVs for both the society and for all road

users.

This paper is concerned with the first point, with this

being focussed around the methodology for the

evaluation of ALFRED concept.

Salanitri, D., Jackson, J. and Periago, C.

Evaluation of a New System in Future L4 Vehicles: Use Cases and Methodology for the SUaaVE European Project.

DOI: 10.5220/0010184302710275

In Proceedings of the 4th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2020), pages 271-275

ISBN: 978-989-758-480-0

271

1.2 The ALFRED Concept

ALFRED concept is intended to intelligently respond

to users’ states by means of a series of models, whose

development forms a key part of the SUaaVE project.

These models set out to take a series of user-based

components into account, with the intention of

adapting the system in a way that will improve

vehicle behaviour across several different functions.

The ALFRED concept will control these varied

components considering user acceptance, ethics,

emotions, and comfort.

2 USE CASES

A key part of the development of any product or

service involves careful thought of how it will be

adopted and used once released into the real world.

Within the field of Human Factors, this process

closely considers the needs of the end user and their

points of interaction (Wilson and Sharples, 2015).

This methodology provides a mean by which design

and development can consider real scenarios under

which something might be used by creating solutions

that are relevant to end users. Use cases define

generic and/or specific scenarios under which a

product or service eventually be used. They are often

comprised of a series of steps which define typical

events based around interactions between a user and

a system (Möller, 2014).

This method will be used for the evaluation of

ALFRED. Throughout development of the system it

is necessary to offer relevant scenarios to its eventual

use. In terms of research studies conducted, this

involves investigation around components of

acceptance, ethics, emotions, and comfort.

Use cases must offer controlled scenarios with

defined variables, whilst maintaining the context of

eventual use. Each of the use cases must offer

definitions for users, context of use, the environment,

and events.

2.1 Definition of the Operational

Domain

The specification of use cases will follow some

general guidelines set out by the definition of an

ODD.

The proposed components of the ODD for the

SUaaVE are as follows:

• Rules for the physical boundaries where

vehicle is able to operate (on what roads can

it travel)

• Rules for vehicle journeys (level of user

control, pick-up and drop-off locations and

possibilities for route override)

• Rules for operational conditions (weather

and lighting conditions)

• Rules/norms for interaction with other road

users (external) – common method of

communication between autonomous

vehicles and other road users, within the

operational domain

2.2 Definition Process for Use Cases

The first step in the definition of the use cases process

is the definition of external factors, which is the

definition of the factors affecting ALFRED’s

operation and its occupants.

The definition of the external factors takes place in

parallel to the development of the ODD. The external

factors can be grouped by:

• Variable Environmental Factors. (E.g.

route, operational zones, obstacles.)

• Fixed Environmental Factors. (E.g.

visibility, temperature, weather, traffic)

• Variable Vehicle Operation Factors. (E.g.

speed, dynamic behaviour)

• Fixed Vehicle Operation Factors. (E.g.

dynamic, visibility, comfort)

• Situational User Factors (Variable). (E.g.

journey purpose, activities)

• User Profile Factors (Fixed). (E.g.

personality, preferences, experiences,

physical capability, cognitive capability)

2.2.1 User, Context, Environment and

Events

Regarding the type of users who will interact with

ALFRED, they were divided into two groups:

primary and secondary.

The primary users are the ones directly interacting

with the vehicle. These are the users inside the vehicle

when ALFRED is functioning.

The secondary users are users not directly interacting

with ALFRED, such as Virtual Road Users (VRUs),

other vehicles drivers, service users and transport

management personnel.

The journey context characteristics are:

• The start and finish location.

• The journey stop(s).

• Purpose of the journey

SUaaVE 2020 - Special Session on Reliable Estimation of Passenger Emotional State in Autonomous Vehicles

272

• User activity before/after the journey.

• User activity during journey.

The interaction between the vehicle and the

environment will affects the user experience,

emotion, acceptance, trust and comfort. During the

scenarios, ALFRED will respond to change in the

operational environment, and this will trigger

different perception by the users. In details, the

operational environment is composed by:

• The physical ODD zones.

• Domain rules and norms: regulations and the

general behaviour patterns within the

operational environment.

• Weather.

• Infrastructure.

• Road condition.

The events happening during the scenario have been

developed in order to trigger controlled reaction to the

user emotional state and perception. More in depth,

the events are used to investigate the reaction that the

users have to certain situations. There are two types

of events, generic and specific.

Generic events refer to the type of situations that are

usual and often occur during a normal journey. They

will be comparable between use cases.

Specific events will be events that do not usually

occur in a normal journey and are proposed to trigger

change in the user status (emotional, psychological

and physical).

2.3 Distinction between Use Case Types

As previously stated, there is a demand for different

types of use cases dependant on specific application.

Within SUaaVE, use cases are to be applied across

multiple investigative and developmental phases,

with distinct applications depending on stage and

methodology.

To meet all of the demands for investigation,

development, evaluation, and demonstration, a

system of categorisation for use cases have been

defined from the outset.

1. Trip use cases

Trip use cases encompass user journeys and define

scenarios from a standpoint of generalised use.

Following the process set out above, they comprise of

a definition of specific users, a context for use, and a

defined environment. Underneath this, are set out a

series of individual events which take the form of a

long scenario. The events defined under a trip use

case are applicable to the scenario context and specify

general occurrences.

These longer use cases can be used when carrying out

assessment regarding the whole system. This is done

by presenting a real-world applicable scenario.

2. Situational use cases

Situational use cases provide the tightly defined and

controllable scenarios required by scientific

investigation and provide the means for specific

demonstration and evaluation. They include defined

users, specified context, and information regarding

the environment. The sequence of events within them

defines more specific occurrences, although these still

join to form a scenario.

Situational events within the use case framework fall

under two main categories:

a. General Events

General events are taken directly from the scenario

defined within the trip use cases. These are applied

within all the longer journey scenarios, although the

exact sequence is dependent on the specific use case.

They define normal occurrences within the use of the

ALFRED vehicle which, although intended to

provoke a user response, will always be regarded as

common happenings.

b. Exceptional Events

Exceptional events define scenarios that can be

regarded as falling outside of normal operation. These

might involve clear discretions by other transport

users or could involve situations in which the vehicle

leaves its functional domain or encounters an error in

its operation. Being situational use cases, exceptional

events always from short scenarios which are

applicable for specific experimentation or for

demonstration and evaluation of special operating

conditions for the ALFRED concept.

2.4 Example of ALFRED Use Cases

Following the method described in the section above

and after the contribution of all the partners for the

SUaaVE project, the ALFRED use cases have been

developed and included as a reference for both the

first and second loop of evaluation. An example of a

trip use case and of a situational use case are depicted

in the images below (Figure1 and Figure 2).

More in details, figure 1 details all the events that

have been chosen to be part of the scenarios. As

explained in the section above, in the situational use

cases, one or more events can be included and the use

cases can be different depending on which specific

aspect is under investigation.

Evaluation of a New System in Future L4 Vehicles: Use Cases and Methodology for the SUaaVE European Project

273

Figure 1: Example of a situational use case.

Trip use cases are more defined and are the same for all

partners. Figure 2 depict an example of a use cases that

has been proposed. The trip use cases will be prone do

modifications depending on the results of the first loop of

evaluation (section 3).

Figure 2.

3 USE CASES AS PART OF THE

EVALUATION FRAMEWORK

The evaluation framework is based upon the testing

of ALFRED with a comparison being made against a

normal connected automated vehicle (CAV) through

multiple phases.

The use cases forming the evaluation framework will

provide detailed definitions joined to the end use of

the vehicle to provide a relatability to the test

outcomes. The use cases will be the reference for the

construction of scenarios for the evaluation of the

acceptance of the vehicle by the users, the emotions

triggered by the events, and the ethics implications in

the use of the vehicle.

At a high level, assessment will comprise of two main

phases of testing known as first loop and second loop.

These two phases encompass formative testing of the

models and of vehicle systems, and subsequently

provide the basis for summative testing following

completion of their development. These two loops

can be viewed following the aforementioned concept

of situational use cases and use cases defining

journeys

3.1 First Loop of Evaluation

The outcomes of this first loop testing will provide a

series of comparative data between the models

developed for integration into ALFRED and the

corresponding response of the CAV. In addition, it

involves the first assessments of the ACE interface

and the performance of the dynamic model, both of

which will be individually evaluated with comparison

made to the CAV response under the same conditions.

These will be the basis for conclusions on the success

of the projects’ first developmental phase. In line with

the project aims, this will be with regards to the way

that models can respond to and manage specific

situations in terms of the users.

In this case, the use of situational use cases offers the

means for evaluation under a specific scenario with

closely defined conditions. Like with the

investigative and development phase, this will

provide controllable and repeatable variables within

defined test scenarios.

3.2 Second Loop of Evaluation

The second loop will take a more global view of the

system assessment, seeking to evaluate how the

system performs in terms of users when experiencing

its use across an entire trip. Like with the first phase

this will include testing against the CAV, but with

more outcomes given more focus regarding to how

the ALFRED concept might perform in the real

world. This stage will include the final evaluations of

the vehicle system, with conclusions assessing the

overall success of the concept.

With this in mind, use cases for second loop testing

will principally take the form of trip scenarios which

are representative of an entire user journey.

4 CONCLUSION

The advancement in the automotive industry require

a massive and rapid change of focus for most of the

disciplines involved. The transformation of the driver

SUaaVE 2020 - Special Session on Reliable Estimation of Passenger Emotional State in Autonomous Vehicles

274

to a passenger require new systems, new design and

new features, both from the technological point of

view and the user centred part. On the Human Factors

point of view, additional aspects such as acceptance,

ethics and trust must be taken into consideration and

other attributes like comfort must be rethought. The

SUaaVE project aims to investigate a new concept for

L4 autonomous cars, a system that adapt to the users’

emotional state, and considers acceptance, trust,

ethical issues and comfort. This system concept,

called ALFRED, is currently under development.

This paper described the proposed methodology to

evaluate it. More in details it described the process of

construction of the use cases, useful to contextualise

the system in a possible real-life scenario and to elicit

specific emotions through predetermined events. The

next phase of the project will include the formative

evaluation (first loop) in driving simulators and

through videos with potential users. The results of the

formative evaluation will guide the further

development of the system and will lead to the

summative evaluation (second loop).

ACKNOWLEDGEMENTS

This project has received funding from the European

Union’s Horizon 2020 research and innovation

programme under grant agreement No. 814999. The

content of this publication is the sole responsibility of

the authors, and in no way represents the view of

INEA or European Commission.

REFERENCES

Diels, C., Erol, T., Kukova, M., Wasser, J., Cieslak, M.,

Payre, W., Miglani, A., Mansfield, N.J., Hodder, &

Bos, J. (2017). Designing for comfort in shared and

automated vehicles (SAV): a conceptual framework.

Presented at the 1st International Comfort Congress

(ICC2017), Salerno, Italy, June 7-8

Möller D.P.F. (2014) Transportation Use Cases. In:

Introduction to Transportation Analysis, Modeling and

Simulation. Simulation Foundations, Methods and

Applications. Springer, London. (pp. 229-334).

Springer, London

Wilson, J. R., & Sharples, S. (Eds.). (2015). Evaluation of

human work. CRC press.

Evaluation of a New System in Future L4 Vehicles: Use Cases and Methodology for the SUaaVE European Project

275