A SIMULATION ENVIRONMENT TO EVALUATE DRIVER

PERFORMANCES WHILE INTERACTING WITH TELEMATICS

SYSTEMS

Gennaro Costagliola, Sergio Di Martino, Filomena Ferrucci

Dipartimento di Matematica e Informatica, Università degli Studi di Salerno,via Ponte don Melillo, Fisciano (SA), Italy

Keywords: Human-Computer Interaction, Human Factors, Virtual Reality, Vehicular Telematics Systems.

Abstract: The evaluation of user interfaces for vehicular telematics systems is a challenging task, since it is necessary

to understand the effects of interaction on driving performances. To this aim, in 2005 we developed and

presented a framework specifically conceived for the indoor evaluation of these systems. In this paper we

present some significant improvements of that proposal. In particular, we describe a graphical analysis tool

able to provide a clear and deep insight about driver behaviors using the high amount of data generated by

the simulator. Moreover, we report on the evaluation analysis that has been performed to assess the

effectiveness of the framework for measuring driving performances.

1 INTRODUCTION

Researches on User Needs report that people want to

be connected any time and any place, even in their

cars (Comunicar, 2002; Microsoft, 2005). To answer

this requirement, information and communication

technologies have been fitted into automobiles,

giving rise to Vehicular Telematics Systems (VTSs),

which represent the in-vehicle convergence of

mobile communications and information processing,

allowing drivers and passengers to stay in contact

with the world outside their car. VTSs allow drivers

to exploit a plethora of features, such as multiple

audio sources (MP3, DAB, and DVD), web

browsing, e-mails, phone calls, voice control, and so

on. Moreover, most advanced VTSs (e.g.: BMW

iDrive, Fiat Connect+ or GM onStar) are starting to

provide services, i.e. advanced functionalities

involving interaction with a support centre. Typical

examples are remote vehicle diagnosis, dynamic

route calculation (taking into account road, vehicle,

traffic and weather conditions), tele-aid, hotel

reservations, etc…

However, the diffusion of VTSs is also causing

concerns about road safety, since these systems can

heavily increase driver’s mental distraction (Burns,

2001; Tijerina, 2001), which is widely recognized as

the most prevalent cause of crash (Toms, 2001).

Thus, since road safety is paramount, it is a short

term priority to limit driver distraction inducted by

VTSs, by enhancing the usability of these systems.

Human-Computer Interaction in the automotive

domain can be considered a new and open research

area (Marcus, 2004). Specific tools and approaches

are required, mostly to assess the visual/cognitive

workload inducted by these systems and to

understand the effects of VTS interaction on driver-

vehicle performances.

Currently, several universities, companies and

research centers, have equipped laboratories with

sophisticated driving simulators, able to simulate a

high variety of physical phenomenon, ranging from

the kinematics effects inducted by different

suspension geometries, to very complex traffic

scenarios (for instance, the NADS (NADS, 2005)).

Some of them are used also for indoor evaluations of

VTSs, but these laboratories usually cost hundreds

of thousands of dollars and are very demanding to

set-up (Green, 2003), being prohibitive for small

institutions specifically focused on HCI research.

Thus, there is a strong needing for simpler VTS

evaluation systems, able in the meantime to assure

high-quality reports. To address this issue, in

(Costagliola, 2005) we proposed a framework

specifically conceived for the indoor evaluation of

VTSs User Interfaces (UIs). The goal of this

framework was to support researchers in an easy

collection of valuable data on driver’s behaviors

(and thus on mental workload), being in the

Costagliola G., Di Martino S. and Ferrucci F. (2006).

A SIMULATION ENVIRONMENT TO EVALUATE DRIVER PERFORMANCES WHILE INTERACTING WITH TELEMATICS SYSTEMS.

In Proceedings of the Eighth International Conference on Enterprise Information Systems - HCI, pages 3-10

DOI: 10.5220/0002498400030010

 SciTePress

meantime cost-effective, by requiring standard

hardware and simple set-ups.

In this paper we describe some improvements of

the framework and report on its evaluation. One of

the most distinguishing features offered by the

proposed framework was the possibility to assess the

navigation assistance provided by VTS, by

automatically generating virtual test tracks starting

from a VTS cartographical database. At the best of

our knowledge, currently no other simulation

facilities offer this specific but important feature,

compelling to evaluate on the field the VTS

navigation assistance module. In the following we

will describe how we have improved the realism of

generated tracks starting from a VTS cartography.

Indeed, the current version of the system is able to

reproduce much more realistic scenarios, taking into

account road types, contextual information, etc… As

a result, the realism and the sense of immersion in

the virtual scenario experienced by the subjects, that

represents a crucial feature for a simulation

environment, has been improved.

Secondly, to handle the huge amount of

experimental data collected per session, we

developed an apposite application, named Telemetry

Analyzer Tool (TAT). The aim of this tool is to

support the data analyst in understanding the

relationships occurring between VTS interactions

and driving performances, by providing a clear,

graphical representation of the subject behaviours,

such as input on controls, followed trajectory, etc…,

related to the timeline of the test. This is a highly

demanded feature (Barr et al., 2003), leading to the

generation of high-quality, repeatable analysis and

reports.

Moreover, we have carried out an evaluation

analysis meant to verify the effectiveness of the

proposed approach for measuring on-road driving

performances. To do that, we have employed a set of

sixteen subjects. The conducted tests report that after

about 20-25 minutes of training, subjects behave on

the simulator in a way similar they drove a real car,

thus supporting the validity of the simulator.

The remainder of the paper is structured as

follows. In section 2 we introduce the main aspects

to consider when dealing with driver distraction, as

well as the approaches used to evaluate it. In section

3 we briefly recall the main characteristics of the

evaluation framework, then we will focus on how

we integrated the driving simulator and the vehicular

navigation module. In section 4 we describe the data

analysis features we developed, while in section 5

we report on the assessment of the framework.

Finally, a discussion on final remarks and future

work will conclude the paper.

2 VTS USER INTERFACE

EVALUATION ISSUES

Usability evaluation of traditional desktop

applications can be considered an established topic

within HCI. It is based on a shared understanding of

basic concepts and extensive guidelines (e.g.

(Nielsen, 1993)).

However, these established concepts,

methodologies, and approaches in HCI are being

challenged by the increasing diffusion of ubiquitous

computing, i.e. applications executed on wearable,

handheld, and mobile computing devices. This move

beyond stationary use is requiring new approaches to

evaluate mobile UIs. Indeed, the main difference is

that mobile systems are typically used in highly

dynamic contexts, where the user is normally busy

in other primary tasks, such as walking, driving,

etc… (Lumsden, 2003).

This holds especially for the evaluation of VTS

UIs. Indeed, static evaluations of these interfaces,

performed with a subject totally focused on the

system, do not provide significant information about

the effectiveness of the UI. Instead, it is necessary to

set up a meaningful test-bed, where subjects are

mainly focused on the primary driving task and

concurrently interact with the VTS. Researchers can

evaluate driver distraction through the analysis of

some indirect indicators on vehicle dynamics.

To accomplish these tests, two approaches can be

adopted:

1. the interaction with a VTS is analyzed while the

user is driving a real car (eventually on a track

closed to the traffic), or

2. the driving is simulated in a laboratory, through

some real-time computer-generated virtual

scenarios.

Each of the two approaches presents advantages

and drawbacks. The former is more realistic,

because the subject drives a “real” car. However, it

requires the availability of a car equipped with

specific instrumentation able both to capture

information such as travel speed and lane position

and to video record the road scene and driver eye

glance (e.g. (Tijerina, 1998)), and possibly of a

closed track. Moreover, usability evaluations on the

field are not easy. Three fundamental difficulties are

reported in the literature. Firstly, it can be very

complicated to establish realistic cases capturing key

situations in the dynamic context above described

(Nielsen, 1998). Secondly, it is far from trivial to

apply established evaluation techniques, such as

observation and think-aloud when an evaluation is

conducted in a field setting (Pascoe, 2000). Thirdly,

field evaluations complicate data collection and

ICEIS 2006 - HUMAN-COMPUTER INTERACTION

limits researchers controls, since subjects are acting

in an environment with a number of unknown

variables potentially affecting the set-up (Johnson

1998). In particular, the last issue heavily applies to

naturalistic tests in the vehicular domain, since data

are usually collected by some video cameras, and

many studies report how time-consuming and labor

intensive is gathering data from these videos (Barr et

al., 2004), providing in the meantime low-quality

reports.

On the contrary, driving a virtual car, simulated

by computer graphics in a laboratory, significantly

reduces the above difficulties, since tests are

accomplished in a safe and controlled environment,

where the risk of personal injury and property

damage is eliminated. Moreover, it is more

comfortable for researchers, which can get a higher

amount of high quality data and carry out more

controlled and repeatable tests, by presenting to

different subjects the same scenarios. The main

difficulty is to provide an adequate degree of realism

in the virtual scenario, since car dynamics and

activities in the subject’s physical surroundings can

be difficult to recreate realistically (Pirhonen et al.,

2002). Thus, in order to ensure significant results of

the tests, it is necessary to develop simulation

environments which exhibit a high level of realism.

3 THE PROPOSED TEST-BED

The test-bed we proposed in (Costagliola, 2005) was

aimed at presenting a realistic simulated driving

environment to conduct indoor evaluations of VTS

UIs, by providing subjects with credible feedbacks

for their actions (e.g.: steering wheel shake when

leaving the lane and going off-road), and sense of

presence in the virtual environment, being in the

meantime cost-effective both in terms of hardware

and personnel resources. The test-bed is intended as

an integrated framework, composed of three main

facilities: a driving simulator running on a graphical

workstation, a VTS, and some tools to analyze

subject’s behaviours, stored in a telemetry database.

In order to provide driving features, we

customized an open-source car simulation engine,

the Racer system (Van Gaal, 2000). We choose this

engine thanks to its many advantages: it provides

satisfactory dynamics of the virtual vehicle by using

6 Degree-Of-Freedom models and motion formulae

from Society of Automotive Engineers, it is very

flexible, since almost all simulation parameters are

customizable through text files, there is enough

documentation, it supports force-feedback devices, it

provides high-quality OpenGL rendering, the tracks

and the scenes are quite easy to create both by

scratch or through many free user-friendly editors,

and last but not least, it is free for non-commercial

use.

Figure 1: The test-bed Architecture.

Special care has been devoted to enhance the

sense of presence of the subject in the virtual

environment. Subjects sit on a car seat, interacting

through a force-feedbacks steering wheel, able to

return realistic haptic feedbacks to driving input, and

two foot pedals. The simulated scenario is projected

onto a wide-screen covering a significant subject’s

angle of view (>135°), while a 5.1 surround system

provides a realistic spatial audio, as suggested by

(Green, 2003).

To get data able to inform on driver distraction,

the simulator was customized to offer some further

features, such as the possibility to generate

asynchronous external events to test driver’s

response times. For instance, to add meaningfulness

to the simulation, other simulated cars were added

on the track with their own (repeatable) behaviors.

To recreate these situations, we exploited the Racer

features related to the Artificial Intelligence (AI),

allowing for a basilar simulation of traffic

conditions, programming different vehicles to follow

specific routes and behaviors on the track.

The simulation engine has been enriched and

complemented by some tools we have developed

specifically to address vehicular issues. The former

tool, allowing for an integration of the VTS

navigator module in the simulated environment, is

described in the following.

3.1 Integration of the Navigator in

the Simulated Environment

The first distinguishing feature offered by our

framework is the integration between the simulation

engine and the VTS navigator module. Currently, at

best of our knowledge, evaluations of the navigation

assistance subsystems can be performed exclusively

on the field, since virtual tracks employed in

simulations usually are not a counterpart of a real

A SIMULATION ENVIRONMENT TO EVALUATE DRIVER PERFORMANCES WHILE INTERACTING WITH

TELEMATICS SYSTEMS

geographic area. Even if dealing with digital

reconstructions of existing roads, driving simulators

do not provide features for converting in real-time

the spatial coordinates of the virtual car into real

GPS ones. Hence the navigator has not knowledge

about the virtual car position, resulting of no use in

indoor evaluations.

To address this issue, we developed two

subsystems: one responsible to generate a Racer

track stating from a real cartography, and one able to

translate the coordinates of the virtual car driven by

the subjects, into a stream of bytes emulating a real

GPS serial sensor sent to the VTS. As a result, the

road virtually driven by the user on the simulator is

shared as a map on the VTS. This permits to exploit,

indoor, many standard navigation features, such as

Map Display and Route Guidance. This integration

is a powerful instrument, enabling to perform many

significant tests. For instance, it allows researchers

to evaluate different modalities (vocal, iconic, etc…)

for providing routing assistance to the user, or the

most appropriate vocabulary to support the way-

finding, as well as to assess the cognitive work

inducted by these different approaches.

In the following we will describe how we have

developed this feature.

3.1.1 Sharing the Cartography

The Racer engine adopts a proprietary graphical

format to represent tracks, named DOF1 and based

on the SGI IFF file format. DOF1 exploits OpenGL

XYZ coordinate system and contains all the

information about the scene graph of the model. In

particular, it holds data about the geometry objects

composing the track, i.e. information about the

vertices and the normals, together with other data,

such as the texture used to render the surfaces.

It is worth pointing out that currently there exist

two standard global cartographical databases

adopted in the automotive domain: the Navteq and

the TeleAtlas one. We focused on the former, thanks

to the availability of a powerful SDK, useful to

manage navigation system applications and to

interact with the Navteq open format SDAL (SDAL,

1999) used for the map database.

In order to obtain a shared cartography, we

developed a tool able to generate an appropriate

DOF1 track file starting from an arbitrary sized

rectangular area of a SDAL map. In particular, this

translator generates the geometry primitives starting

from the parcels that are the basic units of I/O used

in the SDAL format.

Figure 2 (left) shows a

rectangular area of a SDAL map, while

Figure 2

(right) illustrates the corresponding generated DOF1

track.

Another challenging issue was about road width,

since SDAL does not contain such information.

Instead, for each segment representing a section of a

road, there is an associated attribute, the Rank,

indicating its rating, based on characteristics such as

speed limit, road type (e.g., interstate, highway) and

access limitations. For instance, rank 0 represents

the lowest one (local streets), while rank 4 represents

the highest level (national arterial road network).

Basing on this information we defined 10 kinds of

road templates (two for each rank, to consider one or

two-way roads), each one with its own geometries

and textures to represent different number of lanes,

presence of guard-rails, etc…

Figure 2: A real SDAL map (left), and the corresponding

generated DOF1 one (right).

3.1.2 Updating the Localization

To address the second issue concerning with

updating in real-time the position of the car on the

map we have let the simulator to export information

about the car movements and the navigator to accept

this information as if it comes from GPS sensor. In

particular, it was required to get information about

coordinates, speed, and heading of the car. Racer

engine is able to output this information, for various

purposes (multiplayer, logs, etc…). Thus, we

implemented a daemon able to catch this

information and, after some elaboration, to send the

necessary data on a serial port, emulating a real GPS

sensor. In this way, every VTS (or even PDA) able

to be connected to a GPS serial sensor can be linked

to the simulation environment, providing real-time

route guidance information.

4 DATA ANALYSIS

In order to assess a VTS UI, it is important to

quantify the “safety degree” of the considered VTS.

Nevertheless, safety cannot be directly measured

(probably except in retrospect) (Tijerina, 2001). In

the literature, several indirect measures of safety

have been proposed based on the evaluation of the

driver distraction inducted by the system (e.g.

(Camp, 2000)). Let us recall that, when dealing with

ICEIS 2006 - HUMAN-COMPUTER INTERACTION

vehicles, two main kinds of distraction should be

considered, namely the visual one and the cognitive

one. Each of them leads to different problems:

degraded vehicle control (resulting in problems in

lane-keeping, speed maintenance, etc…), and

degraded object/event detection (looked-but-did-not-

see) (Brown, 1994), which is a more insidious to

evaluate, since vehicle control remains largely

unaffected but detection and reactions of unexpected

object and event is degraded (Tijerina, 2001).

Several indicators have been proposed to measure

driver distraction during a test session. Among these,

the most important are speed maintaining, input

smoothness, lane-keeping (see Figure 3), car

following performance, and driver reaction times to

asynchronous events (Tijerina, 2001).

Figure 3: Distance to be measured for Lane-Keeping.

It is worth noting that the result of a VTS

usability assessment is a set of telemetry data, i.e. a

lot of numerical information, such as user input and

vehicle dynamics, describing how the car and the

driver behaved during the experiment. It is important

to properly analyze this valuable amount of data in

order to get information on subjects’ distraction and

highlight potential degraded vehicle controls or

degraded object/event detection. Moreover, these

telemetry data can allow researchers to infer how

subjects’ visual/cognitive workload is influenced

when external factors are changed. For instance, it is

possible to understand the consequences of

exploiting different sensorial channels or different

layouts for VTS graphical user interfaces, by

comparing the gathered test datasets on the same

subjects. In order to properly analyze these data, it is

necessary provide analysts with suitable tools

supporting them during information interpretation.

To this aim, the proposed framework has been

enhanced with a specific data analysis environment,

detailed in the following.

4.1 The Telemetry Analyzer Tool

The Racer provides a built-in logging feature, which

was customized to gather all the needed telemetry

data. In particular, during a test running, each 250

milliseconds we sample the following parameters:

• Time

• User Input

o Steering input

o Force Feedback output

o Throttle input

o Brake input

• Vehicle Dynamics

o yaw, pitch and roll

o vx, vy and vz

o ax, ay and az

o x, y and z

o Overall distance

• VTS I/O (if possible)

o Key pressed on the VTS prototype

o Asynchronous events generated by

VTS

All these data are stored in a log ASCII file.

Separate files are used to store other significant

information, such as details on asynchronous events

generated both by the simulator, (i.e.: AI controlled

cars actions), and/or by the tester (i.e.: actions

required to be achieved by the subject).The amount

of data collected per session in this way can be

really impressive. For instance, a 20 minute

registration (a typical duration for a VTS test

session) generates a data matrix with about 4800

rows and tens of thousand of cells. As a result, it is

almost impossible to effectively analyze this huge

but fundamental amount of data without a suited

supporting tool. Neither standard analytics software

products, such as R, Statistica or Excel can

successfully address this issue, since it is very

difficult for a researcher to understand the most

significant distraction indicator, the lane-keeping

(shown in

Figure 3), without a graphical

representation of the subject’s followed trajectory

overlapped to the testing track.

Thus, there is a strong needing for a supporting

tool, able to graphically visualize the fundamental

parameters describing subjects’ behaviours. To

answer this issue, we developed an integrated

application, named Telemetry Analyzer Tool, whose

main objective was to visually render the driver-

vehicle performances during the whole test session.

In particular, the tool graphically presents both the

temporal/spatial relationships among data (that

usually are difficult to understand in a textual way),

both the main subjects’ actions together with other

A SIMULATION ENVIRONMENT TO EVALUATE DRIVER PERFORMANCES WHILE INTERACTING WITH

TELEMATICS SYSTEMS

numerical information, such as speed, time, and

overall distance.

In the following the main features of the tool and

its user interface are detailed.

4.1.1 Main Features of the Developed Tool

The tool offers three main features. Firstly, it

visually present all subject’s input, time by time. To

control the timeline of the logged data, the tool

provides some widgets, recalling an interface of a

Video Recorder, allowing a researcher to gain a deep

insight on subject’s behaviours, through the whole

test session.

Secondly, it provides a visual representation of

the trajectory followed by the subject, overlapping

the map of the track. To evidence driving errors,

whenever the trajectory followed by the vehicle’s

centre of gravity comes within less than 50 cm of the

left/right edge of the lane, it is represented in red

colour (or in a different shape) for the specific

frame.

Thirdly, it provides some comprehensive report

on the test session, such as the percentage of time

spent too near or too far from the central line, etc…

The resulting user interface is shown in

Figure 4.

Figure 4: The Telemetry Analyzer Tool GUI.

As depicted, it is possible to notice that the UI is

divided in three horizontal zones:

• The uppermost contains the widgets suited to

load track and log files and to manage the

timeline through some controls recalling a

VCR. Through these controls it is possible to

select a specific frame t

, a time-interval t

-t

, or

to playback the entire test session t

-t

. It also

possible to change the time scale, to accelerate

or slow down the playback, as well as to move

frame-by-frame.

• The central section is aimed at dynamically

present visual information about user

behaviours, for the current frame t

. Once

selected an instant of time t

, or for each frame

during the playback, it renders the status of all

user input at t

. In particular, driver input on

brake and throttle are rendered through some

histograms, while the steering wheel angle is

presented both visually and numerically. This

section provides also information about main

parameters on vehicle dynamics, such as

heading, speed, and roll, useful to evaluate

overall input smoothness.

• Finally, a lower section shows a top view of the

track, with the trajectory held by the subject in

the current frame/interval. This section also

encompasses widgets suited to control zoom,

pan, and rotation of the track.

About numerical reports, the tool is able to

invoke the Microsoft Excel DCOM control to

generate graphs of the collected data, on a number of

different parameters. Some examples of generated

scatter plots are visible in Figures 4 and 5.

5 ASSESSING THE PROPOSED

TEST-BED

As stated in section 2, the potential problem with an

indoor simulator is the lack of realism. To this aim,

we conducted a preliminary study to validate the

effectiveness of the simulator in measuring real

driving performances, by involving a group of

sixteen external subjects. To gain insight on the

realism of the framework, we applied methodologies

proposed in other simulators’ assessments (e.g. (Lee,

2002), ()). In the following we provide the details on

the assessment.

5.1 The Subjects

Sixteen volunteers (11 males, 5 females) were

involved in the study. Age of the subjects varied

between 24 and 42 years (mean 31.6). All

participants have a valid driving licence, a good

corrected visual acuity, and years of driving

experience ranged from three to 21 (mean of 10.4).

Annual mileage was between 2,400 and 25,000

(mean of 8,250). We asked them also details on

typical driven roads, mean number of hours spent in

driving per week, and if they usually play at driving

computer games. Four subjects stated that they spent

over 1 hour each week playing computer games that

involved driving (racing). Two subjects reported no

ICEIS 2006 - HUMAN-COMPUTER INTERACTION

previous experiences with driving games. The other

participants reported to play computer games only

occasionally.

5.2 Methodology

The experiment consisted of three driving sessions,

gradually increasing in difficulty. Whenever a

sufficient level of proficiency was rated, subject

move the next step. After completing the driving

tasks, subjects received a questionnaire, to express

their impressions on the simulator.

During the first session of the experiment, we

tested elementary driving skills, such as steering,

acceleration and braking control. Subjects were

instructed to drive on a simple and flat track,

composed of many straights, and few, easy turns,

maintaining a stable speed of 60 km/h, if possible.

Just before a curve, they were asked to reduce speed,

to enable a smoother turning.

The second session of the experiment was based

on a more complex track, representing a hill road,

with turns, tunnels, climbs and descents hills, harder

turns, and other cars. Again, we asked subjects to

maintain a steady speed of 80 Km/h, and to never

pass 100 Km/h, a harder task due to the specific

characteristics of the track.

In the third session, subjects were asked to

perform some secondary tasks on a VTS, while

concurrently continuing to perform the primary

driving task. The track was easier than the previous

one. Subjects had to respect the signals, to keep a

mean speed of about 80 Km/h, if compatible with

track properties and speed limits, and to never pass

100 Km/h.

Approximately each session took 20 minutes per

subject. The selected test tracks were intended as

closed circuits, and each subject was asked to drive

for three laps, to better understand improvements in

vehicle controls.

Obviously, during each session we logged all the

parameters described in Section 4.

5.3 Results

We observed that the simulator has a smooth

learning curve. Subjects familiarized very quickly

with the simulator, reaching adequate performance

after 15-20 minutes of training. In particular, after

some initial hesitations due to the different steering

response, subjects were able to profitably drive the

vehicle, maintaining constant speed and being able

to correctly keep the lane. Moreover, these

behaviours were achieved through smooth input.

A graphical representation of such improvements

is given in

Figure 5 and Figure 6. Such figures plot the

speed hold by Subjects 2 and 5, respectively, in the

same section of the track, on the three different laps

during session 2. We selected for the graph a

segment where the road descents, turn right and then

climbs. Thus it is very demanding to maintain

constant speed, and in particular to not overpass the

100 km/h. Looking at graphs, it is possible to notice

that in the first lap (the blue line), subjects were not

able to smoothly maintain the speed. Indeed, the line

presents steps, and, in case of S5, it varies from 30

to 95 Km/h. Lap 2 (the purple line) and 3 (yellow

line) were smoother, indicating enhancements in

driving performances.

Subject 2

100

120

Seconds

Km/ h

Lap1

Lap2

Lap3

Figure 5: Subject 2 driving behaviour.

Subject 5

100

120

1 10192837465564738291100109118127136145

Km/h

Lap1

Serie2

Serie3

Figure 6: Subject 5 driving behaviour.

Throttle S2

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

1 10 19 28 37 46 55 64 73 82 91 100 109 118 127 136 145 154

Lap1

Lap2

Lap3

Figure 7: Subject 2 throttle behaviour.

Similarly, in Figure 7 it is possible to appreciate

improvements in the throttle control, through the

various laps. Once completed the driving sessions,

subjects filled in a questionnaire, to express their

impressions on the simulator. About results, in

mean, subjects felt the simulator enough realistic

A SIMULATION ENVIRONMENT TO EVALUATE DRIVER PERFORMANCES WHILE INTERACTING WITH

TELEMATICS SYSTEMS

(6.57, in a ramp 0..10), even if more stressing and a

little more difficult than real driving.

However, some concerns come from the steering

wheel, which resulted too much direct in its action,

being different from the real one. This is due to the

economical input device, which is more game-

oriented. Indeed, its excursion is limited to about

200° from full left deflection to full right deflection.

In real car this value is comprised from 360° and

720°. We are currently searching for more realistic

steering wheels. Telemetry data analysis and survey

result are omitted for sake of brevity, but are

available upon request.

6 CONCLUSIONS AND FUTURE

WORK

Safety on the roads is one of the main goals for

everyone involved in the automotive field. The

advent of VTSs can distract user from the main task

of driving the car, with potentially fatal effects.

Nevertheless, it has been estimated that these

systems will become commonplace in the last few

years. Thus, it is a short term priority to investigate

solutions to enhance usability of VTSs and then

limit driver distraction. Nevertheless, the evaluation

of UIs for automotive systems is a challenging and

expensive task, requiring specific methodologies and

tools. To address this issue, we realized a framework

specifically conceived for the indoor evaluation of

VTSs usability. In this paper we reported on some

improvements we developed, aimed at enhancing the

effectiveness of that test-bed. In particular, we

developed some instruments to offer the possibility

to assess also the navigation assistance provided by

VTS in the indoor facilities, thanks to the automatic

generation of realistic simulator tracks starting from

a VTS cartography.

Moreover, to support researchers in an easy

collection of valuable information on driver’s

behaviors (and thus on his/her mental workload), we

developed a specifically suited application, aimed at

providing a graphical representation of the main

driving parameters and subject behaviors. Finally,

we validated the framework in measuring on-road

driving performances, by employing a set of sixteen

subjects, with positive results.

About future work, we are working to add

further realism to the generated scenario, in order to

recreate the surrounding environment, since SDAL

contains further information about the kind of area

(country, national park, urban, etc…). For instance,

for urban zones, we are working to generate

scenarios with buildings, semaphores, etc…

REFERENCES

Barr, et al. 2003. Exploratory Analysis of Truck Driver

Distraction Using Naturalistic Driving Data. 82nd

Annual Meeting of the Transportation Research

Board, Washington, DC, January 12-16, 2003. (P03-

2897).

Brown, I. D., 1994. Driver fatigue. Human Factors, 36

(2), 298-314

Burns, C. Lansdown, C., 2001. E-Distraction: The

Challenges for Safe and Usable Internet Services in

Vehicles

CAMP, 2000. Crash Avoidance Metrics Partnership

(CAMP), Proposed Driver Workload Metrics and

Methods Project

Comunicar, 2002. ComuniCar Project Deliverable D2.2,

“User needs expectation and acceptance”. Web Site:

http://www.comunicar-

eu.org/archive/documents/D2_2.doc

Costagliola, Di Martino, Ferrucci., (2005), A Framework

for the Evaluation of Automotive Telematics Systems.

In Proceedings of the 7

International Conference on

Enterprise Information Systems (ICEIS 2005)

Green, P., 1999. SAE J2364 – Navigation and route

guidance function accessibility while driving.

Warrendale, PA: Society of Automotive Engineers.

Green P. et al., 2003, Audio-Visual System Design

Recommendations from Experience with the UMTRI

Driving Simulator, in DSC North America 2003

Proceedings, Dearborn, Michigan.

Juliussen E., 2003. “The Future of Automotive

Telematics”. In Business briefing: global automotive

manufacturing & technology 2003

Lee H., 2002. The Validity of Driving Simulator to

Measure On-Road Driving Performance of Older

Drivers. In Proceeding of 24

Conference of

Australian Institutes of Transport Research, Sydney,

2002.

Lumsden J., Brewster S. 2003, A Paradigm Shift:

Alternative Interaction Techniques for Use with

Mobile & Wearable Devices, In Proceedings of

CASCON workshop, 2003

Marcus A., 2004. Vehicle User Interfaces: the next

revolution, Interactions, 1.

Microsoft, 2005. “Microsoft Software and Solutions for

the Automotive Industry”. WebSite:

http://www.microsoft.com/automotive/

NADS, 2005. National Advanced Driving Simulator. Web

Site: http://www.nads-sc.uiowa.edu/

Tijerina L., Parmer, E., Goodman, M., 1998. Driver

workload assessment of route guidance system

destination entry while driving: a test track study. In

Proceedings of the 5th ITS World Congress, Seoul,

Korea.

Tijerina L., 2001. Issues in the Evaluation of Driver

Distraction Associated with In-Vehicle Information

and Telecommunications Systems

Van Gaal, 2000. Racer Simulator Engine, available at

www.racer.nl

ICEIS 2006 - HUMAN-COMPUTER INTERACTION