Logical Scenario Derivation by Clustering Dynamic-Length-Segments

Extracted from Real-World-Driving-Data

Jacob Langner

, Hannes Grolig

, Stefan Otten

, Marc Holz

apfel

and Eric Sax

FZI Research Center for Information Technology, 76131 Karlsruhe, Germany

Dr. Ing. h.c. F. Porsche AG, 71287 Weissach, Germany

Keywords:

Scenario Extraction, Logical Scenarios, Vehicle Environment, Real-World-Driving-Data.

Abstract:

For the development of Advanced Driver Assistant Systems (ADAS) and Automated Driving Systems (ADS)

a change from test case-based testing towards scenario-based testing can be observed. Based on current

approaches to deﬁne scenarios and their inherent problems, we identify the need to extract scenarios including

the static environment from recorded real-world-driving-data. We present an approach, that solves the problem

to extract dynamic-length-segments containing a single scenario. These segments are enriched with a feature

vector with information relevant for the feature under test. By clustering these scenarios a logical scenario

catalog is created, containing all scenarios within the test data. Corner cases are represented as well as common

scenarios. An accumulated total length can be calculated for each logical scenario, giving a brief understanding

about existing test coverage of the scenario.

1 INTRODUCTION AND

RELATED WORK

Original Equipment Manufacturer (OEM), research

institutes and start up companies are currently putting

a lot of effort into realizing autonomous driving.

Many have succeeded in ﬁrst trials, however, the ﬁrst

autonomous vehicle is yet to be released. Current

ADAS slowly develop into conditionally automated,

SAE level 3 features, that can take over a driving

task completely in explicitly restricted driving envi-

ronments, e.g. a trafﬁc jam assist on highways up

to 60 km/h. Such features are not yet globally avail-

able, because OEM lack the proof of safety needed for

clearance from the National Motor Vehicle Registra-

tion Authorities. Different standards and regulations

require a separate approval in each country.

According to the ISO26262 (ISO, 2011) safety

goals are set via a risk analysis. To achieve the high-

est possible safety and security on public roads, the

fulﬁllment of these safety goals has to be proven. The

proof can be brought forward analytically or with the

help of test cases (Henzel et al., 2017). For system-

level testing of ADAS and ADS analytic proof is

not feasible due to the complexity and extent of the

software code and input parameter space. Therefore,

common practice is to use test cases.

In order to achieve sufﬁcient testing, the test cases

need to cover all relevant situations for the Feature

Under Test (FUT). Test cases are usually derived from

the feature requirements manually by domain and test

experts. Each functional requirement is covered by at

least one test case. During system level testing all test

cases are evaluated and a ﬁnal test result concludes

the fulﬁllment of each requirement. However, deter-

mining all relevant situations and declaring all feature

requirements, from which the complete set of corre-

sponding test cases can be derived, is not a trivial task.

With the technological advances in environmen-

tal perception, car-to-x communication as well as au-

tomated driving, the complexity of new ADAS and

ADS is steadily increasing. Due to new sensors be-

ing integrated in today’s vehicles, sensor ranges be-

ing increased and background services, e.g. digi-

tal map providers, being utilized, the environmental

information available in the vehicle has grown sig-

niﬁcantly. While these technological advances cer-

tainly enhance the quality of these features and the

possibilities towards more autonomous driving, they

also introduce a manifold of new situations, a vehicle

can encounter, and thereby increase the required test-

ing efforts. The deﬁnition of testable, ISO-conform

requirements - and, thereby, the test case-based ap-

proach - is no longer feasible for the next generation

of ADAS and ADS.

458

Langner, J., Grolig, H., Otten, S., Holzäpfel, M. and Sax, E.

Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data.

DOI: 10.5220/0007723304580467

In Proceedings of the 5th International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS 2019), pages 458-467

ISBN: 978-989-758-374-2

For ADAS and ADS development it is best prac-

tice to additionally test the feature on public roads

in order to experience as many different situations

as possible. Huge efforts are made to drive as many

kilometers as possible during test campaigns and re-

lease approvals, e.g. 36 million test kilometers for

the release of the Mercedes Benz E-Class (W212) in

2009 (Deppe, 2009) and 12 million test kilometers for

the next generation E-Class (W213) in 2015 (Chris-

tiansen, 2015).

Since there is no checklist of situations to be seen

during testing, the required test kilometers are derived

from statistical calculations. Due to the combination

of different environmental situations and trafﬁc par-

ticipant’s maneuvers, the sheer amount of kilometers

needed to achieve a statistically valid assertion sums

up to a vast number. (Wachenfeld and Winner, 2016)

have calculated a theoretical 6.62 billion test kilome-

ters to prove, that an interstate pilot meets the cur-

rent standards for release approval. The number is de-

rived from fatal accidents drawn from annually crash

statistics in Germany and the total sum of driven high-

way kilometers in Germany per year - assuming that

the absence of accidents is the approval criterion for

highly automated driving. (Amersbach and Winner,

2017).

Simulation enables the selective testing of for ex-

ample critical situations, possibly reducing the re-

quired test kilometers to a fraction of the statistically

calculated 6.62 billion kilometers in case of the inter-

state pilot. Thereby, simulation is an important tool

for OEM to even come close to the required test cov-

erage. However, the validity of the simulation has to

be ensured by providing detailed models of the vehi-

cle, environment, sensors and trafﬁc. Efforts for pro-

viding these models are limiting the theoretical scala-

bility of the simulation approaches.

Figure 1: The 5-level structure for scenario description from

the PEGASUS

project.

It has been shown, that for certain ADAS recorded

driving data from real world tests can be utilized to

substitute most of the models needed for the simula-

tion, for an example see (Zofka et al., 2015). Open-

loop features can easily be tested using this data. In

some cases, e.g. predictive features, it even entails

the beneﬁt of the ground truth in form of the recorded

events (Langner et al., 2017). Even closed-loop longi-

tudinal control features have been simulated using the

recorded data from test campaigns (Bach et al., 2017)

(de Gelder and Paardekooper, 2017).

The real-world-driving-data provides a large

amount of realistic driving situations including many

critical ones, which offer better test coverage than

synthetic data and scale better than manually de-

ﬁned test cases. However, the value of the real-

world-driving-data tests is diminished by redundan-

cies. Simply simulating all of the recorded data of-

fers the same control over the tested situations as real

world test drives.

The calculations made by Wachenfeld and Win-

ner are based on a poisson probability distribution of

crashes from German trafﬁc data. The statistical cal-

culations hold true as long as there is no semantic in-

terpretation and selection of the test drives. The con-

tribution towards the proof of safety has to be mea-

sured by the amount of different situations tested, not

by the mere amount of kilometers simulated (Junietz

et al., 2017). An abstract description and the possi-

bility to measure the coverage of these situations is

needed.

Recent research focuses on scenario-based ap-

proaches, where different situations a feature can en-

counter are described on a higher abstraction level. A

scenario contains the static environment, the dynamic

environment including other trafﬁc participants and

their actions as well as the actions of the ego vehi-

cle for a certain period of time (Elrofai et al., 2016).

(Schuldt et al., 2013) have proposed a 4-level scenario

structure for the description of scenarios: the basic

road network, situational modiﬁcations, trafﬁc situa-

tion and environment. In the PEGASUS project the

basic road network is split into street level and traf-

ﬁc infrastructure, resulting in a 5-level scenario struc-

ture (see Figure 1). (Bagschik et al., 2017) within the

PEGASUS project have suggested three different ab-

straction layers for scenarios to offer the needed ﬂex-

ibility in each development phase (see Figure 2).

Concrete scenarios are individual instances of log-

ical scenarios, where the complete static and dynamic

environment as well as the state of the ego vehi-

cle are described in detail. They offer little ﬂexibil-

https://www.pegasusprojekt.de/ﬁles/tmpl/linebreak

PDF-Symposium/04 Scenario-Description.pdf

Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data

459

Figure 2: Scenario abstraction layers (Steimle et al., 2018).

ity and their abstraction level is quite similar to test

cases. By abstracting from the explicit signal values

of these concrete scenarios some generalization can

be achieved. Logical scenarios describe a group of

concrete scenarios by parameter ranges for each sig-

nal, whereas functional scenarios textually describe

logical scenarios based on a functional view. This al-

lows for a high-level description of different scenar-

ios early on, which can be speciﬁed in later stages of

the development process. Test cases can be derived

from concrete scenarios, which in turn can be derived

from logical scenarios. Therefore, test coverage can

be measured on the logical scenario level.

Still, all possible scenarios a feature can undergo

must be known and explicitly speciﬁed to achieve

complete test coverage. There is a lot of ongoing re-

search in the ﬁeld of scenario deﬁnition and standard-

ization. However, a common scenario catalog repre-

senting all scenarios, that can occur while driving in

the real world, is not yet available. Up until now, at-

tempts to create such a common scenario catalog are

limited to certain features and environmental condi-

tions, for example a trafﬁc jam assist with scenarios

that are altogether on highways (P

utz et al., 2016).

Furthermore, it is to be debated, if a universal sce-

nario catalog exists, since relevant scenarios strongly

depend on the FUT, e.g. urban test drives do not mat-

ter for testing of a highway pilot. It can be argued, that

there exists a universal scenario catalog for level 5 au-

tonomous driving and that current level 2 and level 3

features can derive relevant subsets from this catalog.

However, the argument leads nowhere, since this cat-

alog not yet exists.

Instead of explicitly deﬁning all scenarios, they

can also be extracted from recorded real-world-

driving-data (see Figure 2). Several approaches

to extract driving maneuvers of the ego vehicle or

other trafﬁc participants have been published (Gerdes,

2006), (H

ulnhagen et al., 2010). In most cases, ba-

sic maneuvers are explicitly deﬁned and mapped onto

physical parameters and parameter ranges or param-

eter curves. These are then used as a pattern to de-

tect further instances of the same or similar maneu-

ver. (Elrofai et al., 2016) enhance this approach with

data analytics to improve the performance of the de-

tectors. Another approach was published by (Min-

nerup et al., 2015), who use triggers for scenario ex-

traction from real-world-driving-data. However, only

explicitly speciﬁed and searchable scenarios like crit-

ical distance to vehicle in front or amount of steering

per second can be found. (Junietz et al., 2017) derive

an abstract criticality metric to identify critical scenar-

ios based on relative metrics between the ego vehicle

and other dynamic objects. Calibration is done with

manually classiﬁed scenarios and the metric only de-

rives a pre-selection for manual ﬁltering. There are

no approaches known to the authors, that focus on ex-

tracting scenarios including the static environment of

the vehicle.

All these approaches are motivated by ADAS like

an Adaptive Cruise Control (ACC) or Emergency

Brake Assist (EBA) and are focused on extracting

critical situations for the release approval. Most of

the critical situations for these features occur, when

interacting with other trafﬁc participants. Thereby,

according to (Schuldt et al., 2013) the relevant sce-

narios are also mainly based on the dynamic objects.

But, this does not hold true for predictive features,

whose predictions mainly rely on the sensed environ-

ment as well as upfront information about the static

environment from digital maps. The maneuver ex-

traction approaches, therefore, do not yield a viable

scenario catalog for these features.

Furthermore, testing critical situations alone is not

sufﬁcient either. Especially features taking over lon-

gitudinal or lateral control also have a functional qual-

ity aspect besides the functional safety - the driver or

passenger comfort. Merely avoiding accidents is not

sufﬁcient for the passenger acceptance of the ADS.

Comfort ratings will become increasingly important

with further advances towards autonomous driving -

and comfort has to be evaluated in all situations, not

only the critical ones.

In this paper we present a data-driven approach

to segment the real-world-driving-data into scenar-

ios tailored to the FUT. In contrast to the maneu-

ver extraction approaches, we focus on the static en-

vironment - including the street level, trafﬁc infras-

tructure and environment conditions - and its correct

segmentation. The scenarios are automatically ex-

tracted based on selected signals relevant for the FUT.

The approach combines the description of the static

environment and the dynamic objects and derives a

feature-based scenario catalog from the driving data.

By using a dynamic clustering approach we deter-

VEHITS 2019 - 5th International Conference on Vehicle Technology and Intelligent Transport Systems

460

mine a ﬂexible amount of logical scenarios, which

contain all found concrete scenarios within the driv-

ing data. The aggregation of discrete signal values

leads to logical scenarios, which offer a more com-

prehensible scenario catalog for system-level testing.

Further interpretation of the logical scenarios can re-

sult in functional scenarios for the FUT.

2 SCENARIO-EXTRACTION

CONCEPT

Real-world-driving-data consists of many driving sit-

uations, which uninterpreted and on their own offer

little information towards the coverage of all possible

situations. However, extracting all these driving situ-

ations as self-contained concrete scenarios and aggre-

gating them to logical scenarios does. The problem

is to ﬁnd and extract these scenarios within the test

drives.

Our goal is to automatically extract all existing

scenarios relevant for the FUT implicitly from the

data in order to create the feature’s logical scenario

catalog. An example of two logical scenarios is il-

lustrated in Figure 3. The 5-level scenario structure

from the PEGASUS project is used for the scenario

description. Figure 4 reveals the necessary steps: The

ﬁrst step is to select the relevant signals for the FUT

and pre-process these signals for further usage. The

scenario dividing signals are then used to slice the test

drives into segments with dynamic lengths. Then, a

feature vector describing the static and dynamic envi-

ronment for each segment is extracted from the driv-

ing data. The feature vectors are clustered and each

cluster is interpreted as a logical scenario. The fea-

ture vector is also used as a scenario description.

2.1 Data Selection and Pre-Processing

As already stated - for system level testing - the data

selection has to be done with regard to the FUT. For

ADAS and ADS the amount of possible different sit-

uations, accrued from the combination of the static

and dynamic vehicle environment, is a critical part

for testing. Therefore, signals describing the relevant

vehicle environment should be used. Not all infor-

mation, however, can be derived from a single signal.

Some have to be transformed or gathered from addi-

tional information sources. The temperature, wetness

or trafﬁc density for example can be extracted from

weather or trafﬁc data providers, if they are relevant

for the FUT. More abstract values like the time to col-

lision (TTC) can be calculated as well.

When working with recorded real-world-driving-

data, signal pre-processing is required. The data is

noisy and for statistical uses, some assumptions have

to be made. Error and initial values have to be han-

dled. Especially the GPS signal and the velocity have

to be considered carefully as they are the foundation

for further signal processing. Noise in these signals

propagates throughout the analysis.

Real-world-driving-data is commonly recorded

time-based. This time-dependency leads to a distor-

tion in the representation of the static environment

due to differing velocities during the test drives. In

order to compare the test drives, the signals have to

be converted to a track position-based sampling. The

sampling should be equidistant and consistent cross-

samples and with a reasonable distance between two

data points.

Further pre-processing includes derivation of new

signals or reshaping of existing data.

2.2 Scenario Extraction

For predictive features, which rely heavily on the

sensed static environment, it is necessary to identify

all environmental situations, that can be experienced,

and to validate the correct behavior in these situations.

Contrary to the maneuvers, which have clear start and

end points, the challenge for the static environment is

to detect the change points for seemingly smooth tran-

sitions between different environmental situations.

Common machine learning approaches, that need

to partition real-world-driving-data, use either static

segment lengths or time intervals (Langner et al.,

2018) or up- or downsample the sequences to a de-

ﬁned length (Wang et al., 2018) in order to normal-

ize and partition the test drives, which than can be

processed further. By using a static segment length,

single scenarios within the test drives might be sepa-

rated into two subsequent segments (see Figure 5). A

single segment might also contain more than one sce-

nario. With segments containing more than one sce-

nario and scenarios being divided into multiple seg-

ments, grouping these segments by their feature vec-

tors to generate a scenario catalog is pointless. Up- or

downsampling on the other hand distorts the static en-

vironment and reduces comparability of different se-

quences. Consequently, the test drives need to be cut

along their natural scenario transitions.

The core problem is to ﬁnd these transition points.

One solution is to use predeﬁned scenarios, speciﬁed

by experts, to detect the conditions, that start or end

a scenario. While this approach might work well for

single maneuvers, there is no guarantee, that the ex-

perts thought of every possible scenario. Most cer-

Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data

461

Logical Scenario 1

Level 1

– Street Level:

Slope

Street Class

Curviness

Segment

Length

Flat (< 5%)

Village road

Strong curves (> 3 °/m)

Short (10 - 50m)

Level 2

– Traffic Infrastructure:

Speed Limit

Stop

Lights

30km/h; 50 km/h

0 - 1

Level 3

– Temporal Modifications:

Level 4

– Movable Objects:

Braking

Maneuvers

Average

Velocity

0 - 3

30 - 40 km/h

Level 5

– Environment Conditions:

Environmental

Context

Country Code

Urban

Germany

Evaluation:

Accumulated

Length 2.649 m

Logical Scenario 2

Level 1

– Street Level:

Slope

Street Class

Curviness

Segment

Length

Flat (<

5%)

Village

road

Straight (< 1

°/m)

Average (50

- 500m)

Level 2

– Traffic Infrastructure:

Speed Limit

Stop

Lights

50 km/h

- 5

Level 3

– Temporal Modifications:

Level 4

– Movable Objects:

Braking

Maneuvers

Average

Velocity

- 7

- 60 km/h

Level 5

– Environment Conditions:

Environmental

Context

Country Code

Urban; Extra Urban

Germany

Evaluation:

Accumulated

Length

30.267 m

Figure 3: Exemplary scenario clusters for a PCC feature and their aggregated description.

Data Selection

and Pre-

Processing

Scenario Extraction

Dynamic

Segment

Detection

Feature Vector

Extraction

Scenario

Clustering

Tes t

Drives

Figure 4: Required steps for the scenario extraction ap-

proach.

tainly even, there are corner cases, the experts did not

think of. For the larger parameter space of complete

scenarios, including the static vehicle environment, an

approach is needed, that does not rely on explicit sce-

nario deﬁnitions beforehand. By extracting the sce-

narios from the data pool, scenario speciﬁcation is no

longer necessary. All occurring scenarios are implic-

itly derived, if the data pool is sufﬁciently large and

diverse.

Figure 5: Static segment length (left side) vs. dynamic seg-

ment length (right side) with transition points derived from

the curviness. The curve is separated into two segments

with the static approach whereas the dynamic approach is

able to detect the curve as a coherent segment.

In order to derive the transition points, we separate

the relevant signals in scenario-dividing signals and

scenario attributes. A feature’s input space consists

of many signals. Out of the subset of these signals,

which describe the vehicle environment, some signals

are categorical or ordinal, others are continuous.

Aggregation for categorical and ordinal signals

can oftentimes only be done via counting or percent-

based, which are not ideal for a distinct differenti-

ation. We argue, that - in most cases - these sig-

nals are also scenario dividing and, therefore, changes

in these signals naturally lead to scenario transition

points. For example a change in the street type from

highway to country road marks a signiﬁcant change in

the environmental situation and thereby separates two

different scenarios. Furthermore, the percent-based

representation, e.g. 64% highway and 36% country

road, is not a useful description for the street type of

a segment.

Continuous signals, e.g. slope or the ego vehicle’s

velocity, can be handled differently. For the velocity

a mean value can be calculated, the slope can be rep-

resented as two values, a positive and negative value,

so that these do not negate each other. In most cases,

these signals describe a segment in the data and do

not clearly divide it. Thus, they are categorized as

scenario attributes. However, if a continuous signal is

crucial for the FUT, it can also be used as a scenario

dividing signal, e.g. the ego vehicle’s velocity for

the trafﬁc jam assist. Via binning or thresholding the

velocity can be transformed into a low dimensional

space, where the boundaries of the velocity bins or

the thresholds operate as the transition points. Then,

a possible scenario segmentation could include dif-

ferentiation of vehicle standstill, stop-and-go trafﬁc,

slow driving and driving close to the feature’s veloc-

ity limit.

In general, it proved practical to use low dimen-

sional and categorical signals as scenario dividing sig-

nals. Continuous and numeric signals on the other

hand, can be utilized as scenario attributes.

2.3 Scenario Clustering

Each extracted feature vector describes a concrete

scenario within the test data. These concrete scenarios

shall be clustered in order to create groups of concrete

scenarios - the logical scenarios. Scenarios within one

cluster shall be similar and scenarios in different clus-

ters shall be distinguishable. The clusters shall be in-

VEHITS 2019 - 5th International Conference on Vehicle Technology and Intelligent Transport Systems

462

terpretable and shall represent relevant scenarios in

the real world. Since the actual number of different

scenarios is unknown, the number of clusters can not

be explicitly deﬁned. The optimal number of clusters

shall be derived implicitly.

3 EXEMPLARY

IMPLEMENTATION

The proposed approach was implemented for a PCC

feature, which not only considers the front vehicle

like an ACC system does but also incorporates addi-

tional environment information into the velocity pre-

diction (Albrecht and Holz

apfel, 2018). The cor-

responding signals for the environment description

were selected from the real-world-driving-data. The

main focus for the scenario extraction was to create

a scenario catalog for functional system level feature

evaluation. The PCC feature considers curves and

slopes, speed limits, road topology, street class, cross-

ings and the front vehicle for the velocity prediction.

3.1 Data Selection and Pre-Processing

A time-based signal s can be mapped onto the track

position p as s(t) → s(p). Equation 1 displays the

mapping for every measuring point i. ∆t refers to the

sampling interval and v

to the measured ego velocity

in point j.

(p) =

∑

j=0

∆t ∗ v

(1)

The differing velocity leads to a non-equidistant

position-based sampling of the formerly equidistant

time-based sampling. Besides, signals on the bus sys-

tem and thereby in the recorded test data are sam-

pled with different frequencies up to 50 Hz and more.

This leads to distances of 0.56 m for a velocity of

130 km/h. Such short distances are not required to

detect scenarios, that can be several hundred meters

long, e.g. elongated curves on highways. We decided

to sample the data at every 2 m. Though, this is a de-

sign choice, we are convinced, that a more detailed

resolution is not required for the scenario extraction.

With the mapping of the signals to the track position,

they are now independent from the altering ego vehi-

cle’s velocity during the test drives. The static envi-

ronment is represented consistently.

Since real-world-driving-data is noisy, signal ﬁl-

tering has to be applied in order to accurately describe

the vehicle’s environment. Initial and error values

have to be handled with respect to the actual signal

values. Oftentimes, these values are written as ones

in the bit-encoding of the CAN- or ﬂexray message.

Therefore, when converting the bit-stream into dec-

imal numbers, initial and error values are converted

to large decimal numbers. They can not be left un-

changed or set to zero, since both could have un-

foreseen effects towards the clustering of the signal.

For example, there is a difference between a missing

speed limit being represented as 300 km/h, 0 km/h

or NaN, since cluster algorithms use relative distance

functions. For missing speed limits we chose NaN

and for initial values the ﬁrst occurring speed limit

was extrapolated. In unlimited areas the speed limit

was set to 210 km/h, which is the maximum sup-

ported velocity for the PCC feature.

3.2 Scenario Extraction

3.2.1 Dynamic Segment Detection

The street type, the speed limit and the classiﬁcation

of urban, extra urban or highway drive were used as

scenario dividing signals. However, for the PCC there

is, besides these low dimensional signals, one con-

tinuous signal, that has a major inﬂuence on the sys-

tem’s behavior: the curvature. Curves limit the max-

imum possible and the maximum tolerated velocity

and hence depict unique scenarios for the longitu-

dinal control, especially in terms of passenger com-

fort. They shall be evaluated separately from other

road segments. By transforming the curvature into

a summable value, the curviness, and then detecting

signiﬁcant changes in the incline of the accumulated

curviness over the track position, curves can be in-

cluded in the scenario separation. The curviness of-

fers a good abstraction of the overall curvature of a

certain road segment and is calculated as shown in

equation 2

Curviness =

angular change

distance

∆γ

∆d

(2)

The angular change can be derived from multiple

sources, e.g. the yaw rate, the steering wheel angle

or wheel angle. However, in our case all these signals

were too noisy to derive a viable curviness. The best

results were achieved by calculating the azimuth from

the GPS points as shown in Figure 6a and 6b. With the

azimuth as the angular change in each GPS point, the

curviness can be calculated by dividing the angular

change ∆γ by ∆d, the sum of half the distance between

i-1

→ p

and half the distance between p

→ p

i+1

Since the GPS is noisy as well, some ﬁltering has

to be applied. The calculated curviness can be used

as an indicator for implausible GPS points by calcu-

lating the maximum possible curviness that complies

Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data

463

 = min{    , 360°  |  |}

0°/360°

north

90°

east

80°

south

70°

west

(a) Azimuth as the horizontal

angular change of the cardi-

nal direction.

∆

i-1

i+1

(b) Calculation of the az-

imuth of each GPS point

based on the two lines p

i-1

→

and p

→ p

i+1

Figure 6: Calculation of the curviness based on the azimuth

of each GPS point.

with trafﬁc regulations. In Germany the trafﬁc regula-

tions state, that vehicles must be able to drive through

a turning circle with a radius r of 12.5 m. With the

help of equation 3 this leads to a circumference C of

78.54 m for the minimal possible turning circle. The

curviness of this turning circle can be calculated as

shown in equation 4. This maximal possible curvi-

ness can be used as a threshold for ﬁltering the GPS. A

curviness above this threshold is very likely caused by

a GPS error (see Figure 7 for different detected GPS

errors). Since the velocity, which is used to calculate

the curviness, can be noisy as well, the threshold was

set to Curviness

max

= 9

C = 2 ∗ π ∗ r = 78.54 m (3)

Curviness

max

∆γ

∆d

360 °

78.54 m

= 4.58

(4)

Additional ﬁltering of the GPS can be achieved by

comparing the distance between two GPS points and

the driven distance derived from the velocity. Again,

due to possible noise, the thresholds were set to 0.5 <

GPS

driven

< 2.0.

Figure 7: Filtered GPS errors with thresholds derived from

the curviness and relation of GPS distance to driven dis-

tance.

Finally, the transition points can be derived by

building a vector for each scenario dividing signal:

the street type, the speed limit, the classiﬁcation of

urban, extra urban or highway drive and the curvi-

ness. As already mentioned, the transition points for

the curviness are derived from signiﬁcant changes in

the incline of the accumulated curviness over the track

position (see Figure 8). The segments can then be

derived by combining the transition point vectors of

all signals: ρ = ρ

street type

∪ ρ

speed limit

∪ ρ

environment

∪

curviness

. The combined transition point vector is then

sorted and duplicates are removed. The remaining el-

ements depict start and end positions of the segments

in the test drive. Merge strategies can be applied, if

the segmentation is too compartmentalized due to a

large number of scenario dividing signals.

8.865 8.866 8.867 8.868 8.869 8.87 8.871 8.872 8.873 8.874

longitude

48.8265

48.827

48.8275

48.828

48.8285

48.829

48.8295

48.83

48.8305

latitude

Figure 8: Segmentation via change points of the accumu-

lated curviness.

3.2.2 Feature Vector Extraction

Besides the scenario dividing signals each segment

needs an adequate description of its environmental

situation in order to be clustered correctly. For each

segment a describing feature vector has to be ex-

tracted. For several reasons, time or position-based

features are not ideal. For one, the sequence length

varies, which either leads to a different amount of val-

ues in the feature vectors or to a different sampling.

Both lead to a decrease in comparability of the se-

quences. The second reason is, that clustering time-

series with different sample sizes is not trivial.

A better solution is to derive single values for ev-

ery feature by means of aggregation, minima or max-

ima, mean values, etc. For the showcase, 10 fea-

tures were selected leading to a feature vector with

10 scalar values for each segment in the test data. For

an example see Table 1.

VEHITS 2019 - 5th International Conference on Vehicle Technology and Intelligent Transport Systems

464

Table 1: Exemplary segments with their corresponding feature vectors (abbrev: Segment Identiﬁer (SID); Speed Limit (SL);

Amount of Occurring Braking Maneuvers (BM); Amount of Occurring Trafﬁc Lights (TL)).

SID Env Street Type Length

seg

SL Curv Slope

pos

Slope

neg

Vel

avg

BM TL

4 rural rural road 54 70 2.85 0 -7.26 28.21 0 0

5 rural rural road 56 70 2.80 0 -7.37 34.46 0 0

8 rural district road 32 100 4.32 7.84 0 28.56 0 0

31 highway highway 1578 140 0.02 0 -1.70 132.20 2 0

32 rural rural road 142 140 1.16 0 -2.28 49.96 0 1

76 rural rural road 78 140 0.56 0 -3.23 40.39 0 1

3.3 Scenario Clustering

The feature vectors can be used to cluster the seg-

ments. Due to the scalar representation common

cluster algorithms can be used. For the showcase,

the K-Medoids with Partitioning around Medoids

(KPAM) algorithms has been implemented (Kaufman

and Rousseeuw, 2009).

K-medoids is a common partitioning cluster algo-

rithm, that is stable against outliers. A set of N data

points is divided into k clusters, so that the sum of

distances between each data point and its correspond-

ing medoid is minimized. The KPAM algorithm is a

deviation of k-medoids with two phases. The set of

data points or observations O is divided into a sub-

set of medoids or selected objects S and a subset of

unselected objects U with |O| = N and U = O \ S.

For every observation o two values are calculated: the

distance D

between o and the nearest medoid in S

and the distance E

between o and the second nearest

medoid in S.

During the build phase initial medoids are se-

lected. The ﬁrst medoid is the observation with the

lowest sum of distances to all other observations in

O. Iteratively k − 1 medoids are selected so that the

new partitioning is as distinct as possible.

During the swap phase the iteratively selected

medoids are swapped in order to improve the parti-

tioning. Every object pair in SxU is evaluated. If

the sum of distances for the object in U is lower

than for the object in S, both objects are swapped.

This process is repeated until the partitioning is sta-

ble. For the detailed algorithm we refer to (Kaufman

and Rousseeuw, 2009).

We implemented the KPAM and searched for the

optimal k by iterating 1 ≤ k ≤ 30 with 5 iterations for

each k to avoid local optima. The best solution for

each k was kept. For the k-medoid algorithm the opti-

mal k was evaluated with the elbow-method (Ketchen

and Shook, 1996) - the optimal k = 24 was derived.

4 EVALUATION

As there is no ground truth by means of classiﬁed sce-

narios for the test drives or an existing scenario cata-

log for the PCC feature, the evaluation has to be done

empirically. In a ﬁrst step, the quality of the cluster-

ing has to be evaluated. In a second step the inter-

pretability of the clusters as logical scenarios is val-

idated. Some information about the used data set is

given in Table 2.

Table 2: Overview of the used data set for the PCC show-

case.

Number of test drives 115

Included Countries 10

Overall Length 8432 km

Shortest Test Drive 1.13 km

Longest Test Drive 299.19 km

Average Length 73.09 km

Amount Urban 1271 km

Amount Extra Urban 7161 km

We want to demonstrate the results by showcasing

three clusters of the KPAM algorithm, for the simple

reason that showing all clusters goes beyond the scope

of this article. Figure 9, 10 and 11 show three exem-

plary clusters of the KPAM clustering algorithm with

six randomly chosen concrete scenarios each. Each

cluster in itself is supposed to be similar, with larger

dissimilarities between the clusters.

Figure 9 depicts concrete scenarios from a logical

scenario, that could be described as straight country

road drive with little slope. Figure 10 depicts strong

curves on steep serpentines. Figure 11 displays longer

highway drive scenarios on a straight road with high

speed limits. All three clusters each contain similar

segments. On closer inspection, scenario 4 in Cluster

19 attracts attention. It is the only scenario not on

a highway. However, further investigation has shown,

that this road section was formerly deﬁned as a federal

highway and is still classiﬁed as such in the electronic

map, which makes the assignment plausible.

On a feature vector basis the clustering algorithm

Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data

465

Figure 9: Six randomly selected concrete scenarios from

cluster 7 depicting straight segments on country roads.

Figure 10: Six randomly selected concrete scenarios from

cluster 14 depicting strong curves and serpentines.

Figure 11: Six randomly selected concrete scenarios from

cluster 19 depicting long highway segments.

has build valid clusters. The question remains, if the

feature vectors represent valid concrete scenarios and

the clusters represent logical scenarios respectively.

Hence, the next step in the evaluation is to exam-

ine the interpretability of the found clusters as logical

scenarios. Each cluster represents a group of simi-

lar feature vectors describing a segment of the driving

data for which the parameter spaces can be derived

from the global maxima and minima for each fea-

ture. In order to represent these clusters in a human-

readable format, a functional scenario for each cluster

is derived:

All concrete scenarios in Cluster 7 can be classi-

ﬁed as country road drives with average velocity on a

straight and ﬂat road segment. All scenarios are extra

urban.

The concrete scenarios in Cluster 14 are on coun-

try roads as well, however, with a steep incline and

strong curves. The curves are segmented accurately.

Cluster 19 represents highway test drives. All sce-

narios are driven at high velocities on highway seg-

ments with no noticeable curve or slope.

For the PCC feature all three logical scenarios de-

pict different use cases with different driving chal-

lenges as well as driving requirements. It can be ar-

gued, that this approach leads to plausible and useful

logical as well as functional scenarios directly derived

from the data.

5 CONCLUSION & FUTURE

WORK

We have shown, that our approach is well suited

to ﬁnd self-contained environmental scenarios within

real-world-driving-data. The data-driven approach

utilizes the data to extract all occurring concrete sce-

narios, that are gathered during test drives. A cluster

algorithm was used to derive logical scenarios.

The logical scenario catalog can now be used

for scenario-based testing and coverage analysis of

the different existing environmental scenarios. Func-

tional quality can be assessed for different situations,

like different curves, serpentines or corner case clus-

ters. An automated process to identify correlations

between the functional quality and the logical scenar-

ios can be set up.

The number of logical scenarios was derived im-

plicitly for the selected signals and the exemplary data

set. In future work, we will alter the data set by vary-

ing the selected test drives as well as the total length

of the included test drives. The convergency point for

the number of clusters k will be determined. The min-

imal accumulated length of the test drives to ensure

convergence will be derived and we will discuss, if

inferences towards the required test kilometers can be

made.

For ADAS features it is common practice to differ-

entiate between highway, urban and extra urban driv-

ing, because the driving tasks and environmental sit-

uations differ a lot between these three contexts. In

order to get more precise clusters within each context,

we will investigate, whether dividing the data before-

hand into three different groups: highway, urban and

extra urban segments, can improve the clustering re-

sult.

Furthermore, scenarios may be found in other sub-

spaces of the complete feature space. E.g. to distin-

guish a curve scenario from a steep incline or a high-

way scenario, different subspaces need to be looked

at. For the highway scenario the street class is rele-

vant, but slope and curviness may not be, since both

VEHITS 2019 - 5th International Conference on Vehicle Technology and Intelligent Transport Systems

466

are usually virtually zero on highways. For steep in-

clines the street class can be neglected but the slope

plays a signiﬁcant role. As these examples demon-

strate, subspaces may play a vital role in detecting dif-

ferent environmental scenarios and, therefore, these

subspaces will be investigated by utilizing projective

clustering algorithms, e.g. K-Means with Splitting

and Merging (KSM). Experiments with the KSM will

also include clustering with only a few clusters to

investigate the most distinctive features. A low k

should result in a distinction of the most separating

subspaces. These will be investigated further.

REFERENCES

Albrecht, M. and Holz

apfel, M. (2018). Vorausschauend

efﬁzient fahren mit dem elektronischen co-piloten.

ATZextra, 23(5):34–37.

Amersbach, C. and Winner, H. (2017). Functional decom-

position: An approach to reduce the approval effort

for highly automated driving. In 8. Tagung Fahreras-

sistenz, M

unchen.

Bach, J., Otten, S., Holz

apfel, M., and Sax, E. (2017).

Reactive-replay approach for veriﬁcation and valida-

tion of closed-loop control systems in early develop-

ment. In SAE Technical Paper 2017-01-1671.

Bagschik, G., Menzel, T., Reschka, A., and Maurer, M.

(2017). Szenarien f

ur entwicklung, absicherung und

test von automatisierten fahrzeugen. In 11. Work-

shop Fahrerassistenzsysteme. Hrsg. von Uni-DAS e. V,

pages 125–135.

Christiansen, M. (2015). 5komma6 - mercedes-

benz: In-geheimer-mission-auf-abnahmefahrt-mit-

der-neuen-mercedes-e-klasse-w213.

de Gelder, E. and Paardekooper, J.-P. (2017). Assessment

of automated driving systems using real-life scenarios.

In Intelligent Vehicles Symposium (IV), 2017 IEEE,

pages 589–594. IEEE.

Deppe, P. (2009). Mercedes-benz passion: Mercedes-benz-

praesentiert-in-genf-limousine-und-coupe-der-neuen-

e-klasse.

Elrofai, H., Worm, D., and den Camp, O. O. (2016). Sce-

nario identiﬁcation for validation of automated driving

functions. In Advanced Microsystems for Automotive

Applications 2016, pages 153–163. Springer.

Gerdes, A. (2006). Automatic maneuver recognition in the

automobile: the fusion of uncertain sensor values us-

ing bayesian models. In Proc. 3rd International Work-

shop on Intelligent Transportation.

Henzel, M., Winner, H., and Lattke, B. (2017). Heraus-

forderungen in der absicherung von fahrerassisten-

zsystemen bei der benutzung maschinell gelernter und

lernender algorithmen. UNI DAS eV (eds.), 11:136–

148.

ulnhagen, T., Dengler, I., Tamke, A., Dang, T., and

Breuel, G. (2010). Maneuver recognition using prob-

abilistic ﬁnite-state machines and fuzzy logic. In In-

telligent vehicles symposium (IV), 2010 IEEE, pages

65–70. IEEE.

ISO, I. (2011). 26262: Road vehicles-functional safety. In-

ternational Standard ISO/FDIS, 26262.

Junietz, P., Schneider, J., and Winner, H. (2017). Metrik

zur bewertung der kritikalit

at von verkehrssituationen

und-szenarien. In 11. Workshop Fahrerassistenzsys-

teme.

Kaufman, L. and Rousseeuw, P. J. (2009). Finding groups

in data: an introduction to cluster analysis, volume

344. John Wiley & Sons.

Ketchen, D. J. and Shook, C. L. (1996). The application of

cluster analysis in strategic management research: an

analysis and critique. Strategic management journal,

17(6):441–458.

Langner, J., Bach, J., Otten, S., Sax, E., and Holz

apfel, M.

(2017). Framework for using real driving data in au-

tomotive feature development and validation. In 8.

Tagung Fahrerassistenz, M

unchen.

Langner, J., Bach, J., Ries, L., Otten, S., Holz

apfel, M., and

Sax, E. (2018). Estimating the uniqueness of test sce-

narios derived from recorded real-world-driving-data

using autoencoders. In 2018 IEEE Intelligent Vehicles

Symposium (IV), pages 1860–1866. IEEE.

Minnerup, P., Kessler, T., and Knoll, A. (2015). Collect-

ing simulation scenarios by analyzing physical test

drives. In Intelligent Transportation Systems (ITSC),

2015 IEEE 18th International Conference on, pages

2915–2920. IEEE.

utz, A., Zlocki, A., and Eckstein, L. (2016). Absicherung

hochautomatisierter fahrfunktionen mithilfe einer

datenbank relevanter szenarien.

Schuldt, F., Saust, F., Lichte, B., Maurer, M., and Scholz, S.

(2013). Efﬁziente systematische testgenerierung f

fahrerassistenzsysteme in virtuellen umgebungen. Au-

tomatisierungssysteme, Assistenzsysteme und Einge-

bettete Systeme F

ur Transportmittel.

Steimle, M., Bagschik, G., Menzel, T., Wendler, J. T.,

and Maurer, M. (2018). Ein beitrag zur termi-

nologie f

ur den szenarienbasierten testansatz automa-

tisierter fahrfunktionen. AAET-Automatisiertes und

vernetztes Fahren. Hrsg. von ITS Niedersachsen,(zur

Ver

offentlichung angenommen).

Wachenfeld, W. and Winner, H. (2016). The release of au-

tonomous vehicles. In Autonomous Driving, pages

425–449. Springer.

Wang, W., Ramesh, A., and Zhao, D. (2018). Clustering

of driving scenarios using connected vehicle datasets.

arXiv preprint arXiv:1807.08415.

Zofka, M. R., Kuhnt, F., Kohlhaas, R., Rist, C., Schamm,

T., and Z

ollner, J. M. (2015). Data-driven simulation

and parametrization of trafﬁc scenarios for the devel-

opment of advanced driver assistance systems. In In-

formation Fusion (Fusion), 2015 18th International

Conference on, pages 1422–1428. IEEE.

Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data

467