The Need for Location-based Machine Learning Models for Level 5

Automated Vehicles

Kaushik Madala and Hyunsook Do

Department of Computer Science and Engineering, University of North Texas, Denton, U.S.A.

Keywords:

Safety Of The Intended Functionality (SOTIF), ISO 21448, ISO 26262, Machine Learning Safety,

Autonomous Vehicle Safety, SAE Level 5 of Driving Automation.

Abstract:

Assuring safety of machine learning (ML) models in autonomous vehicles is a challenging task. This is be-

cause of the complex operational design domain (ODD) settings under which we need to validate ML models.

In particular, autonomous vehicles with level 5 of driving automation need to operate under any ODD condi-

tions, and should ensure safety of both road users and passengers. However, deploying common ML models

across a ﬂeet of vehicles to operate in multiple regions can complicate the safety assurance process.Even when

an ML model is found to be causing a crash due to an ODD condition occurring in only one of the regions, we

still should update it across the ﬂeets of all regions. If we can limit its update within that particular region, we

can reduce the complexity of safety assurance. In this paper, we propose the location-based machine learn-

ing models for level 5 automated vehicles to address this problem and how they will be helpful compared to

deploying instances of global ML models which are same across a company’s ﬂeets of vehicles.

1 INTRODUCTION

Automated vehicles are one of the highly complex

systems that rely on machine learning (ML) models

to perform tasks such as perception and motion plan-

ning (Chen et al., 2015; Aradi, 2020; Pfeiffer et al.,

2017). The widely used machine learning algorithms

are deep neural networks (Grigorescu et al., 2020).

Some of the machine learning models generated us-

ing deep neural networks are object detection mod-

els (used to classify and locate objects), reinforcement

learning models (used to predict a potential next state

based on current state), and adversarial learning mod-

els (used to analyze robustness of other ML models).

While the current state-of-the-art autonomous vehi-

cles still rely on safety-drivers or remote operators

to ensure safety and prevent accidents or operate in

a limited geographical location, the ultimate goal is

to reach SAE level 5 of driving automation (Commit-

tee et al., 2018), where the vehicle can autonomously

travel to any place and ensure safety of passengers

and other road users (e.g., other vehicles, pedestrians)

without human feedback.

Performing safety assurance for autonomous ve-

hicles with level 5 of automation can be very chal-

lenging. This is because of the extensive operation

design domain (ODD) (BSI/PAS, 2020) that we need

to consider for such vehicles. ODD includes fac-

tors such as environmental conditions (e.g., snowy

weather), road users (e.g., pedestrian, bicyclists),

and scenery/road infrastructure (e.g., pavement, road

markings, road type, trafﬁc signs). To ensure safety

of autonomous vehicles, they should comply with the

safety standards: ISO 26262 (ISO, 2018) and ISO

21448 (ISO/PAS, 2020). While ISO 26262 deals with

functional safety (FuSa), ISO 21448 deals with safety

of the intended functionality (SOTIF).

The ML models we use in an autonomous ve-

hicle also need to comply with the ISO 26262 and

ISO 21448 standards. In the case of an autonomous

vehicle with level 5 of driving automation, compli-

ance with ISO 26262 and ISO 21448 requires to sufﬁ-

ciently demonstrate that the vehicle has an acceptable

or reasonable level of risk under all ODD settings. If

we assume that the autonomous vehicles of level 5

are being operated in different regions

across United

States of America (USA), it implies we need to vali-

date safety of ML models in all these regions. Ensur-

ing safety across different regions requires thorough

representative data sets of all regions and correspond-

ing ODD elements. Moreover, we must perform a

A region can be a city or a county or a state or a group of

states

654

Madala, K. and Do, H.

The Need for Location-based Machine Learning Models for Level 5 Automated Vehicles.

DOI: 10.5220/0010483706540661

In Proceedings of the 7th International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS 2021), pages 654-661

ISBN: 978-989-758-513-5

thorough error analysis of these ML models and un-

derstand the impact of outputs from ML models on

the behavior of the automated vehicle after sensor fu-

sion.

After the deployment, if we ﬁnd a situation in

which one of the ML models being used in an auto-

mated vehicle results in a crash under some ODD set-

tings, we will need to update the ML model or mod-

ify software to ensure the risk is reduced. If we update

the ML model, then before deploying the updated ML

model into the automated vehicle, we should perform

change impact analysis (Kretsou et al., 2021) to com-

ply with ISO 26262 and ISO 21448. This process en-

sures that the updated ML model is not compromising

the FuSa or SOTIF of the automated vehicle.

Such an analysis, where we need to ensure safety

of an ML model and automated vehicle over multi-

ple locations/regions, can be highly time-consuming.

If the ﬂeet of vehicles is operating in a limited zone,

the complexity of analysis may be reduced. However,

if the ﬂeet of automated vehicles are heterogeneous,

i.e., there are different vehicle types that use differ-

ent technologies, the safety analysis can be even more

challenging. This complexity can add more difﬁcul-

ties with maintaining global models whose instances

are deployed across a company’s ﬂeet of vehicles.

When we analyze safety of ML models in au-

tomated vehicles, we need to consider the tempo-

ral nature of data, i.e., how data can change with

time, the amount of data being collected from each

vehicle and time taken to scrutinize it, and the co-

existence of vehicles which have no driving automa-

tion or partial driving automation (SAE level 3 or be-

low). These factors add further complications when

using instances of the same global models across the

ﬂeet of cars of a company because data changes with

time and driving behaviors of vehicles with no or par-

tial driving automation can vary with locations.

To overcome these limitations and to reduce the

complexity, we propose the idea of location-based

ML models for automated vehicles. The location-

based ML models will limit the ODD and thereby re-

duce the time taken for change impact analysis and

safety assessment. By using location-based mod-

els, safety assessments can be done per each model,

thereby approving releases along with certiﬁcations

per region. Safety assessment per region can also aid

companies in limiting the recalls of vehicles to a par-

ticular region when an issue is found only within that

region. We detail the engineering aspects we need

to take into account when we use location-based ML

models for automated vehicles to achieve level 5 of

driving automation. Under the assumption that all

regions have respective location-based models, note

that using location-based models does not make an

automated vehicle belong to SAE level 4. The auto-

mated vehicle will still belong to SAE level 5 as it

can still operate anywhere without relying on human

intervention.

The rest of the paper is organized as follows.

Section2 discusses ISO 26262 and ISO 21448, and

what aspects we need to consider for ML safety anal-

ysis according to these standards. Section 3 details the

concept of change impact analysis and its application

to ML models to comply with ISO 26262 and ISO

21448. Section 4 describes location-based ML mod-

els and how they aid the companies from both engi-

neering and safety point of views. Section 5 provides

insights and potential future directions and ﬁnally, we

conclude in Section 6.

2 ISO STANDARDS AND ML

MODELS

Automated vehicles should be compliant with two

ISO safety standards: ISO 26262 (ISO, 2018) and

ISO 21448 (ISO/PAS, 2020). ISO 26262 is the func-

tional safety (FuSa) standard for automotive vehicles

and ISO 21448 is the standard for safety of the in-

tended functionality (SOTIF). FuSa aims at reducing

risks to an acceptable level, which can occur due to

malfunctions of hardware and software in a vehicle.

SOTIF, on the other hand, aims at reducing unknown

risks to a reasonable level and identiﬁes gaps in re-

quirements. SOTIF is performed under the assump-

tion that the system follows its requirements as stated.

The violation of the expected behavior from require-

ments due to hardware and software malfunctions fall

under FuSa.

In the case of an ML model, as part of ISO 26262,

we need to ensure the hardware on which the ML

model is deployed and software which the ML model

uses are not susceptible to malfunctions. ISO 26262

also requires the safety analysts to ensure a change in

ML hardware, ML software, or an ML model does not

compromise the functional safety of the system. The

standard also requires to ensure the conﬁgurations of

ML hardware, ML software, and an ML model are

compatible even after updates.

On the other hand, ISO 21448 requires to analyze

an ML model considering all possible scenarios that

are possible in the ODD in which its predictions can

result in safety issues. This includes ODD conditions

under which an ML model cannot perform well, in-

correct predictions by an ML model, and limitations

of an ML model because of the data used, the ML

training process followed, and an ML model’s depen-

The Need for Location-based Machine Learning Models for Level 5 Automated Vehicles

655

dencies on other ML models if it is relying on their

input to take a decision. ISO 21448 is done in con-

junction with ISO 26262. Hence, an update in an ML

model should be done through change impact anal-

ysis considering both the standards. As a result, an

updated ML model to be compliant with ISO 21448

requires a thorough analysis with respect to its predic-

tions, ODD conditions, as well as its impact on any

dependent ML components.

Note that a given ODD can have many operating

environments, and within each operating environment

many scenarios are possible. Also note that, a sce-

nario in ISO 21448 is deﬁned as a combination of

scenes, events, and actions with a deﬁned set of goals

and values including vehicle maneuvers. According

to ISO 21448, we can deﬁne every scenario as a tem-

poral sequence of situations that can occur within an

operating environment. An ML model makes a deci-

sion based on each situation. For example, if we con-

sider a sequence of frames from a camera, the entire

sequence can be considered a scenario, each frame

can be considered a situation. Hence, validation of

ML models will need to be done at a situation-level to

gain enough conﬁdence for possible scenarios across

the ODD. Thus, analyzing safety of an ML model

even within a limited ODD can be a time-consuming

task. For automated vehicles with level 5 of driving

automation, which need to be able to travel anywhere

without human feedback, performing SOTIF analysis

to ensure compliance with ISO 21448 will require to

analyze a large number of scenarios and correspond-

ing possible situations for them. Also note that in or-

der to have compliance with ISO 21448, we should

also demonstrate that the automated vehicle has a very

low possibility to face unknown hazardous scenarios

by exploring scenarios in the real world or by us-

ing probabilistic modeling and functional decompo-

sition (ISO/PAS, 2020). Hence, performing a change

impact analysis when an ML model is updated can

require a signiﬁcant amount of effort.

3 CHANGE IMPACT ANALYSIS

In this section, we describe the change impact anal-

ysis and what should be considered to perform a

change impact analysis when an ML model is up-

dated/modiﬁed in autonomous vehicles.

Change impact analysis (Kretsou et al., 2021)

is the process of identifying the effect of a change

in a system, typically a change in software. To

date, many researchers proposed various mechanisms

for performing change impact analysis of require-

ments (Goknil et al., 2014), UML models (Briand

et al., 2003), object-oriented programs (Ren et al.,

2004), and aspected oriented programs (Zhao, 2002).

For example, Zhao (Zhao, 2002) proposed a program

slicing based technique for performing change im-

pact analysis of aspect oriented programs. The ma-

jor underlying goal of these change impact analysis

approaches is to identify if the new change added

to a system can affect the prior veriﬁed functional-

ity. ISO 26262 also requires change impact analy-

sis when a component or sub-system in a vehicle is

modiﬁed (ISO, 2018). While there are some existing

approaches on performing change impact analysis in

automotive systems (Cui et al., 2018; K

aßmeyer et al.,

2015; Durisic et al., 2013), none of them have consid-

ered ML models.

The change impact analysis of ML models needs

to identify the effect of the updated ML models on

the FuSa and SOTIF of the automated vehicles. This

implies that we need mechanisms to identify if the up-

dated model might result in any new kinds of incorrect

predictions that the previous version of the ML model

did not, and if the updated model is compatible with

ML software and ML hardware. To better understand

the change impact analysis of an ML model, let us

consider an example of a wrong-way detection sys-

tem in an automated vehicle as shown in Figure 1. As

the ﬁgure shows the wrong way detection system is an

ML model which classiﬁes whether a vehicle is go-

ing in a wrong way based on the predictions received

from the trafﬁc sign identiﬁcation system and road

marking identiﬁcation system, which are the other set

of ML models.

Cameras

of the

vehicle

Road

marking

identification

system

Traffic sign

identification

system

Wrong way

detection

system

Motion planning

and movement

control system

Images

Images with

traffic signs detected

Images with

road markings detected

Figure 1: Example of a wrong way detection system of an

automated vehicle, which uses inputs from trafﬁc sign de-

tection system and road marking detection system.

Let us assume that to detect a new sign, the traf-

ﬁc sign identiﬁcation system’s ML model is updated

and deployed. In this case, as a part of change im-

pact analysis, we should ensure the update in this ML

model not only identiﬁes the new sign but should also

have acceptable performance for the other signs that

are detected previously. We also need to ensure that

the update in this model does not result in any incor-

rect predictions by a wrong way detection model, and

that the updated model is compatible with the hard-

ware and software on which it is deployed. We need

to perform these steps to ensure the addition of the

VEHITS 2021 - 7th International Conference on Vehicle Technology and Intelligent Transport Systems

656

updated model does not lead to new hazards. If the

updated model leads to new hazards, we should en-

sure they have a reasonable and acceptable level of

risk. We can observe from these steps that the change

impact analysis for an ML model even in a limited

ODD can be a tedious and time-consuming process.

4 LOCATION-BASED MODELS

An autonomous vehicle can collect a large amount of

data (in the order of terabytes) in one hour of oper-

ation. This data can also contain information related

to inputs that resulted in crash sequences. Often such

recently collected data including the crash sequence

data are analyzed to propose the solutions to mitigate

the crashes occurred. Such solutions can include re-

training ML models by adding new data, using other

sensor modalities in the car to mitigate the issues, and

modifying the algorithms (e.g., sensor fusion related

information). Hence, updating ML models can be

commonly performed to reduce crashes or to improve

performance and efﬁciency (Yang et al., 2018; Baylor

et al., 2017).

Whenever an ML model is updated, as mentioned

in the previous section, a change impact analysis

should be performed to ensure that the changes in the

ML model do not compromise safety. For safety as-

sessment teams, in order to certify an autonomous

vehicle, they need sufﬁcient evidence, which can

demonstrate that the updated model will not result

in new safety issues that did not exist before. If we

assume the ﬂeet of vehicles to be operating in mul-

tiple regions in a country, then before deploying the

same model across all ﬂeets of vehicles and updating

the models in all vehicles simultaneously, we have to

make sure that the updated model does not result in

crashes in all regions. If the vehicle company decides

to recall the vehicles or shift to autonomous operation

under remote monitoring, the car owners will suffer a

great deal of inconvenience due to the absence or lim-

ited availability of these vehicles during such a ser-

vice period, thereby damaging the company’s repu-

tations. To reduce the effort for performing change

impact analysis, and restriction of operation to spe-

ciﬁc regions in the case of accidents, we propose to

use location-based models.

Figure 2 illustrates the approach that uses

location-based models and how it is different from the

approach that uses a global ML model across multi-

ple regions. In this illustration, we considered three

regions where ﬂeets of vehicles are deployed. As de-

scribed earlier, a global ML model approach (shown

in Figure 2a) deploys the same model across differ-

ent ﬂeets in different regions. On the other hand,

location-based models have region-speciﬁc ML mod-

els. If one of the vehicles is moving from one region

to another, the vehicle should replace the previously

used model with the new region’s model. This will

require a component that fetches the model relevant

to the region (represented as “Model fetcher” in Fig-

ure 2a.

As shown in Figure 2, let us assume that ﬂeets

of vehicles are deployed across three regions such

that region 1 has snowfall between October and April

(e.g., St. Cloud in Minnesota), region 2 has no snow-

fall (e.g., Phoenix in Arizona), and region 3 has snow-

fall only between December and February (e.g., Bal-

timore in Maryland). Let us also assume multiple

crashes were observed in March during snow in re-

gion 1 due to incorrect outputs from the ML model. In

this case, for global ML model deployment as shown

in Figure 2b, when we deploy the updated model af-

ter retraining into ﬂeets of vehicles, it will need to be

deployed across ﬂeets in all three regions. As a result,

we need to ensure the updated model will not compro-

mise safety for any vehicle in the ﬂeets of all three re-

gions. This will require a comprehensive change im-

pact analysis, simulation, veriﬁcation, and sufﬁcient

evidence to show the new model will not increase the

number of safety issues.

On the other hand, for location-based model de-

ployment, if the regional model 1 in region 1 is result-

ing in safety issues due to snowfall in March, the other

regional models need not be updated because 1) the

regional model 1 is trained for ODD corresponding to

region 1, and 2) regions 2 and 3 are not susceptible

to snowfall in March. Hence, the ﬂeets of vehicles in

regions 2 and 3 can continue to operate without updat-

ing their ML models. Since we are only updating the

ML model corresponding to region 1, we should sufﬁ-

ciently demonstrate safety by only considering region

1’s ODD. The ﬂeets in other regions do not need to

wait for the updated model to be deployed across all

regions to continue operation without compromising

safety. This also helps in performing regional anal-

yses of crashes, which makes it easier for analysts

to perform error analysis and ablation studies of ML

models. It also helps us to have a better understanding

about how data changes in each region and thereby to

consider modiﬁcation at both local and global levels

as needed. If such an understanding of how change

of data with time needs to be analyzed at a global

model level, it can be a very difﬁcult task as currently

only limited automation support is available to ana-

lyze such changes. Note that by using location-based

model deployment, if a global model is intended to

be released across all regions, it can be done region

The Need for Location-based Machine Learning Models for Level 5 Automated Vehicles

657

ML models

Region 1

Region 2

Region 3

Region 1

Region 3

Region 2

Instances of

ML models

Instances of

ML models

Instances of

ML models

Regional

model 1

Regional

model 2

Regional

model 3

Snow from Oct-Apr

Snow from Dec-Feb

No snow

Snow from Oct-Apr

Snow from Dec-Feb

No snow

Model

fetcher

Using a global set of ML models across fleets of vehicle from

multiple regions

Using a location-based ML models across fleets of vehicle

from multiple regions

(a) Global ML model deployment vs location-based ML model deployment.

ML model

Region 1

Region 2

Region 3

Region 1

Region 3

Region 2

Instances of

ML model

Instances of

ML model

Instances of

ML model

Updated ML

model

Updated ML

model

Updated ML

model

Snow from Oct-Apr

Snow from Dec-Feb

No snow

Snow from Oct-Apr

Snow from Dec-Feb

No snow

Using a global ML model across fleets of vehicle from multiple regions

An ML Model resulted in a

multiple crashes due to

snow in region 1 in March

Updated ML

model

Updating

model

The updated ML

model needs to be

verified and

validated across

three regions

(b) The effect of update in global ML model deployment.

Region 1

Region 3

Region 2

Regional

model 1

Regional

model 3

Regional

model 2

Snow from Oct-Apr

Snow from Dec-Feb

No snow

Model

fetcher

Using location-based ML models across fleets of vehicles from multiple regions

Region 1

Region 3

Region 2

Regional

model 1

Regional

model 3

Regional

model 2

Snow from Oct-Apr

Snow from Dec-Feb

No snow

Model

fetcher

Region 1’s ML Model

resulted in a multiple

crashes due to snow in

region 1 in March

Update

Region 1

model

No update

The updated ML

model needs to be

verified and

validated across

only region 1

Figure 2: Illustration of global ML model deployment and location-based ML model deployment, and the effect of updating

a problematic ML model in both the approaches.

VEHITS 2021 - 7th International Conference on Vehicle Technology and Intelligent Transport Systems

658

by region (e.g., the updated model can be released in

region 1 initially and released eventually in regions 2

and 3 which do not have possibility for snowfall dur-

ing March), thereby making the global model deploy-

ment convert to location-based model deployment.

Preliminary Analysis: To examine whether it is

possible to have conditions that can be speciﬁc to

a certain region, we conducted a preliminary analy-

sis by considering 50 images picked randomly from

nuScenes dataset (Caesar et al., 2020), and 50 im-

ages from Berkeley Deep Drive (BDD) dataset (Yu

et al., 2020). The nuScenes dataset contains images

collected from the front and rear cameras in Singa-

pore and Boston, where as the BDD dataset has im-

ages collected from the front camera in New York,

San Francisco, Berkeley, and Bay area. We used

Yolo v4 (Bochkovskiy et al., 2020; Wang et al., 2020)

pretrained model on the 100 images with the focus

on identifying pedestrians, trucks, cars, and trafﬁc

lights. In nuScenes data, we found that for the im-

age shown in Figure 3, pedestrians were not detected

due to low visibility of people walking on a sheltered

sidewalk. From BDD dataset, we did not ﬁnd any im-

ages with a sheltered sidewalk similar to the one we

found in nuScenes dataset. This implies that the pres-

ence of the sheltered sidewalk is found in Singapore

or Boston, but not in other locations from which BDD

dataset is collected. From this observation, we can in-

fer that there can be ODD conditions that are speciﬁc

to the regions. While we analyzed very few images,

we believe that analyzing multiple datasets and model

predictions from them might uncover ODD condi-

tions that are available in one dataset but not other

dataset, thereby indicating that location-based mod-

els can be beneﬁcial. If we update the Yolov4 model

to detect the pedestrians under sheltered pathways, we

should reanalyze the conditions under which Yolov4

is unable to predict objects correctly. If such an up-

dated model is used in an automated vehicle, we need

to demonstrate if the updated model is going to result

in any new hazards or not. If the error analysis of the

updated model shows new patterns among incorrect

predictions, we will need to verify the ML model for

various possible scenarios that can occur within the

ODD.

5 DISCUSSION

From the preliminary analysis, we can observe that

location-based models can aid in moving towards

SAE level 5 of driving automation. Both global ML

model and location-based ML model deployment ap-

proaches have their own pros and cons as shown in Ta-

ble 1. For diverse ODD settings, heterogeneous ﬂeets

of vehicles, change of data with time, consideration

of behavior of other road vehicles with no automation

and partial automation, and V2X communications, it

would be better to use a location-based ML models

as it can reduce the effort and time for analysis when

updates are performed to ML models in automated

vehicles. It would also help in performing a thorough

error analysis and in gaining a sufﬁcient conﬁdence

that the automated vehicle is safe enough to be certi-

ﬁed.

Open Questions and Future Directions: While

location-based ML model deployment is helpful,

there are also some open questions and directions we

should investigate.

1. Mechanisms to Change the ML Model with

Regions: When using location-based deployment,

if one vehicle currently operating on one region

needs to travel to another region, the vehicle will

need to replace its ML model once it reaches an-

other region. However, when should the vehicle

request for the model from the other region? What

if there is no service available to remotely down-

load the model at the entrance to another region?

We need techniques and look-a-head scheduling

strategies to ﬁgure out when we need to replace

the model from previous region with the model

from the region we are entering without affect-

ing the continuous ﬂow of driving. Should we

use combination of the models during the transi-

tion from one region to another, or should mod-

els be trained by having some common boundary

space than a strict line of separation between re-

gions? We should investigate to ﬁnd answers to

these questions. We also need to consider means

to ensure a vehicle’s safety if the current model

in the region is not available or redacted due to

some compliance reasons. In such a case, we

need to decide if we continue to use the previ-

ous model or shift to an emergency global safety

model, which operates with a vehicle traveling by

choosing lower speed roads and areas.

2. Mechanisms to Ensure Compatibility of ML

Models Across Regions with ML Hardware and

Software: When an automated vehicle enters a

new region and the ML model from the region is

downloaded to be used, the updated ML model

should have compatibility with ML hardware and

software in the car. If we are not able to ensure

this function automatically, the safety of the au-

tomated vehicle can be compromised. Hence, re-

search efforts in this direction should be consid-

ered.

3. Granularity of Locations: One of the open-end

The Need for Location-based Machine Learning Models for Level 5 Automated Vehicles

659

(a) Original image with pedestrians under sheltered side-

walk highlighted.

(b) Predictions after applying the pre-trained Yolo v4

model to the original image.

Figure 3: Example of image where pedestrians are not detected due to a location-based ODD condition.

Table 1: Pros and cons of global ML model deployment and location-based ML model deployment.

Global ML model deployment Location-based ML model deployment

Pros

• No need to replace the ML model as the

vehicle moves from one region to an-

other.

• Once deployed the ML model can be

used anywhere within the ODD.

• Global model might be able to adapt to

new ODD elements in a region, if they

already exist in other regions.

• A problematic ML model within a loca-

tion/region can be updated without affect-

ing the operation of ﬂeets in other re-

gions/locations and without updating ML

models in other regions.

• It reduces the effort and time required for

change impact analyses and paves the way

for a thorough error analysis and gaining

sufﬁcient evidence for safety certiﬁcation.

Cons

• Signiﬁcantly a large amount of effort re-

quired for change impact analysis, error

analysis, and data selection.

• A failure of an ML model at one region if

resulted in halting of vehicles may result

in decreased reliability for the use of a

company’s service by users.

• Requires a mechanism to replace an ML

model based on locations/regions.

• Lessons learned from error analysis of one

region may not be transferred to other re-

gions.

questions for location-based models is “what level

of granularity should we have for a location or

a region in location-based models?”. Should we

consider a location to be an area within a zipcode

or a city/town/village or a county or a state or a

group of states. We believe that having models

based on the zones in the large cities and hav-

ing models for the entire place for the small city,

town, or village will be a good starting point.

However, more research and studies should be

conducted to identify the suitable and beneﬁcial

level of granularity.

6 CONCLUSIONS

In this paper, we proposed and discussed the location-

based ML model approach in order to achieve safe de-

ployment for automated vehicles with SAE level 5 of

driving automation. We discussed its pros and cons

compared to deploying the same set of ML models

across ﬂeets of vehicles operating in different loca-

tions.

As discussed in the previous section, there are

many research directions with this topic. As part of

future work, we plan to conduct a thorough empirical

study on how location-based model and global ML

model deployment processes affect the change impact

analysis, system engineering process, and availability

VEHITS 2021 - 7th International Conference on Vehicle Technology and Intelligent Transport Systems

660

of vehicles. We also plan to explore what factors we

need to consider for self-learning ML models to as-

sure safety of the vehicle.

ACKNOWLEDGEMENTS

We thank Carlos Avalos-Gonzalez from kVA by UL

for feedback on the topic.

REFERENCES

Aradi, S. (2020). Survey of deep reinforcement learning

for motion planning of autonomous vehicles. arXiv

preprint arXiv:2001.11231.

Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y.,

Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L.,

Koo, C. Y., Lew, L., Mewald, C., Modi, A. N., Poly-

zotis, N., Ramesh, S., Roy, S., Whang, S. E., Wicke,

M., Wilkiewicz, J., Zhang, X., and Zinkevich, M.

(2017). Tfx: A tensorﬂow-based production-scale ma-

chine learning platform. In Proceedings of the 23rd

ACM SIGKDD International Conference on Knowl-

edge Discovery and Data Mining, KDD ’17, page

1387–1395, New York, NY, USA. Association for

Computing Machinery.

Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y. M. (2020).

Yolov4: Optimal speed and accuracy of object detec-

tion. arXiv preprint arXiv:2004.10934.

Briand, L. C., Labiche, Y., and O’Sullivan, L. (2003). Im-

pact analysis and change management of uml mod-

els. In International Conference on Software Main-

tenance, 2003. ICSM 2003. Proceedings., pages 256–

265.

BSI/PAS (2020). 1883:2020. Operational Design Do-

main (ODD) taxonomy for an automated driving sys-

tem (ADS) – Speciﬁcation.

Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E.,

Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Bei-

jbom, O. (2020). nuscenes: A multimodal dataset for

autonomous driving. In Proceedings of the IEEE/CVF

Conference on Computer Vision and Pattern Recogni-

tion, pages 11621–11631.

Chen, C., Seff, A., Kornhauser, A., and Xiao, J. (2015).

Deepdriving: Learning affordance for direct percep-

tion in autonomous driving. In Proceedings of the

IEEE International Conference on Computer Vision,

pages 2722–2730.

Committee, S. O.-R. A. V. S. et al. (2018). Taxonomy and

deﬁnitions for terms related to driving automation sys-

tems for on-road motor vehicles. SAE Standard J,

3016.

Cui, L., Hu, J., Park, B. B., and Bujanovic, P. (2018). De-

velopment of a simulation platform for safety impact

analysis considering vehicle dynamics, sensor errors,

and communication latencies: Assessing cooperative

adaptive cruise control under cyber attack. Trans-

portation Research Part C: Emerging Technologies,

97:1–22.

Durisic, D., Nilsson, M., Staron, M., and Hansson, J.

(2013). Measuring the impact of changes to the com-

plexity and coupling properties of automotive soft-

ware systems. Journal of Systems and Software,

86(5):1275–1293.

Goknil, A., Kurtev, I., Van Den Berg, K., and Spijkerman,

W. (2014). Change impact analysis for requirements:

A metamodeling approach. Information and Software

Technology, 56(8):950–972.

Grigorescu, S., Trasnea, B., Cocias, T., and Macesanu,

G. (2020). A survey of deep learning techniques

for autonomous driving. Journal of Field Robotics,

37(3):362–386.

ISO (2018). 26262. Road Vehicles – Functional Safety.

ISO/PAS (2020). 21448. Road vehicles - Safety of the in-

tended functionality.

aßmeyer, M., Schulze, M., and Schurius, M. (2015). A

process to support a systematic change impact analy-

sis of variability and safety in automotive functions.

In Proceedings of the 19th International Conference

on Software Product Line, SPLC ’15, page 235–244,

New York, NY, USA. Association for Computing Ma-

chinery.

Kretsou, M., Arvanitou, E.-M., Ampatzoglou, A., Deligian-

nis, I., and Gerogiannis, V. C. (2021). Change impact

analysis: A systematic mapping study. Journal of Sys-

tems and Software, 174:110892.

Pfeiffer, M., Schaeuble, M., Nieto, J., Siegwart, R., and

Cadena, C. (2017). From perception to decision: A

data-driven approach to end-to-end motion planning

for autonomous ground robots. In 2017 IEEE In-

ternational Conference on Robotics and Automation

(ICRA), pages 1527–1533.

Ren, X., Shah, F., Tip, F., Ryder, B. G., and Chesley, O.

(2004). Chianti: A tool for change impact analysis of

java programs. SIGPLAN Not., 39(10):432–448.

Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y. M. (2020).

Scaled-yolov4: Scaling cross stage partial network.

arXiv preprint arXiv:2011.08036.

Yang, K., Wang, X., and Yu, R. (2018). A bayesian dy-

namic updating approach for urban expressway real-

time crash risk evaluation. Transportation Research

Part C: Emerging Technologies, 96:192–207.

Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F.,

Madhavan, V., and Darrell, T. (2020). Bdd100k: A

diverse driving dataset for heterogeneous multitask

learning. In Proceedings of the IEEE/CVF Conference

on Computer Vision and Pattern Recognition, pages

2636–2645.

Zhao, J. (2002). Change impact analysis for aspect-oriented

software evolution. In Proceedings of the Interna-

tional Workshop on Principles of Software Evolution,

IWPSE ’02, page 108–112, New York, NY, USA. As-

sociation for Computing Machinery.

The Need for Location-based Machine Learning Models for Level 5 Automated Vehicles

661