Government’s Response to Ethical Dilemmas in Autonomous Vehicle

Accidents: An Australian Policy Evaluation

Amir Raﬁee

1,3 a

, Hugh Breakey

2 b

, Yong Wu

3,4 c

and Abdul Sattar

1,4 d

School of Information and Communication Technology, Grifﬁth University, Brisbane, Australia

Grifﬁth University’s Institute for Ethics, Governance and Law, Brisbane, Australia

Grifﬁth Business School, Grifﬁth University, Gold Coast, Australia

Institute for Integrated and Intelligent Systems, Grifﬁth University, Brisbane, Australia

{amir.raﬁee, h.breakey, yong.wu, a.sattar}@grifﬁth.edu.au

Keywords:

Autonomous Vehicle, AI Policy, AI Ethics, Automated Driving System, AI Regulation, Safety Assurance

System, Self-Driving Cars, Ethical Dilemmas.

Abstract:

As Autonomous Vehicles (AVs) rapidly progress and become widely deployed, governments worldwide grap-

ple with addressing the ethical challenges associated with AVs in dilemma situations that result in loss of

human life. They are tackling these issues through the formulation of policies and guidelines, the establish-

ment of dedicated research centres exploring the ethical implications of AVs, and seeking public opinions

on how self-driving cars should handle such moral dilemmas. In this paper, we will evaluate the Australian

government’s strategies for addressing the ethical issues related to AV accidents. We will critique the Decision

Regulation Impact Statement (DRIS) released by the National Transport Commission (NTC) in 2018, which

assessed the safety assurance options for Automated Driving Systems (ADSs). We will critically examine the

NTC’s decision to exclude ethical considerations for AVs from the DRIS and the Automated Driving System

Entity’s (ADSE) Statement of Compliance. We will contend that safety and ethics are intrinsically linked.

Furthermore, we argue that relying solely on the safety criteria may prove inadequate when addressing ethical

dilemmas. Consequently, we advocate for the inclusion of AV ethical considerations, especially in complex

ethical dilemmas, in future dialogues, even if a clear consensus on ethical decision-making by ADSs remains

elusive. In conclusion, we will propose recommendations for the Australian government to enhance the devel-

opment, deployment, and acceptance of AV technology.

1 INTRODUCTION

The anticipated impact of Autonomous Vehicles

(AVs) on the road transport system, including im-

provements in safety, productivity, environmental

sustainability, mobility, and accessibility, has led to

efforts by countries worldwide to develop regulatory

frameworks that enable the legal and safe operation of

this technology. Additionally, governments are mo-

tivated to position themselves as leaders in promot-

ing emerging technologies and industries that can cre-

ate job opportunities and stimulate export revenues

(NTC, 2022).

However, the introduction and use of AVs also in-

https://orcid.org/0000-0002-8149-4780

https://orcid.org/0000-0002-5170-2091

https://orcid.org/0000-0003-2206-8594

https://orcid.org/0000-0002-2567-2052

troduce new risks, which are particularly concerning

due to the emerging and evolving nature of this tech-

nology. Software malfunctions have been identiﬁed

as a potential risk associated with AVs (Noy et al.,

2018). Safety engineers anticipate that systemic tech-

nical failures or inadequate maintenance and servic-

ing of the Automated Driving System (ADS) could

become serious hazards, much like human errors

(Kira, 2017). Furthermore, governments worldwide

grapple with the ethical challenges of AVs in morally

complex situations, such as ethical dilemmas. These

arise when an AV faces a decision that could poten-

tially harm individuals, such as choosing between the

safety of its passengers or the safety of pedestrians in

a collision (FCAI, 2023).

Governments are tackling these issues through the

formulation of policies and guidelines, the establish-

ment of dedicated research centres exploring the eth-

ical implications of AVs, and seeking public opinions

1152

Raﬁee, A., Breakey, H., Wu, Y. and Sattar, A.

Government’s Response to Ethical Dilemmas in Autonomous Vehicle Accidents: An Australian Policy Evaluation.

DOI: 10.5220/0012451200003636

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2024) - Volume 3, pages 1152-1161

ISBN: 978-989-758-680-4; ISSN: 2184-433X

on how self-driving cars should handle ethical dilem-

mas in accidents that result in loss of human life.

These efforts aim to ensure the safe, ethical, and so-

cially responsible development, deployment, and reg-

ulation of AVs, all while considering the cultural, le-

gal, and societal variations across different countries

and regions (Krueger et al., 2016).

In this paper we will explore the regulatory regime

governing AVs in Australia and evaluate the Aus-

tralian government’s response to address ethical chal-

lenges linked to AV accidents, especially in dilemma

situations. Resolving ethical dilemmas is crucial

for ensuring safety, fostering public trust, complying

with legal frameworks, maintaining moral responsi-

bility, shaping public perception, and preventing un-

intended consequences. Addressing these challenges

is therefore, fundamental to the ethical and responsi-

ble development and deployment of AV technology

(Raﬁee et al., 2023). The second section of our anal-

ysis begins by looking at the safety assessment cri-

teria mentioned in the National Transport Commis-

sion’s (NTC’s) Decision Regulation Impact Statement

(DRIS), which was published in 2018. We will crit-

ically examine the rationale behind their decision to

exclude ethical considerations from the DRIS. The

NTC argued that a separate ethical considerations cri-

terion is unnecessary, as such considerations are al-

ready captured by the proposed safety criteria. Their

concern was that introducing a distinct ethical crite-

rion might result in redundancy and potential conﬂicts

with the established safety criteria. The NTC also ref-

erenced a solution proposed by Swedish academics,

suggesting that if an ADS can assess its operational

capability to handle unexpected situations and adapt

its behavior, it could effectively address ethical dilem-

mas (NTC, 2018; Johansson and Nilsson, 2016). We

contend that effective management of safety risks ne-

cessitates the inclusion of ethical considerations in fu-

ture discussions, and we present arguments support-

ing this stance. In conclusion, we will make some

recommendations for the Australian government and

policy makers to better support the development, de-

ployment, and acceptance of AV technology.

2 SAFETY ASSURANCE FOR

AUTOMATED DRIVING

SYSTEM

In 2018 the NTC published the report, “Safety As-

surance for Automated Driving Systems: Decision

Regulation Impact Statement”. The aim of the DRIS

was to evaluate potential approaches for ensuring the

safety of ADS. This involved an analysis of safety

concerns related to AVs and an examination of possi-

ble government actions in response to these concerns.

The DRIS used a multi-criteria analysis to assess op-

tions for the safety assurance of ADSs. This approach

enabled the assessment of both quantitative and qual-

itative information against criteria linked to expected

impacts, encompassing costs and beneﬁts associated

with various policy options. Given the uncertain na-

ture of future of AVs and the lack of empirical data, a

fully quantitative cost-beneﬁt assessment was deemed

impractical. The analysis focuses on road safety, AV

uptake, regulatory costs, and technology ﬂexibility,

assigning weight based on materiality tests that pri-

oritise signiﬁcant road safety and economic beneﬁts

resulting from increased adoption. The assessment

spawned a 10-year timeframe, employing a compar-

ative scale to rate options against a baseline. The

validity of outcomes was tested against different de-

ployment scenarios, ensuring a robust framework for

evaluating policy impacts on automated vehicle safety

assurance (NTC, 2018).

The DRIS outlined four options to address the

safety assurance of AVs. The baseline option (option

1) did not develop a novel safety assurance system.

Option 2 enhanced the existing regulatory framework

by introducing a safety assurance system, requiring

Automated Driving System Entities (ADSEs) to self-

certify. This option was updated based on feedback

received during the consultation process. Option 3

suggested a nationally administered safety assurance

system, applied at both the ﬁrst supply and in-service

stages. It includes speciﬁc offenses and compliance

measures against non-compliant ADSEs. Option 4

incorporated all elements of Option 3 and introduced

a “primary safety duty”, imposing a general duty on

ADSEs to ensure safety. Stakeholder input has in-

formed updates to these options, ensuring a compre-

hensive consideration of safety assurance measures

for automated vehicles. After considering various

options through consultations, option 2 was recom-

mended.

2.1 Mandatory Self-Assessment

There are eleven safety criteria that the ADSEs must

self-certify against to demonstrate their processes for

managing safety risks before their ADS can be sup-

plied in the Australian market. In examining regu-

latory options for AVs, the assessment centered on

their adaptability to uncertainties. The criteria con-

sidered encompassed the feasibility of implementa-

tion by 2020, in accordance with industry forecasts

and governmental goals. The evaluation also factored

Government’s Response to Ethical Dilemmas in Autonomous Vehicle Accidents: An Australian Policy Evaluation

1153

in the ability to transition in tandem with evolving in-

ternational regulations. These criteria were developed

with the aim of balancing safety and innovation. Ad-

ditionally, the assessment acknowledged the dynamic

development of ADS technology, necessitating regu-

latory responsiveness to emerging safety risks (NTC,

2018).

The safety criteria have been integrated into the

existing framework for the initial supply of vehicles

under the Road Vehicle Standards Act (RVSA) 2018.

The administration of this framework falls under the

Department of Infrastructure, Transport, Regional

Development, and Communications (DITRDC). Fur-

thermore, DITRDC is currently in the process of in-

corporating these safety criteria into the Australian

Design Rules (ADR 90/01). Consequently, there have

been no further amendments to the safety criteria for

initial supply as outlined in the NTC’s DRIS, afﬁrm-

ing its ongoing relevance. Full descriptions of the

safety criteria and obligations, are included in (Ap-

pendix .1).

The safety criteria are considered to be principle-

based since they are formulated as overarching princi-

ples rather than speciﬁc, prescriptive rules. Principle-

based regulations provide a framework of broad prin-

ciples and goals that guide behavior and decision-

making, allowing for innovation and adaptation to

evolving technology. These principles are intended to

guide the development, deployment, and regulation of

AVs in a manner that prioritises safety without being

overly restrictive. They accommodates technological

advancements and allow for a more agile response to

emerging safety concerns and advancements in tech-

nology without the need for frequent regulatory revi-

sions.

In this section, we provide a summary of the four

key principle-based safety criteria set by the govern-

ment for the Statement of Compliance (SOC). The

SOC is a mechanism for transparency and account-

ability, demonstrating that the ADSE takes responsi-

bility for the safety of its ADS and that it aligns with

the principles and requirements set forth in the regula-

tory framework. We focus on these principles as they

are intended to ensure the safe operation of AVs and

are especially important to the decision-making as-

pects of AVs in ethical dilemmas. The NTC suggests

that the safety criteria can serve as a framework for

addressing safety dilemmas with ethical implications.

Consequently, they deem a separate criterion speciﬁ-

cally for ethical considerations as unnecessary. How-

ever, they also acknowledged that these safety criteria

may not assist the ADS in making decisions involving

human lives and emphasised that such decisions can-

not be programmed into AVs regardless (NTC, 2018).

1. Safe System Design and Validation Processes.

Applicants must provide a comprehensive justi-

ﬁcation for their chosen design, validation, and

veriﬁcation procedures, emphasising the system’s

safety throughout its entire operational life. The

applicant must also address safety-critical con-

cerns such as maintenance, repairs, system fail-

ures, and the end-of-life phase of the ADS when

support is no longer available. Additionally, if

the ADS is intended as an aftermarket device, it

should clearly specify compatibility with different

vehicle types in its design.

2. Operational Design Domain (ODD). Applicants

must identify the ODD of the system and demon-

strate its ability to operate safely within this de-

ﬁned domain. It should also show that the ADS

cannot operate outside of its deﬁned ODD and has

the capability to transition to a minimal risk con-

dition if it ﬁnds itself outside this domain. For ex-

ample, if the ADS cannot safely operate at night

or in certain weather conditions, then these should

be outside the scope of its deﬁned ODD, and the

ADS should be incapable of operating in such

conditions (NTC, 2018).

3. Minimal Risk Condition. The ADS may be un-

able to operate safely where there are system

faults, including as a result of a crash, where there

is a deterioration of vehicle hardware or physical

tampering, or where the ADS is outside its ODD.

Applicants are obligated to provide evidence of

the ADS’s capability to recognise such situations

where safe operation is not feasible, along with

the measures it will employ to mitigate risks.

4. On-road Behavioural Competency. Applicants are

required to demonstrate how the ADS will re-

spond effectively to various situations, to ensure

safe operation. This involves actions like disen-

gaging the system or safely stopping the vehicle.

The documentation should also outline how the

system will interact safely with other road users,

covering areas such as detecting objects, avoiding

accidents, handling unusual or unexpected situa-

tions within its ODD, and interacting with differ-

ent road users, including those who may be vul-

nerable.

2.2 Exclusion of Ethical Considerations

In the DRIS, the NTC explained that while they

recognise that ADSs might encounter safety dilem-

mas with ethical implications, they do not consider

it necessary to include a separate ethical considera-

tions criterion as they are already largely captured by

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

1154

the safety criteria. They argued that introducing a

separate ethical considerations criterion might lead to

substantial redundancy and possible conﬂicts with the

existing safety criteria (NTC, 2018). The NTC also

highlighted that there might be a potential solution

for addressing safety dilemmas with ethical implica-

tions in ADSs, referencing a conference paper by two

Swedish academics (Johansson and Nilsson, 2016).

This solution suggests that if an ADS can evaluate its

operational capability to handle unexpected situations

and adjust its behaviour accordingly, it might help re-

solve ethical dilemmas. Consequently, they decided

to exclude ethical considerations from their discus-

sions.

3 DISCUSSION

In this section we will critically examine the NTC’s

decision to exclude ethical considerations for AVs

from the DRIS and the ADSE’s SOC. We will as-

sess each argument and offer our perspective on why

these rationales may not be adequate to justify the ex-

clusion of ethical considerations from future discus-

sions. We will contend that safety and ethics are in-

herently connected, with ethics playing a vital role in

safety by establishing acceptable levels of harm. Fur-

thermore, we argue that relying solely on the safety

criteria may prove inadequate when addressing ethi-

cal dilemmas, especially considering that the capabil-

ities of forthcoming AVs at levels 4 and 5 of automa-

tion have been overly exaggerated. These are vehi-

cles where no driver is ever needed, or there might be

an option for human override but not a requirement

(Shadrin and Ivanova, 2019). Consequently, we ad-

vocate for the inclusion of AV ethical considerations

as part of future discussions, particularly when deal-

ing with complex ethical dilemmas. It is imperative

to establish this as a mandatory criterion, even if a

clear consensus on ethical decision-making by ADSs

remains elusive.

3.1 Argument-1 “Ethical

Considerations Captured by Safety

Criteria”

The initial argument revolved around the NTC’s per-

spective that ethical concerns were mostly addressed

by the safety criteria. These criteria require ADSEs

to self-certify, demonstrating how they manage safety

risks before introducing their ADS to the Australian

market. They argued that introducing a separate ethi-

cal considerations criterion might lead to substantial

redundancy and possible conﬂicts with the existing

safety criteria (NTC, 2018).

Firstly, we deﬁne moral dilemmas in the context

of AVs. Subsequently, we present a hypothetical sce-

nario illustrating such dilemmas. While this exam-

ple may seem improbable, it remains conceivable and

thus merits examination. We will then assess the ad-

equacy of the four safety criteria emphasised by the

NTC in the DRIS for addressing such situations.

Subsequently, we shall assess whether these crite-

ria align with speciﬁc objectives aimed at ensuring the

ethical acceptability and fairness of AVs (Raﬁee et al.,

2023). This evaluation holds signiﬁcance, especially

in achieving widespread AV adoption, which is nec-

essary in unlocking the potential beneﬁts associated

with fully autonomous vehicles on our roads (Maurer

et al., 2016).

3.1.1 Ethical Dilemmas Deﬁned

An ethical dilemma refers to a situation in which the

ADS faces a complex moral decision that may result

in the loss of human life, and there is uncertainty or

disagreement about the right course of action. These

dilemmas arise from the need for the self-driving car

to make split-second decisions in real-world scenarios

where there may be conﬂicting ethical principles or

values. For instance, should it prioritise protecting its

passengers even if it means causing harm to pedestri-

ans? There might be other situations where the ADS

faces conﬂicting priorities. Exceeding the speed limit

for example to avoid a dangerous situation could be

viewed as an ethical decision that conﬂicts with legal

compliance (Nyholm and Smids, 2016).

As the development of AVs progresses, address-

ing these ethical dilemmas becomes a critical aspect

of ensuring the responsible and ethical deployment

of AV technology. However, reaching a consensus

on the best ethical framework is challenging, as soci-

etal values and individual preferences may vary. The

dilemma scenarios would require manufacturers to in-

corporate regulatory rules for the behaviour of AVs

during programming. The ethics programming would

provide instructions on how to respond to situations

when an accident appears imminent (Wu, 2020). This

is not simply pre-setting the conditions, but these ve-

hicles must show behavioural patterns in situations of

necessity, where protecting one interest unavoidably

inﬂicts harm on another (Coca-Vila, 2018).

3.1.2 The Safety Criteria Conundrum

In a highway scenario where an AV is closely fol-

lowed by a motorcyclist, an ethical dilemma arises

when a child unexpectedly appears between parked

Government’s Response to Ethical Dilemmas in Autonomous Vehicle Accidents: An Australian Policy Evaluation

1155

cars. Faced with only two options, the AV can either

engage the brakes to protect the child, resulting in a

deadly collision with the motorcyclist, or maintain-

ing its course, causing harm to the pedestrian to spare

the motorcyclist. This crash scenario demonstrates a

situation where the AV must make instant decisions

without the possibility of manual takeover, common

in levels 4 or 5 of driving automation. Achieving a

minimal risk condition, where all parties avoid seri-

ous harm, is challenging in such circumstances, em-

phasising the complexity of ethical decisions in AV

technology.

To enable an ADS to effectively respond to un-

expected situations, the ADSE must demonstrate its

ability to anticipate and plan for a wide range of po-

tential crash scenarios. However, it is essential to

acknowledge that, as evidenced by incidents such as

the Tesla Model S and Uber Technologies crashes in

2016 and 2018 respectively, anticipating every con-

ceivable crash scenario is nearly impossible (Yadron

and Tynan, 2016). Technological progress occurs

incrementally, and it would be unrealistic to expect

ﬂawless AV deployment. Drawing parallels with the

development of modern commercial airliners, which

took decades to achieve nearly zero airline crashes

through the gradual improvement of autopilot systems

and computer integration (Dewilde, 2022), empha-

sises the time and evolution required for advancing

AV technology.

The ADS faces additional challenges compared

to modern commercial airliners due to other factors,

such as complex roadways, variable speed limits,

changing environmental conditions, and the presence

of manually operated vehicles. These factors, com-

bined with the ADS’s ability to independently iden-

tify and manage risks, introduce further complexity.

Such variability might result in ADSs reacting differ-

ently to identical dilemma situations, creating unpre-

dictability for both other ADSs and fellow road users.

We have argued that presuming future AVs at lev-

els 4 and 5 of automation are immune to crashes and

moral decision-making scenarios is improbable. Even

if the ADSE adheres to the NTC’s safety criteria, it is

crucial to assess their ADS against speciﬁc objectives

proposed by some authors to ensure ethical accept-

ability and fairness (Raﬁee et al., 2023). Firstly, en-

suring transparency regarding programmed priorities

and assigning blame in case of accidents is crucial.

This transparency not only informs potential AV cus-

tomers about how the AV will handle moral dilemmas

but also allows policymakers to anticipate and address

legal implications. Secondly, the ADSE must min-

imise public outrage and discrimination by demon-

strating that their ADS treats all individuals fairly, re-

gardless of factors like age, gender, race, or social sta-

tus. Finally, to foster public acceptance, manufactur-

ers and policymakers must address concerns that may

discourage potential AV buyers about their decision

to adopt AV technology (Raﬁee et al., 2023).

The outlined objectives serve as a foundational

guide for policymakers and manufacturers in navigat-

ing moral dilemmas related to potential human life

loss in autonomous cars. However, the ethical land-

scape of AVs is complex and continually evolving.

Policymakers must recognise the need for additional

objectives, and their selection should be approached

with caution, given potential conﬂicts. Public en-

gagement and interdisciplinary collaboration involv-

ing ethicists, sociologists, psychologists, and technol-

ogists are crucial for a comprehensive understanding

of ethical implications (Jobin et al., 2019). Value con-

ﬂicts may arise, necessitating transparent decision-

making processes to address trade-offs, demographic

prioritisation, and algorithmic transparency. The cho-

sen objectives should reﬂect a balanced approach, ac-

knowledging tensions and striving for ethically sound

solutions. A commitment to responsible develop-

ment, ongoing dialogue, and adaptability will con-

tribute to a robust ethical foundation, fostering public

trust and aligning AV technology with societal values

(Amodei et al., 2016).

3.2 Argument-2 “Existing Solution to

AV Moral Challenges”

The NTC’s second reason for the exclusion of ethi-

cal principles criterion in the DRIS, was inﬂuenced

by a proposed solution for AVs in dealing with moral

dilemmas. The proposal originated from a conference

paper by two Swedish academics, which suggests that

if an ADS can assess its operational capacity to handle

unexpected situations and adjust its behaviour accord-

ingly, the probability of facing ethical dilemmas will

be relatively low (Johansson and Nilsson, 2016). In

our assessment we will examine the assumptions that

were used in the paper and evaluate the arguments in

order to determine their soundness.

3.2.1 Irrelevance of Trolley like Scenarios

Johansson and Nilsson ﬁrst argue that other publi-

cations regard the “Trolley Problem” like thought-

experiments as a real and important issue to address

in the context of AVs and not as a constructed prob-

lem used to discuss ethical dilemmas. Through this

assertion, their aim is to demonstrate that such events

are rare or nearly impossible, deeming them irrelevant

(Johansson and Nilsson, 2016). The “Trolley Prob-

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

1156

lem” is a classic ethical dilemma involving a runaway

trolley on a track about to harm and kill ﬁve people.

You have the choice to either divert the trolley to a

side track by pulling a lever, causing harm to one per-

son, or do nothing and let it harm the ﬁve people on

the main track. The determination of the “correct”

decision remains a subject of ongoing philosophical

debate, as both choices appear reasonable and defen-

sible. It is essential to clarify that the value lies not in

arriving at a deﬁnitive answer but rather to cultivate

the ability to engage in meaningful debates for either

option. This perspective is particularly highlighted by

the different variations explored in Thomson’s 2019

study, emphasising the complexities inherent in such

ethical dilemmas (Thomson, 2019).

Although the “Trolley Problem” might appear ab-

stract and even fantastical due to the simplicity of its

setup and the intricacies of its variations, it is deemed

a valuable representation of moral dilemmas. Un-

doubtedly AVs will encounter complex moral situa-

tions that cannot be overlooked. With speciﬁc re-

gard to AVs, the notion of a singular, precisely de-

ﬁned moral dilemma may be deemed highly unlikely.

ADS may have the ability to optimise their behavior

over a time horizon, minimising the likelihood of fac-

ing a singular, immediate moral dilemma. It might be

reasonable to argue that ASDs should then prioritise

minimizing the risk of encountering such dilemmas in

the future. For instance, in a moral dilemma involv-

ing an AV, the vehicle could proactively slow down to

provide more reaction time, signal to nearby vehicles

to maintain a safer distance, or prompt a motorcyclist

to either keep a greater distance or pass the vehicle

safely. Regardless, the fact remains both taking ac-

tion and non-action can lead to severe consequences

for human well-being. In such cases, a thoughtful

evaluation of the harm imposed, often affecting in-

nocent third parties not initially part of the situation,

is crucial when comparing the potential loss of life

resulting from non-action or default action.

3.2.2 Vienna Convention

The Vienna Convention (VC) on Road Trafﬁc, estab-

lished in 1968, serves as an international treaty with

the goal of harmonising rules and regulations for road

trafﬁc among its participating nations. The conven-

tion aims to facilitate international road trafﬁc and

improve road safety by standardising trafﬁc regula-

tions. Citing Article 8 of the VC, the authors em-

phasise the signiﬁcance of drivers maintaining control

over their vehicles, underlining the driver’s responsi-

bility to effectively manage their vehicles for overall

road safety (UNECE, 1968). The authors contend that

this argument implies drivers should possess the fore-

sight to anticipate dilemma situations and adjust their

driving behavior accordingly. They propose a shift

in focus from addressing ethical dilemmas to educat-

ing drivers on accident avoidance. Emphasising the

societal responsibility placed on drivers, they point

out that obtaining a driver’s license requires demon-

strating the ability to control the vehicle, even in un-

foreseen events (Johansson and Nilsson, 2016). They

present a scenario where a child unexpectedly appears

in front of the car, highlighting that the driver can-

not hold the child responsible. They suggest that it is

the driver’s duty to anticipate such situations through

foresight and adjust their driving behaviour accord-

ingly. Their recommended approach involves direct-

ing attention to implementing robust safety measures

in AVs to proactively prevent such scenarios.

Firstly, the challenge individuals encounter in

consistently behaving ethically, even after learning

correct conduct, arises from a complex interplay of

psychological, behavioral economic, and ethical fac-

tors. Psychological factors such as cognitive disso-

nance and moral licensing play crucial roles. Cogni-

tive dissonance emerges when personal beliefs clash

with actions, leading individuals to justify unethi-

cal behavior. Moral licensing occurs when prior

ethical actions permit subsequent unethical conduct

(Schwartz, 2008; Kahneman, 2011; Bandura, 2014).

From a behavioral economics standpoint, individu-

als often display predictably irrational behavior inﬂu-

enced by cognitive biases (Kahneman, 2011). Time

discounting results in people prioritising immediate

gains over long-term ethical considerations. Ethi-

cal decision-making is further complicated by vari-

ations in moral development and situational ethics.

Moral development inﬂuences an individual’s capac-

ity to prioritise ethical considerations over personal

interests, while situational factors affect the consis-

tent application of ethical principles across diverse

scenarios (Kohlberg, 1973; Bandura, 2014). Addi-

tionally, making thoughtful ethical decisions involv-

ing various alternatives with speciﬁc stakes in unfa-

miliar situations requires time, focus, and careful con-

sideration. In emergency situations, drivers lack these

resources, relying on instinct and intuition, often in a

surprised and panic-stricken manner. Ethics training

offers no beneﬁt in such scenarios, resulting in zero

net effect. However, the programming of AVs may

inﬂuence their performance in ethical dilemmas due

to their capabilities.

Secondly, we dispute the authors’ claim that

“dealing with dilemma situations are not covered in

any driver instruction textbook, as the primary fo-

cus has been on education drivers to avoid acci-

dents”. A more precise statement might be that spe-

Government’s Response to Ethical Dilemmas in Autonomous Vehicle Accidents: An Australian Policy Evaluation

1157

cialised training speciﬁcally designed to prepare hu-

man drivers for such situations is not mandatory. At

least in Australia, safe driving courses are available

and play a crucial role in educating drivers about re-

sponsible and safe driving practices, covering both

theoretical knowledge and practical skills. These

courses provide instruction in defensive driving tech-

niques, equipping participants with strategies to antic-

ipate and effectively respond to potential road hazards

(PCYC, 2023; Sense, 2023).

Finally, we could draw a ﬁtting analogy between

the programming of AVs to navigate ethical dilemma

situations and the concept of insurance. Much like

individuals choose car or home insurance to safe-

guard their assets, even if they never end up ﬁling

a claim, the proactive preparation of AVs for ethical

decision-making can be seen as a form of assurance.

While AVs are anticipated to be signiﬁcantly safer

than human drivers, the foresighted consideration of

how they would handle moral dilemmas is similar to

an insurance policy, providing a layer of preparedness

for unforeseen situations.

3.2.3 Core Safety Proposition

The authors make a reference to the ISO26262 ter-

minology, which is an international standard for

functional safety of electrical and electronic systems

within the automotive industry. It provides guidelines

and requirements to ensure the safety of automotive

systems, focusing on the prevention and control of

hardware and software failures that could lead to haz-

ardous situations. The standard is applicable to the

entire life-cycle of a vehicle, from concept to decom-

missioning (Hommes, 2012). They explain that, in

the context of functional safety, the risk is measured

by taking into account the severity and frequency of a

hazardous event. If the event’s severity is very low or

the likelihood of a hazardous event is deemed highly

improbable (E0), there is no requirement for a speciﬁc

risk measure, and in the context of AV, ethical con-

siderations can be disregarded. The authors proceed

with the assumption that once all hazardous events,

even those with minimal probability, are taken into

account, it can be asserted that the ADS functionality

behaves safely (Johansson and Nilsson, 2016).

The authors suggest that operational algorithms

play a crucial role in determining tactical moves in

driving scenarios. Using the example of an overtak-

ing manoeuvre, the operational capability of a system

is key in deciding whether a tactical move is executed.

If the operational algorithm is conﬁdent in its capabil-

ities, it will implement the manoeuvre. However, if a

task is considered “risky”, it should be avoided. The

level of operational capability inﬂuences the driving

style, with higher capability allowing for a “normal”

style and lower capability resulting in a more conser-

vative, but always safe, driving approach (Johansson

and Nilsson, 2016).

Their argument is primarily based on the idea

that all potential risks can be identiﬁed in advance,

a notion we previously discussed as nearly impossi-

ble. Additionally, we clariﬁed that the progress of AV

technology will occur gradually, resembling the evo-

lution seen in commercial airliners, meaning that it

will not attain perfection overnight. This highlights

the importance of ethical considerations in the devel-

opment and deployment of AV technology.

Even if we assume that an ADS could anticipate

every hazardous scenario beforehand, there are situa-

tions where it would have to limit its tactical choices

to match the operational capability necessary for plan-

ning the most unexpected events. This could reduce

its operational capability to the extent that coming to

a stop would be deemed the safest tactical choice for

a particular scenario. Such unpredictable starts and

stops would not only perplex other ADSs but also

manual drivers and other road users, including pedes-

trians, who would not be able to anticipate the AV’s

actions due to their radical nature. In other words, a

default policy of “just stop” can be a risky and unpre-

dictable default response, compared to more sophisti-

cated ethical actions.

4 RECOMMENDATION FOR

AUSTRALIAN

POLICYMAKERS

An immediate and widespread deployment of fully

automated vehicles appears unlikely, particularly in

the near future. Instead, the progression of AVs is an-

ticipated to follow a gradual trajectory, evolving from

controlled and specialised conditions (operational de-

sign domains) to more complex scenarios. This grad-

ual approach, coupled with extensive planning, regu-

latory considerations, and infrastructure development,

carries signiﬁcant implications for how the govern-

ment should anticipate and handle these transforma-

tive changes (iMOVE, 2023). In this section, we have

compiled a set of recommendations speciﬁcally tai-

lored for the Australian government and policymak-

ers, with the goal of enhancing the safety of ADSs

and fostering greater acceptance within the commu-

nity.

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

1158

4.1 Regulation

Traditionally, governments have been responsible for

overseeing road and vehicle safety, and community

feedback consistently indicates an expectation for

government involvement in ensuring the safety of

AVs (NTC, 2017). Given the novelty of these ve-

hicles and the uncertainty surrounding their safety

performance, government engagement offers reassur-

ance and clarity to industry, insurers, and consumers,

establishing a foundational safety standard as AVs be-

come more common.

The absence of clear regulations may leave gov-

ernments without the means to ensure safety, causing

uncertainty among various stakeholders. Research

suggests the necessity for a delicate regulatory ap-

proach that balances two conﬂicting goals. First,

it aims to protect public safety from potential haz-

ards arising from underdeveloped and inadequately

engineered ADSs that may lead to accidents. Sec-

ond, it aims to promote technological advancements

in vehicle technology for future safety improvements

(Nowakowski et al., 2015). Although the government

cannot directly dictate how car manufacturers pro-

gram their ADS, they can establish ethical rules and

guidelines prioritising safe testing over technological

advancements for public safety (iMOVE, 2023).

4.2 Educate and Inform

The government can play a crucial role in fostering

public acceptance of AVs by implementing strategic

educational initiatives. Collaborating with commu-

nity leaders and early adopters from diverse back-

grounds can serve as a catalyst for broader accep-

tance within the community. Public information cam-

paigns, highlighting ongoing and upcoming trials and

deployments, can be instrumental in disseminating

key information. Emphasis should be placed on com-

municating the beneﬁts of ADS, such as enhanced

mobility and accessibility, while addressing prevalent

concerns related to safety, costs, and liability. Tack-

ling these critical issues head-on can serve as a pivotal

factor in fostering greater acceptance (Orr, 2003).

Additionally, trust in new technology often grows

with increased exposure and repeated use. Encour-

aging pilot programs and demonstrations that allow

the public to interact with ADS in controlled environ-

ments can contribute signiﬁcantly to building trust.

By proactively addressing concerns, showcasing ben-

eﬁts, and involving community inﬂuencers, the gov-

ernment can actively contribute to the successful inte-

gration and acceptance of AV technology.

4.3 The Ethical Dimension

The development and deployment of AVs is intrinsi-

cally linked to both safety and ethics. While AVs hold

the potential to enhance road safety, their introduction

poses ethical challenges related to decision-making in

complex scenarios. The safety risk of AVs cannot be

adequately managed unless ethical considerations are

incorporated into discussions and the development of

these technologies.

The interplay between ethics and safety is evi-

dent in the need to ensure that AVs are programmed

to prioritise safety while making ethically sound de-

cisions. Developers, regulators, and the wider pub-

lic must be engaged in discussions about the ethical

considerations that shape AV behavior. The decision-

making process should be transparent and subject to

public scrutiny. The programming of these algorithms

should ensure that ethical choices are in harmony with

the values upheld by society.

As we previously discussed, AVs will be equipped

with algorithms that determine how they respond to

various scenarios. Their design should focus on acci-

dent avoidance whenever feasible. In the exceptional

instances where an accident is unavoidable, AVs need

to be programmed to decide how harm is allocated

among all parties involved, whether they are passen-

gers, pedestrians, or other road users.

5 CLOSING REMARKS

While the NTC’s recommended safety criteria offer a

foundation for addressing safety dilemmas that carry

ethical implications, they remain at a high level and

lack speciﬁc guidance for ADSEs in navigating the

intricate task of programming their ADS with consid-

erations for the mentioned principles. The core idea

behind the safety criteria is that every AV should op-

erate within its predeﬁned operational design or be

capable of transitioning into a minimal risk condition

in emergency situations. However, we have demon-

strated that this might not always be feasible, as there

are instances where the ADS might need to make dif-

ﬁcult choices that could result in signiﬁcant injury or

even the loss of human life.

The German Ethics Commission on Automated

Driving, established by the Federal Ministry of Trans-

port and Digital Infrastructure, addresses the ethical

considerations related to the programming of AVs.

One of the key principles endorsed by the commis-

sion opposes the programming of AVs to make de-

cisions between human lives, emphasising a com-

mitment to avoiding scenarios where AV program-

Government’s Response to Ethical Dilemmas in Autonomous Vehicle Accidents: An Australian Policy Evaluation

1159

ming intentionally prioritises one life over another

(Luetge, 2017). However, the NTC, while acknowl-

edging the German Ethics Commission’s stance, re-

jects the necessity for such considerations and discus-

sions, asserting that programming decisions involving

human lives are deemed impractical under any cir-

cumstances. We recognise the rarity of dilemma sit-

uations and the current state of AV technology, and

therefore understand that prioritising one road user

over another for the sake of equal treatment may not

be feasible. However, as mentioned earlier, these in-

evitable situations are likely to be encountered by an

ADS eventually. Therefore, we assert that the inclu-

sion of ethical considerations for AVs, especially in

complex ethical dilemmas, should be part of future

discussions, even in the absence of a clear consensus

on ethical decision-making by ADSs.

REFERENCES

Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schul-

man, J., and Man

e, D. (2016). Concrete problems in

ai safety. arXiv preprint arXiv:1606.06565.

Bandura, A. (2014). Social cognitive theory of moral

thought and action. In Handbook of moral behavior

and development, pages 69–128. Psychology press.

Coca-Vila, I. (2018). Self-driving cars in dilemmatic situ-

ations: An approach based on the theory of justiﬁca-

tion in criminal law. Criminal Law and Philosophy,

12(1):59–82.

Dewilde, M. (2022). A comparative study between

aerospace manufacturers boeing and airbus.

FCAI (2023). FCAI submission to NTC discussion paper

on control of automated vehicles.

Hommes, Q. V. E. (2012). Review and assessment of the iso

26262 draft road vehicle-functional safety. Technical

report, SAE Technical Paper.

iMOVE (2023). Promoting community readiness and up-

take of cavs. Online.

Jobin, A., Ienca, M., and Vayena, E. (2019). The global

landscape of ai ethics guidelines. Nature machine in-

telligence, 1(9):389–399.

Johansson, R. and Nilsson, J. (2016). Disarming the trolley

problem–why self-driving cars do not need to choose

whom to kill. In Workshop CARS 2016-Critical Auto-

motive applications: Robustness & Safety.

Kahneman, D. (2011). Thinking, fast and slow. macmillan.

Kira, J. (2017). Consultant report: Safety assurance system

for automated vehicles in Australia.

Kohlberg, L. (1973). The claim to moral adequacy of a

highest stage of moral judgment. The journal of phi-

losophy, 70(18):630–646.

Krueger, R., Rashidi, T. H., and Rose, J. M. (2016). Prefer-

ences for shared autonomous vehicles. Transportation

research part C: emerging technologies, 69:343–355.

Luetge, C. (2017). The german ethics code for automated

and connected driving. Philosophy & Technology,

30:547–558.

Maurer, M., Gerdes, J. C., Lenz, B., and Winner, H. (2016).

Autonomous driving: technical, legal and social as-

pects. Springer Nature.

Nowakowski, C., Shladover, S. E., Chan, C.-Y., and Tan,

H.-S. (2015). Development of california regulations to

govern testing and operation of automated driving sys-

tems. Transportation Research Record, 2489(1):137–

144.

Noy, I. Y., Shinar, D., and Horrey, W. J. (2018). Automated

driving: Safety blind spots. Safety Science, 102:68–

78.

NTC (2017). Regulatory options to assure automated vehi-

cle safety in Australia: discussion paper.

NTC (2018). Safety assurance for automated driving sys-

tems decision regulation impact statement.

NTC (2022). Regulatory framework for automated vehicles

in Australia.

Nyholm, S. and Smids, J. (2016). The ethics of accident-

algorithms for self-driving cars: An applied trol-

ley problem? Ethical theory and moral practice,

19(5):1275–1289.

Orr, G. (2003). Diffusion of innovations, by everett rogers

(1995). Retrieved January, 21:2005.

PCYC (2023). Pcyc driver education.

Raﬁee, A., Wu, Y., and Sattar, A. (2023). Philosophical and

legal approach to moral settings in autonomous vehi-

cles: An evaluation. In Social Licence and Ethical

Practice, volume 27, pages 95–114. Emerald Publish-

ing Limited.

Schwartz, H. (2008). Predictably irrational: the hidden

forces that shape our decisions. Business Economics,

43(4):69–72.

Sense, R. (2023). Road sense driver education course.

Shadrin, S. S. and Ivanova, A. A. (2019). Analytical review

of standard sae j3016 ”taxonomy and deﬁnitions for

terms related to driving automation systems for on-

road motor vehicles” with latest updates. Avtomobil’.

Doroga. Infrastruktura., (3 (21)):10.

Thomson, J. J. (2019). Killing, letting die, and the trolley

problem. In Death, Dying and the Ending of Life, Vol-

umes I and II, pages V2 17–V2 30. Routledge.

UNECE (1968). Convention on road trafﬁc done at vienna

on 8 november 1968.

Wu, S. S. (2020). Autonomous vehicles, trolley prob-

lems, and the law. Ethics and Information Technology,

22(1):1–13.

Yadron, D. and Tynan, D. (2016). Tesla driver dies in ﬁrst

fatal crash while using autopilot mode. The Guardian.

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

1160

APPENDIX

Safety Criteria for Automated Driving

Systems

The ADSE, rather than government, will be responsi-

ble for testing and validating the safety of the ADS or

function and documenting these processes. The role

of government is to satisfy itself that the applicant has

processes in place to identify and manage the safety

risks. This broad approach was agreed to by trans-

port ministers in November 2017, subject to analysis

through a Regulation Impact Statement (NTC, 2018).

Eleven safety criteria that require the applicant to

demonstrate its processes for managing safety risks:

1. Safe system design and validation processes

2. Operational design domain (ODD)

3. Human–machine interface (HMI)

4. Compliance with relevant road trafﬁc laws

5. Interaction with enforcement and other emer-

gency services

6. Minimal risk condition

7. On-road behavioural competency

8. Installation of system upgrades

9. Verifying for the Australian road environment

10. Cybersecurity

11. Education and training.

Government’s Response to Ethical Dilemmas in Autonomous Vehicle Accidents: An Australian Policy Evaluation

1161