Needs and Challenges Concerning Cyber-risk Assessment in the

Cyber-physical Smart Grid

Gencer Erdogan

1 a

, Inger Anne Tøndel

2 b

, Shukun Tokas

1 c

, Michele Garau

3 d

and Martin Gilje Jaatun

2 e

Sustainable Communication Technologies, SINTEF Digital, Oslo, Norway

Software Engineering, Safety and Security, SINTEF Digital, Trondheim, Norway

Energy Systems, SINTEF Energy, Trondheim, Norway

Keywords:

Cyber-risk, Cybersecurity, Cyber-physical, Smart Grid, IoT, Needs, Challenges, Success Criteria.

Abstract:

Cyber-risk assessment methods are used by energy companies to manage security risks in smart grids. How-

ever, current standards, methods and tools do not adequately provide the support needed in practice and the

industry is struggling to adopt and carry out cyber-risk assessments. The contribution of this paper is twofold.

First, we interview six companies from the energy sector to better understand their needs and challenges.

Based on the interviews, we identify seven success criteria cyber-risk assessment methods for the energy sec-

tor need to fulﬁll to provide adequate support. Second, we present the methods CORAS, VAF, TM-STRIDE,

and DA-SAN and evaluate the extent to which they fulﬁll the identiﬁed success criteria. Based on the evalu-

ation, we provide lessons learned in terms of gaps that need to be addressed in general to improve cyber-risk

assessment in the context of smart grids. Our results indicate the need for the following improvements: 1) ease

of use and comprehensible methods, 2) support to determine whether a method is a good match for a given

context, 3) adequate preparation to conduct cyber-risk assessment, 4) manage complexity, 5) adequate support

for risk estimation, 6) support for trustworthiness and uncertainty handling, and 7) support for maintaining

risk assessments.

1 INTRODUCTION

The ongoing digitalization of the electric power grid

is resulting in complex cyber-physical smart grid sys-

tems. In such systems, it is no longer possible to sepa-

rate the digital part of the system from the more tradi-

tional power part of the system, as these technologies

are becoming deeply integrated. From a cybersecurity

perspective, this tight integration exposes the power

grid to many cyber-risks introduced by digitalization,

e.g., via IoT systems that are increasingly used in the

context of smart power grids (Tøndel et al., 2018).

Cyber-risk assessment is the de facto approach

used by large organizations to manage cybersecurity

risks, but current standards, methods and tools do

not adequately provide the support needed in prac-

https://orcid.org/0000-0001-9407-5748

https://orcid.org/0000-0001-7599-0342

https://orcid.org/0000-0001-9893-6613

https://orcid.org/0000-0002-9803-9944

https://orcid.org/0000-0001-7127-6694

tice for smart grid systems. For example, on the one

hand we have widely used cyber-risk assessment ap-

proaches such as ISO 27005 (ISO/IEC 27005:2018,

2018) and NIST 800-39 (NIST 800-39:2011, 2011),

while on the other hand we have risk assessment ap-

proaches that are speciﬁc for power systems (Jakob-

sen et al., 2021; Li, 2014). Although risk assessment

approaches from the cybersecurity and the power do-

mains share some overall characteristics, the indus-

try is struggling to adopt and carry out risk assess-

ments considering cyber-risks, and has limited knowl-

edge on how to best use existing approaches to carry

out a holistic cyber-risk assessment considering the

merged cyber-physical aspect of the future power grid

systems. Moreover, there is a lack of knowledge for

combining an assessment of speciﬁc types of threats

(e.g., cyber) with a more overarching assessment to

obtain a more concrete picture of the overall risk.

This paper explores the industry’s challenges and

needs for carrying out cyber-risk assessment in com-

plex and integrated cyber-physical systems, with fo-

cus on smart grids. Moreover, it explores strategies

Erdogan, G., Tøndel, I., Tokas, S., Garau, M. and Jaatun, M.

Needs and Challenges Concerning Cyber-risk Assessment in the Cyber-physical Smart Grid.

DOI: 10.5220/0011137100003266

In Proceedings of the 17th International Conference on Software Technologies (ICSOFT 2022), pages 21-32

ISBN: 978-989-758-588-3; ISSN: 2184-2833

for moving towards more integrated risk assessment

that includes both cybersecurity and power system

threats, as well as ICT dependability issues. Thus,

the contribution of this paper is twofold. First, we

carry out interviews with representatives from the in-

dustry to better understand the current and envisioned

needs when it comes to cyber-risk assessment related

to smart grids. These interviews lead to the identiﬁca-

tion of success criteria for risk assessment methods in

the context of smart grids. Second, we describe four

different methods for risk assessment we have used

in previous work to assess cyber-risks in smart grids.

For each of these methods, we provide a description

and evaluate the extent to which they meet the success

criteria identiﬁed from the interviews. Based on the

evaluation, we map the four methods to a qualitative

scale representing the level of fulﬁllment of criteria.

We also provide lessons learned in terms of identiﬁed

gaps that need to be addressed to improve cyber-risk

assessment in the context of smart grids.

The rest of the paper is organized as follows. Sec-

tion 2 describes the background and related work.

Section 3 describes our research method. Section 4

describes the ﬁndings from the interviews and the

identiﬁed success criteria. Section 5 describes the

four risk assessment methods used in previous work,

while Section 6 evaluates the extent to which the

methods fulﬁll the identiﬁed success criteria. Fi-

nally, Section 7 concludes the paper and summarizes

lessons learned in terms of identiﬁed gaps.

2 BACKGROUND AND RELATED

WORK

According to ISO 27005, "a risk is a combina-

tion of the consequences that would follow from

the occurrence of an unwanted event and the like-

lihood of the occurrence of the event" (ISO/IEC

27005:2018, 2018). As indicated in Section 1, there

are many standards and specialized approaches for

cyber-risk assessment. The most widely used stan-

dards are developed by ISO and NIST. The litera-

ture offers a wide variety of modelling techniques for

risk identiﬁcation and assessment. Fault tree analy-

sis (FTA) (IEC 61025:2006, 2006), event tree anal-

ysis (ETA) (IEC, 1995) and attack trees (Schneier,

1999) are examples of tree-based approaches and

provide support for reasoning about the sources

and consequences of unwanted incidents, as well

as their likelihoods. Cause-consequence analysis

(CCA) (Nielsen, 1971) and Bayesian networks (Ben-

Gal, 2008) are examples of graph-based notations.

Cause-consequence analysis employs diagrams that

combine the features of both fault trees and event

trees, whereas the latter two serves as mathemati-

cal models for probabilistic and statistical calcula-

tions, respectively. Moreover, whereas alternative ap-

proaches such as CRAMM (Barber and Davey, 1992)

and OCTAVE (Alberts et al., 2003) rely on text and

tables, graph and tree-based approaches use diagrams

as an important means for communication, evalua-

tion, and assessment.

In the context of smart grids, risk assessment is

the process of identifying, estimating and prioritizing

risks to the grid’s operations and assets. The afore-

mentioned steps are part of the standard risk assess-

ment processes (ISO/IEC 27005:2018, 2018; NIST

800-30, 2012). The technological trends underlying

the smart grid suggests a broad spectrum of the ICT

being deployed for more effective grid operations.

This integrated digital-power grid shift also brings

growing attack risks to the smart grid. The energy in-

dustry faces signiﬁcant challenges in managing such

risks. When analyzing risks in today’s power sys-

tems, traditional risk assessment methods should be

integrated with an assessment of cyber-physical inter-

dependencies, in order to highlight potential vulnera-

bilities that can represent a source of hazard. While

traditional risk assessment focuses on hazards with

relatively high probability that come from inherent

properties of the system (e.g., component aging), vul-

nerability assessment can be seen as a method that

aims at identifying hidden vulnerabilities in infras-

tructure systems that can bring to disruptive events,

such as blackouts, economic or social turmoil, etc.

(Kröger et al., 2011). These high-impact and low-

probability events can be too complex to be described

with traditional risk-assessment approaches. Typical

examples of cases where risk-based approaches may

be insufﬁcient for a proper analysis of hidden vulner-

abilities are the cases of emergent behaviors, intricate

rules of interaction, system of systems, broad spec-

trum of hazard and threats (Kröger et al., 2011). A

framework for studying vulnerabilities and risk in the

electricity supply, based on the bow-tie model, is pro-

posed in (Hofmann et al., 2012; Kjølle et al., 2012;

Hofmann et al., 2015). Among the vulnerabilities,

a speciﬁc case is represented by cyber-security risks,

which can be deﬁned as the potential of loss as a re-

sult of cyberattack resulting from the operations of an

information system.

A fundamental work on risks related to the dig-

italization process in power systems has been pro-

posed by the Task Force on Reliability Considera-

tion for Emerging Cyber-Physical Energy Systems

(Aravinthan et al., 2018). The authors emphasize

the necessity of modernizing the reliability and risk

ICSOFT 2022 - 17th International Conference on Software Technologies

assessment methods traditionally adopted in power

systems. A multi-layer modelling approach is sug-

gested, where the power layer, communication and

coupling layer and decision layer interact in order to

enable the power system operation. Each of these

layers are characterized by vulnerabilities that should

be singularly addressed. Conventional risk assess-

ment techniques are primarily focused on the power

layer, and can be primarily classiﬁed into two cate-

gories: analytical methods and simulation methods

(e.g., Monte Carlo simulation) (Billinton and Allan,

1996). In order to include in the power system risk as-

sessment possible failure states in the ICT infrastruc-

tures, novel approaches have been introduced, which

adopt complex network theory (Zhu et al., 2018),

cyber-physical interface matrix (Lei et al., 2015), co-

simulation (Garau et al., 2015), and traditional event

trees (Liu et al., 2019) and reliability block diagrams

(Ding et al., 2018). These works adopt approaches

that are strongly related with the concept of probabil-

ity of failure occurrence, therefore they ﬁnd a difﬁcult

application to scenarios where the threat is deliberate

and there are few statistics available to be included

in probabilistic approaches. As a consequence, in

order to model the effect of successful exploitation

of vulnerabilities, risk modelling is performed us-

ing high-level conceptual models, such as ISO/IEC

Common Criteria standard (Aravinthan et al., 2018),

stochastic Petri net models (Ten et al., 2008), Markov

processes (Zhang et al., 2016) and Bayesian attack

graphs (Zhang et al., 2015).

3 RESEARCH METHOD

Figure 1 illustrates our research method, which con-

sists of six steps. In Step 1, we conducted four inter-

views with four companies in the energy sector and

two interviews with two sectorial organizations. The

two sectorial organizations are the Computer Emer-

gency Response Team for the electric power sector

(KraftCERT) and the Norwegian Water Resources

and Energy Directorate (NVE). The energy compa-

nies are not named due to conﬁdentiality. Thus, we

carried out in total six interviews. Table 1 lists the in-

terviews we carried out, including date, duration, par-

ticipants, and the type of company/organization inter-

viewed. The interview team consisted of two partici-

pants; one taking the role as interviewer and one tak-

ing the role as secretary. The interviews were semi-

structured, covering the following topics:

• Current practice in cybersecurity and risk man-

agement in the energy sector.

• Risk management and cybersecurity approaches

that work well based on the interviewee’s experi-

ence.

• Needs and challenges within risk management

and cybersecurity in the energy sector.

The main task of the secretary was to note the

questions asked by the interviewer, as well as the an-

swers provided by the interviewee. However, we did

allow for the secretary to also come with questions

sporadically, in which case the interviewer would take

notes. In addition to the time spent on conducting the

interviews, the interviewer and the secretary spent ap-

proximately 1 hour after each interview to tidy up the

transcribed interview draft.

All interviewed companies/organizations are Nor-

wegian. We recruited the interviewees by asking them

directly through our own network, but also asking

companies and organizations from the Centre for In-

telligent Electricity Distribution project (CINELDI,

2022), which is the project in which this work was

carried out. The interviewees were people with differ-

ent roles, including Chief Information Security Ofﬁ-

cer (CISO), Cybersecurity expert, and Senior Project

Manager.

The output of Step 1 was a set of interview notes.

The interview notes were used as input to Step 2, in

which the interview team coded the collected data us-

ing the MAXQDA tool. The coding was mainly in-

ductive, but with some high level organizing codes to

structure the material (current practice; works well;

challenges; needs). In Step 3, the interview team went

through all the codes and highlighted the notes that

indicated a need or a challenge the energy sector was

experiencing with respect to risk assessment. For this,

we used memos in MAXQDA that were linked to the

coded segments.

In Step 4, we identiﬁed a set of success crite-

ria based on the needs and challenges indicated by

the interviews. The success criteria represent crite-

ria for risk assessment approaches to successfully as-

sess cyber risks in (the future) cyber-physical smart

grids (according to the needs indicated by the inter-

viewees). In Step 5, we described four risk assess-

ment approaches we have used in industrial cases

within the energy sector to assess risks in smart grids.

The approaches we describe are CORAS, the Vulner-

ability Analysis Framework, Threat Modeling with

STRIDE, and Stochastic Activity Network. These

approaches were selected because of two main rea-

sons: 1) the authors have years of experience in ap-

plying these methods in the energy sector as well as

other industrial context, and 2) these approaches sup-

port risk assessment from different yet complemen-

tary perspectives, and we wanted to assess the feasi-

bility of the approaches with respect to the identiﬁed

Needs and Challenges Concerning Cyber-risk Assessment in the Cyber-physical Smart Grid

^ƚĞƉϭ͗ŽŶĚƵĐƚ

ŝŶƚĞƌǀŝĞǁƐǁŝƚŚ

ŽƌŐĂŶŝǌĂƚŝŽŶƐĨƌŽŵƚŚĞ

ĞŶĞƌŐǇƐĞĐƚŽƌ

^ƚĞƉϮ͗ŽĚŝŶŐŽĨ

ŝŶƚĞƌǀŝĞǁŶŽƚĞƐ

^ƚĞƉϯ͗ŶĂůǇƐŝƐŽĨ

ĐŽĚĞĚŝŶƚĞƌǀŝĞǁŶŽƚĞƐ

^ƚĞƉϰ͗/ĚĞŶƚŝĨǇƐƵĐĐĞƐƐ

ĐƌŝƚĞƌŝĂďĂƐĞĚŽŶŶĞĞĚƐ

ĂŶĚĐŚĂůůĞŶŐĞƐ

^ƚĞƉϱ͗ĞƐĐƌŝďĞƌŝƐŬ

ĂƐƐĞƐƐŵĞŶƚŵĞƚŚŽĚƐ

ĨŽƌƚŚĞƐŵĂƌƚŐƌŝĚ

^ƚĞƉϲ͗ǀĂůƵĂƚĞƌŝƐŬ

ĂƐƐĞƐƐŵĞŶƚŵĞƚŚŽĚƐ

ǁ͘ƌ͘ƚ͘ƐƵĐĐĞƐƐĐƌŝƚĞƌŝĂ

^ƵĐĐĞƐƐĐƌŝƚĞƌŝĂ

ZŝƐŬĂƐƐĞƐƐŵĞŶƚ

ŵĞƚŚŽĚƐ

&ĞĂƐŝďŝůŝƚǇŽĨŵĞƚŚŽĚƐ

ĂŶĚůĞƐƐŽŶƐůĞĂƌŶƚ

ŽĚĞĚŝŶƚĞƌǀŝĞǁ

ŶŽƚĞƐ

/ŶƚĞƌǀŝĞǁŶŽƚĞƐ

/ĚĞŶƚŝĨŝĞĚŶĞĞĚƐ

ĂŶĚĐŚĂůůĞŶŐĞƐ

Figure 1: Research method.

success criteria.

Finally, in Step 6, we evaluate the four risk assess-

ment approaches with respect to the identiﬁed suc-

cess criteria; we discuss the extent to which the risk

assessment approaches fulﬁll the success criteria and

the gaps that need to be addressed. This evaluation

also acts as a basis for lessons learned, summarized

in Section 7.

4 IDENTIFIED SUCCESS

CRITERIA

This section describes the success criteria identiﬁed

based on the interviews, as explained in Section 3. In

total, we identiﬁed seven success criteria (SC) for risk

assessment approaches, addressing needs and chal-

lenges in the industry pointed out by the interviewees.

In the following, we present each success criterion

and describe their motivation based on the interviews.

SC1 Be easy to comprehend and use by people who

are not experts in risk assessment. Interviewees

state that it is essential that risk assessments are easy

to do also by people who are not experts in cybersecu-

rity and risk assessment. Several interviewees express

that quantitative methods are not currently an option

for them, and that there is a need to start with very

easy methods. One interviewee even states that it is

more important that a method is easy to use than the

quality of the results of the analysis, because if the

method is too complex and requires too much effort

it will meet resistance and the risk assessment will

probably not be carried out. Currently, many of the

companies seem to opt for using the same methods for

cyber risk as for other risk. In the companies, there is

a limited number of people that have the competence

to do risk assessments related to cyber risk, and infor-

mation security experts become a bottleneck if they

have to be involved in all such assessments. Thus,

there is a push towards system owners taking on more

responsibility for assessing risk, and at least one of the

companies are training project managers in perform-

ing risk assessments that include information security.

Note also that we talked with relatively large compa-

nies within this sector. However, one interviewee ex-

plains that more than half of the Distribution System

Operators (DSO) are small companies with less than

50 employees. And such companies are unlikely to

have dedicated in-house cybersecurity experts. If the

risk analyst does not have the necessary competence,

support, or training, interviewees explain that one risk

is that the analyst just ticks that a risk assessment has

been performed without the risks being properly as-

sessed.

SC2 Provide support to determine whether the

method is a good match for a given context. There

is a large variety in current practice and current abil-

ity to perform cybersecurity risk assessments among

the companies in the energy sector. A method that is

suitable for a larger company with dedicated informa-

tion security experts may not be suitable for a smaller

company without such experts. Based on the inter-

views, it seems that especially for those with limited

competence, it is difﬁcult to know how to start ana-

lyzing cyber risk and what questions to consider in

the assessment. Further, there are different types of

risk assessments that are performed in the companies,

ranging from yearly risk assessments to smaller as-

sessments as part of procurement or changes. There

is a clearly stated need to start with simple assessment

approaches, but at the same time the complexity of the

target of analysis may point to a need to move towards

more complex assessment approaches in some cases,

including when companies have become more mature

in their approach to cybersecurity risk assessments.

ICSOFT 2022 - 17th International Conference on Software Technologies

Table 1: Interviews conducted. CISO = Chief Information Security Ofﬁcer. PM = Project Manager.

No. Date Duration Interview team interviewee Organization

1 28.09.2021 1 hour

1 Interviewer

1 Secretary

1 Cybersecurity Expert KraftCERT

2 15.10.2021 1 hour

1 Interviewer

1 Secretary

1 CISO Energy company

3 19.10.2021 1 hour

1 Interviewer

1 Secretary

1 CISO Energy company

4 04.11.2021 1 hour

1 Interviewer

1 Secretary

1 CISO

1 Senior PM

Energy company

5 05.11.2021 1 hour

1 Interviewer

1 Secretary

1 CISO Energy company

6 22.11.2021 1 hour

1 Interviewer

1 Secretary

1 Cybersecurity Expert NVE

SC3 Support preparation for risk assessment, in-

cluding establishing a common understanding of

concepts and build necessary knowledge of partic-

ipants from IT and OT. When cybersecurity is con-

sidered in the more traditional risk assessments, it is

experienced as being abstract. Interviewees tell of

experiences where cybersecurity is represented with

only one scenario in combination with other types of

threats, e.g., technical failures, extreme weather con-

ditions. In many of the companies there seem to be

a division between IT and OT, though some explain

that understanding across IT and OT has improved,

e.g., through participating in workshops. One of the

interviewees explains that there commonly is a lack

of training of people that become involved in a risk

assessment. One example pointed out is that individ-

uals from OT are involved (which is encouraged) in

risk assessments without any prior understanding of

cyber risk and the risk assessment process, thus lead-

ing to misunderstandings and challenges during as-

sessment. IT and OT people may, e.g., disagree on

the interpretation of key concepts such as likelihood

and consequence and have a different understanding

of criticality. In the sector, there is some support ma-

terial available from sectorial organizations. How-

ever, there is a need for more support – concrete ex-

amples and lists of scenarios are highlighted in the

interviews – to motivate for risk assessments, help un-

derstand what may happen, and improve quality. It is

difﬁcult to contribute meaningfully to a risk assess-

ment without some basic understanding of a potential

attack, what techniques can be used, and how such

attacks can be mitigated. Furthermore, though peo-

ple from OT are experts in their domain they might

not have the knowledge needed to evaluate cyber risk,

e.g., know the architecture of the OT systems.

SC4 Manage complexity in the risk assessment,

considering the target of analysis. The analysis

target is complex, and the complexity is increasing,

which makes it difﬁcult to do good risk assessments.

There are several reasons for these challenges. There

are ongoing changes in work processes and in systems

and their use, and some of these changes happen grad-

ually. Often, manual systems are seen as backups, but

eventually the organization looses experience in us-

ing these manual backup systems, and thus they lose

much of their value. This gradual change can be dif-

ﬁcult to capture in risk assessments. For example, if

an assessment uses a previous analysis as a starting

point, it is easy to become inﬂuenced of the previous

conclusions and not see what has changed and the as-

sumptions that may no longer be valid. Furthermore,

there are connections and dependencies between sys-

tems that may be difﬁcult to capture in an assessment.

Interviewees provide examples that though OT sys-

tems are clearly mission-critical, other systems like

Advanced Metering Infrastructure (AMI) may also be

critical as they are necessary for other key functions,

such as being able to bill customers. However, these

other systems may not get enough attention. It is

challenging to understand how one risk affects other

risks. Assessments are often done for single systems

or for single types of incidents, but it is challenging to

understand any relations between these and combine

analysis results to get a more holistic view of the risk.

SC5 Support risk estimation, e.g., likelihood and

consequence estimation, as well as ranking of as-

sets considered in the risk assessment. There is a

need to know what are the most critical assets and

work processes to protect. Risk estimation is often

done through estimation of the likelihood and con-

sequence of certain incidents. However, the criteria

that are used to estimate likelihood and consequence

in assessments of other types of risk may not be rele-

vant when assessing cyber risk. Moreover, intervie-

wees tell that disagreements between different pro-

fessions often happen related to likelihood and con-

sequence estimation. When it comes to consequence,

the main challenges are related to estimation of in-

direct consequences (e.g., reputation). One intervie-

wee points to security economy as important mov-

ing forward, to make the economic costs of secu-

Needs and Challenges Concerning Cyber-risk Assessment in the Cyber-physical Smart Grid

rity incidents clearer to the decision makers. When

it comes to likelihood estimation, this is considered

particularly difﬁcult as one is dealing with malicious

threats. Several interviewees consider replacing like-

lihood estimates with evaluations of threat actors and

their capacity and intention, and the vulnerabilities

present. However, changing the method into some-

thing that is different from what is used for assess-

ments of other types of risks in the company is not

without challenges. For example, this makes it dif-

ﬁcult to aggregate results from different analysis to

support decision-making. Furthermore, interviewees

explain that there is not enough data to use for esti-

mating likelihood, and point to the risk of underes-

timating the likelihood for things that have not yet

happened. One interviewee explains that support for

reuse of likelihood estimates would be highly useful.

Support for reuse would reduce the need to involve

key experts in every analysis. Many of the assess-

ments are of objects that have similar characteristics.

Moreover, many aspects about the threats are similar

for other companies of the same type.

SC6 Provide support for increasing trustworthi-

ness of the risk assessment results, as well as man-

age and represent uncertainty. Criticism of current

risk assessments is that they are subjective and that

they are not able to identify all important issues to

consider, to improve cybersecurity. Due to challenges

related to risk estimation (SC5), a few interviewees

point to the need to consider uncertainty in the risk es-

timates. Trustworthiness in risk estimation is impor-

tant, to be conﬁdent in what to report to management,

and in providing arguments for how security invest-

ments are important for the business. Several of the

interviewees move towards more pentesting and sys-

tem monitoring, as these are considered more effec-

tive than risk assessments in identifying vulnerabili-

ties. Thus, this brings up possibilities for combining

risk assessments with pentesting and monitoring, in

ways that increase trustworthiness in assessment and

effectiveness in testing and monitoring. Some inter-

viewees envision a future with more real time risk as-

sessments, and wish for more tool support that can

help them in the risk assessments and that are able

to bring in data as support, e.g., to identify relevant

threat scenarios.

SC7 Facilitate risk management through docu-

mentation, maintenance of assessments, and ex-

pression of risk treatments. As pointed out by one

interviewee, risk assessment does not necessarily im-

ply risk management. Though an analysis identify

many risks, it may not be straight forward to know

what to do about these risks. Another interviewee

points out that the more traditional way of thinking

within this sector, that everything should be secure,

may not work moving forward, and that there will

be a need to build resilience into the system so that

they can tolerate some cyber-incidents taking place.

Regarding documentation, one interviewee explained

about a lack of culture for documenting risk analy-

sis. Moreover, interviewees point to the importance

of having updated risk assessments. However, it is

challenging to ensure such updates are made when-

ever there are changes made in the systems. Further-

more, with increasing number of systems, scalability

of the assessment approach is also an issue, especially

if information security experts need to be involved or

even responsible for such assessments. Another chal-

lenge is communicating the results of the risk assess-

ment in a way that is comprehensible to management

and that puts the cyber-risk topic on their agenda. On

the positive side, one interviewee tells about regular

reporting of cyber risk to the board, and another tells

about using threat modeling in the management group

at a high level, to discuss why attacks are possible and

what can be done. On the other hand, one interviewee

points to the risk assessment as difﬁcult to communi-

cate to the management.

5 RISK ASSESSMENT METHODS

This section describes the four risk assessment ap-

proaches we have used in industrial cases within the

energy sector: CORAS, the Vulnerability Analysis

Framework (VAF), Threat Modeling with STRIDE

(TM-STRIDE), and Dependability analysis with

Stochastic Activity Network (SAN). Because of space

restrictions, we provide a brief description of each ap-

proach and refer to other sources for further details.

5.1 CORAS

CORAS is a method for conducting security risk as-

sessment (Lund et al., 2011). In the CORAS method,

a security risk assessment is conducted in eight steps:

1) preparations for the analysis, 2) customer presenta-

tion of the target, 3) reﬁning the target description us-

ing asset diagrams, 4) approval of the target descrip-

tion, 5) risk identiﬁcation using threat diagrams, 6)

risk estimation using threat diagrams, 7) risk evalua-

tion using risk diagrams, and 8) risk treatment using

treatment diagrams.

CORAS provides a customized language for

threat and risk modelling, and comes with detailed

guidelines explaining how the language should be

used to capture and model relevant information dur-

ing the various steps of security risk assessment. The

ICSOFT 2022 - 17th International Conference on Software Technologies

CORAS method provides a web-based tool (CORAS

Tool, 2022) designed to support documenting, main-

taining and reporting assessment results through risk

modelling.

CORAS is a general approach to cybersecurity

risk assessment and has been applied to a large va-

riety of risk assessment targets and concerns within

numerous domains, including security, safety, law,

civil protection, emergency planning, defense, health,

and energy (Lund et al., 2011; Omerovic et al., 2019;

Omerovic et al., 2020).

5.2 The Vulnerability Analysis

Framework (VAF)

The Vulnerability Analysis Framework (VAF)

(Gjerde et al., 2011; Hofmann et al., 2015) is an

analysis approach aimed at identifying and analyzing

extraordinary events that can impact power system

security. The key concepts in VAF are susceptibility

(i.e., the extent to which a system is susceptible to

a threat), and coping capacity (i.e., the extent to

which a system is able to cope with the negative

consequences of a potential threat). These are

concepts used in bow-tie diagrams, and VAF can

utilize bow-tie diagrams for some of its six analysis

steps: 1) identify critical consequences, 2) identify

outages leading to critical consequences, 3) identify

threats that can cause the critical outages, 4) identify

vulnerabilities, susceptibility and coping capacity,

5) identify factors inﬂuencing coping capacity, and

6) vulnerability evaluation, identify existing and

missing barriers against critical outages.

The VAF has been used for analysis focusing on

the more traditional threats experienced in power sys-

tems, such as meteorological events and technical

failures. However, it has also been successfully used

for analysis of a cyber-physical power system where

cyber threats were included in the analysis (Tøndel

et al., 2021). This resulted in the recommendation that

interdependencies were identiﬁed and documented

from Step 3 and onwards, e.g., using the interdepen-

dence types identiﬁed by Rinaldi et al. (Rinaldi et al.,

2001); physical, cyber, geographical, and logical.

5.3 Threat Modeling with STRIDE

(TM-STRIDE)

Threat modeling is a process that reviews the secu-

rity of any connected system, identiﬁes problem ar-

eas, and determines the risk associated with each area.

We refer to the result as a threat model, even though it

might not necessarily satisfy the formal requirements

of a "model". Incidentally, threat modelling is part of

what McGraw refers to as Architectural Risk Analy-

sis (McGraw, 2006).

The STRIDE mnemonic (Spooﬁng, Tampering,

Repudiation, Information disclosure, Denial of ser-

vice, Elevation of privilege) was introduced by Mi-

crosoft, and gained prominence through Swidersky &

Snyder’s (Swiderski and Snyder, 2004) book on threat

modeling and Howard & Lipner’s (Howard and Lip-

ner, 2006) book on the Microsoft Security Develop-

ment Lifecycle. A later book by Shostack (Shostack,

2014) also covers a number of compatible soft-

ware tools, including the Microsoft Threat Model-

ing tool (Bygdås et al., 2021), which conforms to the

methodology presented by Swidersky & Snyder.

The ﬁrst step in this threat modeling approach is

to draw a data ﬂow diagram (DeMarco, 1979) which

helps to understand the system’s attack surface by

providing an overview of entities, processes and data

stores, identifying trust boundaries and sketching how

data ﬂows in the system. The resulting threat model

is thus a visual representation of four main elements:

the assets within a system, the system’s attack surface,

a description of how the components and assets inter-

act, and threat actors who could attack the system and

how the attack could occur.

5.4 Dependability Analysis with SAN

(DA-SAN)

A novel approach for dependability analysis of power

systems is proposed by Zerihun, Garau, and Helvik

(Zerihun et al., 2020) based on Stochastic Activity

Network (SAN) modelling. SAN is a variant of Petri

Nets (Sanders and Meyer, 2000) and provides a ﬂexi-

ble formalism which is particularly suitable for com-

plex interacting entities, through the input and output

ports that allow representing interaction with simple

conditional statements. The approach provides an ef-

ﬁcient method to analyze the impact of ICT vulnera-

bilities on power system operation.

Major events such as failure and repair within

power system and ICT systems are modelled along

with the ICT infrastructure management (MANO sys-

tem, VM redundancy, etc.) with the SAN formal-

ism. The power ﬂow and power system operation cal-

culations are performed with numerical solvers, in-

cluded in the SAN model with external C++ libraries

purposely developed. The tool implemented exploits

and enhances the inherent advantages of the SAN for-

malism: efﬁcient computation simulation, structured

modelling, and modularity and ﬂexibility.

In (Zerihun et al., 2020), the SAN method is eval-

uated on a test distribution network, where the impact

of ICT internal and external vulnerabilities on the per-

Needs and Challenges Concerning Cyber-risk Assessment in the Cyber-physical Smart Grid

formances of a state estimation calculation is quanti-

tatively analysed. Among internal vulnerabilities, ra-

dio link failures, server failures, measuring devices,

etc. have been considered. Among external vulnera-

bilities, the impact of signal fading due to rain precip-

itation has been inspected.

6 EVALUATION OF RISK

ASSESSMENT METHODS

Figure 2 illustrates a comparison of the risk assess-

ment methods CORAS, the Vulnerability Analysis

Framework (VAF), Threat Modeling with STRIDE

(TM-STRIDE), and Dependability Analysis with

SAN (DA-SAN) in a scale reﬂecting their fulﬁllment

of the success criteria described in Section 4. The

placement of each method in the scale in Figure 2 is

based on the authors’ expert knowledge and experi-

ence in using the methods as outlined in Section 5.

Companies within the energy sector need risk as-

sessment approaches that are easy to comprehend and

use (SC1). The methods VAF and CORAS have em-

pirically been shown to be easy to comprehend and

use by people with different backgrounds (Solhaug

and Stølen, 2013; Lewis and Smith, 2010). How-

ever, we believe VAF is slightly easier to comprehend

by personnel of the energy sector companies because

VAF uses concepts and constructs that are commonly

used in the energy sector. CORAS has also been

used in many industrial risk assessments for the en-

ergy sector (Omerovic et al., 2019; Omerovic et al.,

2020). Threat modelling using Data Flow Diagrams

is a widely used approach, and it is therefore reason-

able to argue that it is easy to use, in particular con-

sidering cyber risks. The approach DA-SAN needs

specialized expertise and may not be easy to use un-

less one has the speciﬁc competence and skills. Al-

though VAF, CORAS, and TM-STRIDE may be eas-

ier to comprehend and use compared to DA-SAN,

none of the methods fully meet the SC1 criterion.

Based on the interviews and our experiences, we ar-

gue that not many of the existing risk assessment ap-

proaches are easy to comprehend and use for non-

experts in the energy sector because most approaches

do not have domain-speciﬁc support for the energy

sector (see Section 2).

Considering the criterion SC1, and the fact that all

the identiﬁed criteria described in Section 4 points out

the need for some kind of support to more easily carry

out risk assessment, indicates that comprehensibility

and ease of use is the most important success crite-

rion. One way of addressing this challenge would be

to make the existing approaches more light-weight,

but this would come at the cost of expressiveness and

the methods’ ability to handle complexity. Thus, to

successfully achieve criterion SC1, it is necessary to

develop risk assessment methods that are easy for the

energy sector to use, as well as providing guidelines to

select from a variety of approaches that balances be-

tween ease of use and the need for assessing complex

scenarios. According to the interviews, such guide-

lines would pave the way for a faster uptake of cyber-

risk assessment knowledge in the energy sector.

With respect to support to determine whether the

method is a good match for a given context (SC2), all

the methods do provide general guidelines for the an-

alyst to understand the context in which the method

may be applied. However, these general guidelines

are meant for security experts and are not an ade-

quate support for non security experts in the energy

sector as they are struggling to answer questions like:

"how can I carry out a simple high-level risk assess-

ment even if I don’t have cyber-risk expertise?", "what

questions should I consider when assessing risks?",

"which method should I use if I have a complex tar-

get of analysis?", and so on. Thus, the energy sec-

tor needs guidelines to select appropriate risk assess-

ment methods considering the competence of those

who will carry out the assessment, as well as the ob-

jectives of the planned risk assessment. For example,

the VAF method may be used to identify and explore

the most critical unwanted incidents. These incidents

may be used as input to the CORAS method, which

may help identify the chain of events that may cause

the unwanted incidents, including exploited vulnera-

bilities. The threat scenarios and vulnerabilities iden-

tiﬁed using CORAS may in turn be used as input to

the TM-STRIDE method to analyze how the vulner-

abilities are exploited from a data-ﬂow perspective.

Finally, the DA-SAN method may be used to iden-

tify the consequences of the identiﬁed vulnerabilities

and unwanted incidents on a power-grid system using

simulation techniques.

Regarding SC3, among the methods we have con-

sidered, CORAS and TM-STRIDE have thorough

steps to prepare a risk assessment in terms of es-

tablishing the context and making sure that all in-

volved stakeholders have a common understanding of

the context, concepts, and objectives of the risk as-

sessment. The VAF method also has the necessary

steps to prepare an assessment, but is slightly easier

to use in the context of the energy sector because it

does not require any vocabulary speciﬁc to power sys-

tem security or cybersecurity. The DA-SAN method

has preparation steps in terms of modelling the tar-

get. Though it is important to obtain a common un-

derstanding of the context, concepts, and objectives,

ICSOFT 2022 - 17th International Conference on Software Technologies

^ϭ ^Ϯ ^ϯ ^ϰ ^ϱ ^ϲ ^ϳ

,ŝŐŚ

ĨƵůĨŝůŵĞŶƚ

ŽĨĐƌŝƚĞƌŝĂ

>Žǁ

ĨƵůĨŝůŵĞŶƚ

ŽĨĐƌŝƚĞƌŝĂ

s&

dDͲ^dZ/

KZ^

Ͳ^E

KZ^

dDͲ^dZ/

Ͳ^E

s&

KZ^

dDͲ

^dZ/

Ͳ^E

s&

KZ^Ͳ^E

dDͲ^dZ/

s&

Ͳ^E

KZ^

s&

dDͲ^dZ/

Ͳ^E

KZ^

s&

dDͲ^dZ/

Ͳ^E

KZ^

s&

dDͲ^dZ/

Figure 2: Comparison of methods with respect to fulﬁllment of the success criteria described in Section 4.

the energy sector needs support in terms of domain-

speciﬁc cyber-risk example scenarios as well as train-

ing material about cyber-risk assessment to properly

prepare participants of risk assessment and help con-

tribute meaningfully during an assessment. These as-

pects may be included as part of a method, for exam-

ple during the preparation of an assessment partici-

pants can be introduced to risk assessment with exam-

ple scenarios speciﬁc to the energy sector. However,

a proper educational support would be to train the rel-

evant people using facilities such as cyber ranges that

are capable of simulating cyber-attacks on energy in-

frastructure. Our previous work shows that cyber-risk

training using cyber ranges are effective for a variety

of domains such as energy distribution, railroad trans-

port, and education (university) (Erdogan et al., 2020;

Erdogan et al., 2021).

The infrastructure of energy systems are becom-

ing increasingly complex, and it is therefore neces-

sary to manage the complexity of the target of analy-

sis (SC4). The methods we consider in this paper have

mechanisms in place to address complexity. However,

while the methods DA-SAN and TM-STRIDE lack

the capability to express risks the target of analysis is

exposed to as part of the target models, the methods

VAF and CORAS lack the expressiveness to repre-

sent the target of analysis as part of the risk models.

Each aforementioned method have of course been de-

veloped for their speciﬁc purpose, but it is reasonable

to argue that a method capable of capturing both the

target of analysis and risks could be beneﬁcial when

assessing risks in the context of the energy sector be-

cause of its cyber-physical aspects. According to the

interviews, one aspect that is especially important to

consider in the context of complexity, is the ability to

maintain risk assessments over time. With the digi-

talization of the energy systems, changes (both from

a cyber perspective and from a physical perspective)

may happen frequently. Whenever an update is intro-

duced in the energy systems, then it is important to

consider this change in the risk assessment as well.

The CORAS method has explicit support to consider

a risk picture before a change is introduced, and after

a change is introduced in the target system.

The success criterion SC5 points out the need for

support for risk estimation and ranking of assets. The

methods VAF and TM-STRIDE do not provide sup-

port in estimating risks, but rather rely on external

methods to estimate risks. DA-SAN supports risk es-

timation in terms of quantiﬁcation of the consequence

of failure states in the Cyber Physical Power System

(CPPS), while CORAS mainly supports likelihood

estimation using the CORAS calculus. Ideally, ac-

cording to the interviews, a risk assessment method

should provide guidelines for both likelihood and

consequence estimation. One possible approach is to

combine different methods to fully achieve SC5. For

example, DA-SAN can support CORAS with con-

sequence estimates, while CORAS can support DA-

SAN with likelihood estimates. Another option is to

Needs and Challenges Concerning Cyber-risk Assessment in the Cyber-physical Smart Grid

develop method-independent support for risk estima-

tion for the energy sector to support risk estimation in

a broader set of methods.

For all risk assessments, it is important that the

assessments are trustworthy and that the uncertainty

of the results are considered as part of the assessment

(SC6). The methods CORAS and VAF actively in-

volve people with different backgrounds in the risk

assessment process to obtain information from rele-

vant experts, and thereby increase the trustworthiness

of the risk assessment. TM-STRIDE offers no direct

support in relation to trustworthiness and uncertainty

assessment. Among the four methods considered in

this paper, DA-SAN is the only method that pro-

vides mechanisms (and tools) to quantitatively assess

the uncertainty of the risk assessment to increase the

trustworthiness. DA-SAN does this as part of the sim-

ulation. Trustworthiness and uncertainty are in gen-

eral very important factors for decision support when

evaluating whether to invest in new security mecha-

nisms, either for physical security (security of supply)

or software security.

Regarding SC7, the methods CORAS and VAF

use the diagrams produced in the risk assessment as a

basis for documentation and communication with the

stakeholders. These methods also support the identi-

ﬁcation of risk treatments and may therefore help de-

cision makers to identify and select appropriate risk

treatments. CORAS also supports change manage-

ment of assessment results, as described above re-

lated to maintenance of risk assessments. The meth-

ods TM-STRIDE and DA-SAN mainly create and

use models to identify risks, but also to document

the ﬁndings. Maintenance of risk assessment re-

sults and treatment identiﬁcation are not supported by

TM-STRIDE and DA-SAN. One important challenge

none of the methods are able to support is continu-

ous updated risk assessments. Based on our experi-

ence, we believe this challenge is not supported by

current risk assessment methods for the energy sector

in general, but it is something that must eventually be

supported to cope with the tsunami of data produced

by the IoT devices that will be integrated in the en-

ergy systems. Dynamic and real-time risk assessment

must inevitably be addressed and properly supported,

but the current state of risk assessment in the energy

sector shows that basic needs and challenges (as de-

scribed in this section) must be addressed before the

dynamic/real-time aspects can be supported.

7 CONCLUSIONS AND LESSONS

LEARNED

The energy sector is struggling to adopt and carry out

risk assessments considering cyber risks in the con-

text of smart grids. In this paper, we have interviewed

representatives from the energy sector to better un-

derstand the current and envisioned needs and chal-

lenges of risk assessment methods for smart grids.

Based on the needs and challenges, we identify a set

of success criteria that should be fulﬁlled for the en-

ergy sector to successfully carry out cyber-risk assess-

ment. Then we evaluate the methods CORAS, TM-

STRIDE, VAF, and DA-SAN with respect to the iden-

tiﬁed success criteria. The methods CORAS, TM-

STRIDE, VAF, and DA-SAN are methods the authors

have used in previous work to carry out risk assess-

ment of energy systems and smart grids. Based on the

evaluation, we discuss the extent to which the afore-

mentioned methods fulﬁll the success criteria and dis-

cuss gaps that need to be addressed in general.

We interviewed four Norwegian energy com-

panies and two sectorial organizations. The two

sectorial organizations are the Computer Emer-

gency Response Team for the electric power sector

(KraftCERT) and the Norwegian Water Resources

and Energy Directorate (NVE). The energy compa-

nies are not named due to conﬁdentiality. Based on

the needs and challenges described by the intervie-

wees, we identiﬁed seven success criteria cyber-risk

assessment methods for the energy sector need to ful-

ﬁll. In short, these are related to: ease of use and

comprehensible methods (SC1), support to determine

whether a method is a good match for a given context

(SC2), adequate preparation to conduct cyber-risk as-

sessment (SC3), manage complexity (SC4), adequate

support for risk estimation (SC5), adequate support

for trustworthiness and uncertainty handling (SC6),

and support for documenting and maintaining risk as-

sessments and identifying appropriate risk treatments

(SC7). The reader is referred to Section 4 for a de-

tailed description of the success criteria.

Although the number of interviewees are only

seven (four CISOs, two cybersecurity experts, and

one senior project manager), all interviewees are

people with years of experience in cybersecurity

within the energy sector. Moreover, in the case of

KraftCERT and NVE, the interviewees are experts

who are aware of the general needs and challenges in

the industry and critical infrastructure sector in gen-

eral. It is therefore reasonable to argue that the above-

mentioned success criteria need to be addressed for

the critical infrastructure sectors in general.

The methods we evaluated in this paper (CORAS,

ICSOFT 2022 - 17th International Conference on Software Technologies

TM-STRIDE, VAF and DA-SAN) fulﬁll the above

success criteria to a certain extent, but none of the

methods fulﬁll all the success criteria. The reader is

referred to Section 6 for a detailed discussion about

the gaps that need to be addressed. In summary, we

conclude with the following lessons learned.

1. Considering the fact that all success criteria (SC1-

SC7) point to the need for some kind of support

to more easily carry out risk assessment, we see

that there is especially a need to improve the com-

prehensibility and ease of use of risk assessment

methods for the energy sector in general.

2. There is a need for support in helping risk analysts

in the energy sector, including people both from

IT and OT, in selecting the right risk assessment

method for the right context. There is also a need

for domain-speciﬁc training material and example

scenarios to help participants contribute meaning-

fully during an assessment (SC2 and SC3).

3. There is a need for improving comprehensibil-

ity and ease of use of methods, but on the other

hand, there is also a need for managing complex-

ity of risk assessments to consider complex target

of analyses. These may be two conﬂicting needs,

but they indicate that risk assessment methods for

the energy sector need to be easy to comprehend

and use, but also able to sufﬁciently consider a

complex target of analysis (SC4).

4. Risk assessment methods for the energy sector

need to support risk quantiﬁcation, trustworthi-

ness and uncertainty handling (SC5 and SC6).

5. The risk assessment needs to be easy to maintain,

and the risk assessment results need to better pro-

vide decision support (SC7).

ACKNOWLEDGEMENTS

This work has been conducted as part of the CINELDI

project (257626) funded by the Research Council of

Norway.

REFERENCES

Alberts, C., Dorofee, A., Stevens, J., and Woody, C. (2003).

Introduction to the octave approach. Technical report,

Carnegie-Mellon University.

Aravinthan, V., Balachandran, T., Ben-Idris, M., Fei, W.,

Heidari-Kapourchali, M., Hettiarachchige-Don, A.,

Jiang, J. N., Lei, H., Liu, C.-C., Mitra, J., Ni, M.,

Papic, M., Parvania, M., Sephary, M., Singh, C., Sri-

vastava, A., Stefanov, A., Sun, H., and Tindemans,

S. (2018). Reliability modeling considerations for

emerging cyber-physical power systems. In Proc.

2018 IEEE International Conference on Probabilis-

tic Methods Applied to Power Systems (PMAPS’18),

pages 1–7. IEEE.

Barber, B. and Davey, J. (1992). The use of the CCTA risk

analysis and management methodology CRAMM in

health information systems. Medinfo, 92:1589–1593.

Ben-Gal, I. (2008). Bayesian networks. Encyclopedia of

Statistics in Quality and Reliability, 1.

Billinton, R. and Allan, R. N. (1996). Reliability Evaluation

of Power Systems. Plenum Press, New York, 2 edition.

Bygdås, E., Jaatun, L. A., Antonsen, S. B., Ringen, A., and

Eiring, E. (2021). Evaluating threat modeling tools:

Microsoft TMT versus OWASP Threat Dragon. In

Proc. 2021 International Conference on Cyber Sit-

uational Awareness, Data Analytics and Assessment

(CyberSA’21), pages 1–7. IEEE.

CINELDI (2022). Centre for Intelligent

Electricity Distribution (CINELDI).

https://www.sintef.no/projectweb/cineldi/. Accessed

February 28, 2022.

CORAS Tool (2022). CORAS Risk Modelling Tool.

https://stverdal.github.io/. Accessed February 28,

2022.

DeMarco, T. (1979). Structure Analysis and System Speci-

ﬁcation, pages 255–288. Springer Berlin Heidelberg.

Ding, Z., Xiang, Y., and Wang, L. (2018). Incorporating

Unidentiﬁable Cyberattacks into Power System Reli-

ability Assessment. In 2018 IEEE Power Energy Soci-

ety General Meeting (PESGM’18), pages 1–5. IEEE.

Erdogan, G., Hugo, Å., Romero, A., Varano, D., Zazzeri,

N., and Žitnik, A. (2020). An approach to train and

evaluate the cybersecurity skills of participants in cy-

ber ranges based on cyber-risk models. In Proc. 15th

International Conference on Software Technologies

(ICSOFT’20), pages 509–520. SciTePress.

Erdogan, G., Romero, A., Zazzeri, N., Žitnik, A., Basile,

M., Aprile, G., Osório, M., Pani, C., and Kechaoglou,

I. (2021). Developing cyber-risk centric courses and

training material for cyber ranges: A systematic ap-

proach. In Proc. 7th International Conference on In-

formation Systems Security and Privacy (ICISSP’21),

pages 702–713. SciTePress.

Garau, M., Celli, G., Ghiani, E., Soma, G. G., Pilo, F.,

and Corti, S. (2015). ICT reliability modelling in co-

simulation of smart distribution networks. In Proc.

1st International Forum on Research and Technolo-

gies for Society and Industry Leveraging a better to-

morrow (RTSI’15), pages 365–370. IEEE.

Gjerde, O., Kjølle, G. H., Detlefsen, N. K., and Brønmo,

G. (2011). Risk and vulnerability analysis of power

systems including extraordinary events. In Proc. 2011

IEEE Trondheim PowerTech, pages 1–5. IEEE.

Hofmann, M., Kjølle, G., and Gjerde, O. (2012). De-

velopment of indicators to monitor vulnerabilities in

power systems. In Proc. 11th International Proba-

bilistic Safety Assessment and Management Confer-

ence (PSAM’11), pages 1–10. Curran Associates, Inc.

Needs and Challenges Concerning Cyber-risk Assessment in the Cyber-physical Smart Grid

Hofmann, M., Kjølle, G. H., and Gjerde, O. (2015). Vul-

nerability analysis related to extraordinary events in

power systems. In Proc. 2015 IEEE Eindhoven Pow-

erTech, pages 1–6. IEEE.

Howard, M. and Lipner, S. (2006). The Security Develop-

ment Lifecycle. Microsoft Press, Redmond, WA.

IEC (1995). Dependability management–part 3: Applica-

tion guide–section 9: Risk analysis of technological

systems.

IEC 61025:2006 (2006). IEC 61025:2006 Fault tree analy-

sis (FTA). International Standard, International Elec-

trotechnical Commission.

ISO/IEC 27005:2018 (2018). ISO/IEC 27005:2018 - Infor-

mation technology - Security techniques - Information

security risk management. Standard, International Or-

ganization for Standardization.

Jakobsen, S. H., Garau, M., and Mo, O. (2021). An open-

source tool for reliability analysis in radial distribu-

tion grids. In Proc. 2021 International Conference on

Smart Energy Systems and Technologies (SEST’21),

pages 1–6. IEEE.

Kjølle, G. H., Utne, I. B., and Gjerde, O. (2012). Risk anal-

ysis of critical infrastructures emphasizing electricity

supply and interdependencies. Reliability Engineer-

ing & System Safety, 105:80–89.

Kröger, W., Zio, E., and Schläpfer, M. (2011). Vulnerable

systems. Springer, London.

Lei, H., Singh, C., and Sprintson, A. (2015). Reliability

analysis of modern substations considering cyber link

failures. In Proc. 2015 IEEE Innovative Smart Grid

Technologies - Asia (ISGT’15), pages 1–5. IEEE.

Lewis, S. and Smith, K. (2010). Lessons learned from real

world application of the bow-tie method. In Proc.

6th Global Congress on Process Safety, pages 22–24.

OnePetro.

Li, W. (2014). Risk Assessment of Power Systems: Models,

Methods, and Applications. John Wiley & Sons.

Liu, Y., Deng, L., Gao, N., and Sun, X. (2019). A Reli-

ability Assessment Method of Cyber Physical Distri-

bution System. Energy Procedia, 158:2915–2921.

Lund, M., Solhaug, B., and Stølen, K. (2011). Model-

Driven Risk Analysis: The CORAS Approach.

Springer.

McGraw, G. (2006). Software Security: Building Security

In. Addison-Wesley.

Nielsen, D. S. (1971). The cause/consequence diagram

method as a basis for quantitative accident analysis.

Risø National Laboratory.

NIST 800-30 (2012). Special Publication 800-30 Guide for

Conducting Risk Assessments. Standard, National In-

stitute of Standards and Technology.

NIST 800-39:2011 (2011). NIST Special Publication 800-

39 - managing information security risk organization,

mission, and information system view. Standard,

NIST. Accessed: 2021-10-25.

Omerovic, A., Vefsnmo, H., Erdogan, G., Gjerde, O.,

Gramme, E., and Simonsen, S. (2019). A feasibil-

ity study of a method for identiﬁcation and modelling

of cybersecurity risks in the context of smart power

grid. In Proc. 4th International Conference on Com-

plexity, Future Information Systems and Risk (COM-

PLEXIS’19), pages 39–51. SciTePress.

Omerovic, A., Vefsnmo, H., Gjerde, O., Ravndal, S., and

Kvinnesland, A. (2020). An industrial trial of an ap-

proach to identiﬁcation and modelling of cybersecu-

rity risks in the context of digital secondary substa-

tions. In Proc. 14th International Conference on Risks

and Security of Internet and Systems (CRISiS’19),

pages 17–33. Springer.

Rinaldi, S. M., Peerenboom, J. P., and Kelly, T. K. (2001).

Identifying, understanding, and analyzing critical in-

frastructure interdependencies. IEEE Control Systems

Magazine, 21(6):11–25.

Sanders, W. H. and Meyer, J. F. (2000). Stochastic activity

networks: Formal deﬁnitions and concepts. In School

organized by the European Educational Forum, pages

315–343. Springer.

Schneier, B. (1999). Modeling security threats. Dr. Dobb’s

journal, 24(12).

Shostack, A. (2014). Threat modeling: Designing for secu-

rity. John Wiley & Sons.

Solhaug, B. and Stølen, K. (2013). The coras language-

why it is designed the way it is. In Proc. 11th Interna-

tional Conference on Structural Safety and Reliability

(ICOSSAR’13), pages 3155–3162. Citeseer.

Swiderski, F. and Snyder, W. (2004). Threat Modeling. Mi-

crosoft Press, Redmond, WA.

Ten, C.-W., Liu, C.-C., and Manimaran, G. (2008). Vul-

nerability Assessment of Cybersecurity for SCADA

Systems. IEEE Transactions on Power Systems,

23(4):1836–1846.

Tøndel, I. A., Foros, J., Kilskar, S. S., Hokstad, P., and

Jaatun, M. G. (2018). Interdependencies and relia-

bility in the combined ICT and power system: An

overview of current research. Applied computing and

informatics, 14(1):17–27.

Tøndel, I. A., Vefsnmo, H., Gjerde, O., Johannessen, F.,

and Frøystad, C. (2021). Hunting dependencies: Us-

ing bow-tie for combined analysis of power and cyber

security. In Proc. 2020 2nd International Conference

on Societal Automation (SA’20), pages 1–8.

Zerihun, T. A., Garau, M., and Helvik, B. E. (2020). Ef-

fect of Communication Failures on State Estimation

of 5G-Enabled Smart Grid. IEEE Access, 8:112642–

112658.

Zhang, Y., Wang, L., Xiang, Y., and Ten, C.-W. (2015).

Power System Reliability Evaluation With SCADA

Cybersecurity Considerations. IEEE Transactions on

Smart Grid, 6(4):1707–1721.

Zhang, Y., Wang, L., Xiang, Y., and Ten, C.-W. (2016). In-

clusion of SCADA Cyber Vulnerability in Power Sys-

tem Reliability Assessment Considering Optimal Re-

sources Allocation. IEEE Transactions on Power Sys-

tems, 31(6):4379–4394.

Zhu, W., Panteli, M., and Milanovi

c, J. V. (2018). Relia-

bility and Vulnerability Assessment of Interconnected

ICT and Power Networks Using Complex Network

Theory. In Proc. 2018 IEEE Power Energy Society

General Meeting (PESGM’18), pages 1–5. IEEE.

ICSOFT 2022 - 17th International Conference on Software Technologies