The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for

SME Cybersecurity Awareness

Costas Boletsis

1 a

, Sefat Noor Orni

and Ragnhild Halvorsrud

1 b

SINTEF Digital, Oslo, Norway

Department of Informatics, University of Oslo, Oslo, Norway

ﬁ

Keywords:

CJML, Cybersecurity, Modelling, User Journey, Visualisation.

Abstract:

Improving security posture while addressing human errors made by employees are among the most chal-

lenging tasks for SMEs concerning cybersecurity risk management. To facilitate these measures, a domain-

speciﬁc modelling tool for visualising cybersecurity-related user journeys, called the HORM Diagramming

Tool (HORM-DT), is introduced. By visualising SMEs’ cybersecurity practices, HORM-DT aims to raise

their cybersecurity awareness by highlighting the related gaps, thereby ultimately informing new or updated

cyber-risk strategies. HORM-DT’s target group consists of SMEs’ employees with various areas of technical

expertise and different backgrounds. The tool was developed as part of the Human and Organisational Risk

Modelling (HORM) framework, and the underlying formalism is based on the Customer Journey Modelling

Language (CJML) as extended by elements of the CORAS language to cover cybersecurity-related user jour-

neys. HORM-DT is a fork of the open-source Diagrams.net software, which was modiﬁed to facilitate the

creation of cybersecurity-related diagrams. To evaluate the tool, a usability study following a within-subject

design was conducted with 29 participants. HORM-DT achieved a satisfactory system usability scale score of

80.69, and no statistically signiﬁcant differences were found between participants with diverse diagramming

tool experience. The tool’s usability was also praised by participants, although there were negative comments

regarding its functionality of connecting elements with lines.

1 INTRODUCTION

Small and medium enterprises (SMEs) are increas-

ingly being exposed to cyber-risks. In 2017 alone,

61% of SMEs were exposed to malware cyberattacks

(Khan et al., 2020). Despite their increasing expo-

sure to cyberattacks, SMEs rarely conduct thorough

cyber-risk assessments, and more than half of them

provide either an outdated or no cyber-risk strategy at

all (Benz and Chatterjee, 2020; Paulsen, 2016; The

National Center for the Middle Market, 2016; Bo-

letsis et al., 2021). This may be due to internal is-

sues that arise when attempting to set up cyber-risk

strategies, such as ‘having small IT teams, inade-

quate security budgets, and disagreements between

IT and business leadership teams regarding cyberse-

curity risk management’ (Boletsis et al., 2021). It has

been reported that the most challenging tasks for cy-

bersecurity risk management in SMEs are i) deﬁning

https://orcid.org/0000-0003-2741-8127

https://orcid.org/0000-0002-3774-4287

and taking the ﬁrst step towards improving their secu-

rity posture and ii) addressing human errors made by

employees, which is the ‘human element’ sometimes

referred to as the biggest internal threat for SMEs

(Arctic Wolf, 2017; Meshkat et al., 2020; Symantec,

2019; Boletsis et al., 2021).

To address the ﬁrst challenge, it has been sug-

gested that mapping SMEs’ current practices and the

potential threats they face would be a useful ini-

tial move in cybersecurity risk management (Benz

and Chatterjee, 2020; Paulsen, 2016; Meszaros and

Buchalcevova, 2017). This mapping should also ad-

dress the second big challenge by modelling the hu-

man element in cybersecurity-related scenarios, es-

sentially through the documentation of user journeys

related to cybersecurity, which are the visual paths

that users may take when performing an action or us-

ing a service (Stickdorn et al., 2018; Boletsis et al.,

2021). At the same time, end-users/employees should

be involved in the modelling process, always taking

into consideration their diverse technical backgrounds

(Paulsen, 2016; Kullman et al., 2020; Bellamy et al.,

Boletsis, C., Orni, S. and Halvorsrud, R.

The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for SME Cybersecurity Awareness.

DOI: 10.5220/0011786600003417

In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 3: IVAPP, pages

203-213

ISBN: 978-989-758-634-7; ISSN: 2184-4321

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

203

2007; Boletsis et al., 2021).

This paper presents, the HORM Diagramming

Tool (HORM-DT)

, a domain-speciﬁc modelling tool

for visualising cybersecurity-related user journeys,

and evaluates its usability. The tool’s main target

group consists of SME employees with various areas

of technical expertise and different backgrounds, and

so usability is of the essence. By using the tool and

the models produced by it, the aim is to raise cyber-

security awareness so that SMEs can visualise their

practices and the potential cyber threats they face and

identify the gaps. In this way, they can develop up-to-

date or new cyber-risk strategies in an informed way.

The tool was developed as part of the Human and

Organizational Risk Modelling (HORM) framework

to be a comprehensible and easy-to-use cybersecurity-

related framework for capturing the risks that ordi-

nary people may be exposed to (Fair et al., 2022).

HORM is based on the Customer Journey Mod-

elling Language (CJML) (Halvorsrud et al., 2021) and

extended by formalism from the CORAS language

(Lund et al., 2011; Vraalsen et al., 2005) so that it

can address cybersecurity-related user journeys.

2 RELATED WORK

When conducting cybersecurity assessments, sim-

ply providing accurate risk information may not be

enough to ensure that individuals will be able to com-

prehend a risk message and act on it (Nurse et al.,

2011; Slovic, 1999; Skubisz et al., 2009; Boletsis

et al., 2021). However, visualisations can be an im-

portant factor in communicating cyber risks (Bolet-

sis et al., 2021). In the ﬁeld of SME cybersecurity

awareness, a limited number of works have evaluated

SMEs’ cybersecurity practices by utilising visual ele-

ments (Boletsis et al., 2021), including the following:

• CYSEC, a do-it-yourself cybersecurity assess-

ment method for SMEs that automates elements

of a counselling dialogue between a security ex-

pert and employees in the SME to counter cyber

threats (Shojaifar, 2019).

• The SME Cybersecurity Evaluation Tool, which

consists of a 35-question online survey to be com-

pleted by IT leaders to self-rate their maturity

based on the National Institute of Standards and

Technology cybersecurity framework (Benz and

Chatterjee, 2020).

HORM-DT can be accessed at: https://cjml.no/horm2/

and its code is available from the Github repository at:

https://github.com/CostasBoletsis/HORM-DT

HORM website: https://cjml.no/horm/

• The System Security Modeller (SSM), which is an

asset-based risk-analysis tool that provides an in-

formation security perspective to the interactions

between assets across an entire system and can be

also applied in SME settings (Boletsis et al., 2021;

Surridge et al., 2019).

• A gamiﬁed approach for raising awareness sur-

rounding SMEs’ level of cybersecurity and re-

silience (Ponsard and Grandclaudon, 2019), for

which SME employees answer a cybersecurity

quiz and a self-assessment questionnaire.

It can be said that these approaches for mapping

SMEs’ cybersecurity practices focus more on repre-

sentations of infrastructure than on human actors and

their behaviours. At the same time, ‘there is no holis-

tic visualisation approach that could facilitate infor-

mation distribution between employees of different

levels of expertise’ (Boletsis et al., 2021).

Each SME process essentially represents a path-

way through a sequence of events. The modelling

and visualisation of these processes could be covered

by user journey modelling languages (Boletsis et al.,

2021). To that end, the HORM framework was devel-

oped (Fair et al., 2022).

The HORM framework focuses on the ‘human el-

ement’ and its modelling, and it consists of a mod-

elling language, based on a version of the CJML

(Haugstveit et al., 2016; Halvorsrud et al., 2016a;

Halvorsrud et al., 2014; Halvorsrud et al., 2016b) ex-

tended through contributions from the CORAS lan-

guage to fulﬁl cybersecurity-related purposes, and a

set of tools for modelling.

CJML is a visual language for the modelling and

visualisation of service and work processes in terms

of customer or user journeys (Halvorsrud et al., 2021;

Boletsis et al., 2021). Being centred on humans and

their activities, CJML appeals to a broad user group

through its simple and intuitive form (Halvorsrud

et al., 2016a), a feature that is attributed to the user-

centred design methodology that lead its design. The

basic units of CJML are observable touchpoints that

can take the form of a communication event or a non-

communicative activity or action. In CJML, the se-

quence of touchpoints that a user follows to achieve

a speciﬁc goal constitutes a user journey. There are

two types of diagram available in CJML that serve

different purposes (Figure 1). The simple journey di-

agram is suitable for journeys with few actors and

emphasises any deviation from the planned journey.

The swimlane diagram is useful for journeys involv-

ing several actors and emphasises both the initiator

and the recipient of a touchpoint (Halvorsrud et al.,

2021; Halvorsrud et al., 2016a).

CJML has been extended to model cybersecurity-

IVAPP 2023 - 14th International Conference on Information Visualization Theory and Applications

204

Figure 1: CJML’s communication model with a sender

transmitting a message to a receiver through a communi-

cation channel (upper part). The visual representation of

a touchpoint in the case of a journey diagram (left) and a

swimlane diagram (right) (Boletsis et al., 2021).

related user journeys using elements from the

CORAS language. CORAS is ‘a model-driven ap-

proach to risk analysis that consists of a method, a

language and a tool to support the risk analysis pro-

cess’ (Lund et al., 2011). The CORAS graphical lan-

guage, an extension of the Uniﬁed Modelling Lan-

guage (UML) 2.0 speciﬁcation language, allows for

‘documentation of undesirable behaviour in the form

of threat scenarios’ (Vraalsen et al., 2005). In the

CORAS language, a threat is described as ‘using a

threat agent, e.g., a disloyal employee or a computer

virus. The threat agent initiates a threat scenario,

which is a sequence of events or activities leading

to an unwanted incident, i.e., an event resulting in a

reduction in the value of the target asset’ (Vraalsen

et al., 2005). These two new elements of threats and

unwanted incidents have been added in both CJML

formalism and its concrete syntax as icons (Figure

2). Moreover, the CJML users and ‘deviations’ were

redeﬁned to ﬁt the cybersecurity context, such that

the users are SME employees and external users that

make use of SME infrastructure under business-to-

business (B2B) offerings, and the ‘deviations’ are the

cyber threats (Boletsis et al., 2021).

Within the HORM cybersecurity-related context,

CJML has been applied using MS PowerPoint tem-

Figure 2: The icons of threat and unwanted incident that

were integrated from CORAS to CJML to extend CJML

and address cybersecurity-related user journeys.

plates as its tool

. Therefore, rather than relying on

third-party software, there is room and a need for the

development of a standalone modelling and diagram-

ming tool to act as a user interface (UI) for the ex-

tended CJML (i.e., HORM) modelling language (Fig-

ure 3). At the same time, UI usability is important,

and having a standalone, independent modelling tool

would enable developers to tweak the tool and its

source code until they reach satisfactory levels of UI

usability.

3 HORM DIAGRAMMING TOOL

3.1 Development

HORM-DT was developed using the open-source Di-

agrams.net (previously Draw.io) graph drawing soft-

ware by JGraph

(Figure 3), developed in HTML5

and JavaScript. The interface of the Diagrams.net

open-source software enables the user to easily create

and manage diagrams such as ﬂowcharts, wireframes,

organisational charts and network diagrams

. Its main

features include the creation of a wide range of di-

agrams, the availability of templates for diagrams,

display relationships between objects using various

types of graph element (such as shapes, arrows, text,

images and icons), the import of images/icons from

external sources and the storage and export of cre-

ated diagrams in several image formats and other for-

mats, such as XML, PDF, HTML and more (JGraph,

2022). The Diagrams.net open-source software was

Figure 3: The UI and modelling formalism components that

were utilised for the creation of HORM-DT.

MS PowerPoint templates for the extended CJML:

https://cjml.no/horm/CJML Diagram generator v1.pptx

Code repository of the open-source Diagrams.net soft-

ware: https://github.com/jgraph/drawio

Diagrams.net website: https://diagrams.net/

The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for SME Cybersecurity Awareness

205

Figure 4: A screenshot of the HORM-DT UI with the tweaked interface areas, based on a heuristic evaluation, highlighted in

red. HORM-DT can be accessed at https://cjml.no/horm2/

considered and used because i) it is widely pop-

ular among software engineers (Rauer, 2019; Von

Der Assen et al., 2022) and ii) it was presented, anal-

ysed, and/or used in several studies regarding mod-

elling languages and tools (Von Der Assen et al.,

2022; Jaimez-Gonz

alez and Mart

ınez-Samora, 2020;

ardi, 2022; Vakaliuk et al., 2021; Ch

avez-Feria

et al., 2022; Ch

avez-Feria et al., 2021).

3.2 User Interface Design

Although the Diagrams.net software comes with an

already designed UI, a heuristic evaluation of it was of

the essence because HORM-DT should address users

of all technical backgrounds (Figure 3). The heuristic

evaluation was conducted in an informal manner by

the authors, who utilised the ten usability heuristics

for UI design (Nielsen, 2020). The UI design inher-

ited from the Diagrams.net software fulﬁlled almost

all heuristics; however, the following few tweaks were

made to further address the speciﬁc nature and do-

main of our tool:

• Templates: Based on the CJML formalism, a user

journey modelling process should start by using a

template that contains the basic elements of the

concrete syntax. To that end, templates for swim-

lane and journey diagrams, as well as examples

of models, were added and made accessible right

from the landing page. At the same time, a dis-

tinctive ‘Templates’ menu button was added to the

left-hand side icon library (Figure 4) so that users

can access the templates at any point, thereby ad-

dressing the heuristic of providing ‘user control

and freedom’ (Nielsen, 2020). A text message at

the top of the UI was inserted (Figure 4) to remind

users about always using a template when start-

ing to design a cybersecurity-related diagram (ad-

dressing the ‘error prevention’ heuristic; Nielsen,

2020).

• Icon Library: The icon library at the left-hand

side of the UI (Figure 4) was redesigned and

simpliﬁed in relation to the one ‘inherited’ from

the Diagrams.net software. Icon groups of vari-

ous other charts included in Diagrams.net, such

as network charts, ﬂowcharts, tables, wireframes

and business charts, were removed, since they did

not comply with the tool’s cybersecurity-related

intended use. That way, a more focused, min-

imalist design approach was followed (address-

ing the ‘aesthetic and minimalist design’ heuris-

tic; Nielsen, 2020) with the aim of minimising

users’ cognitive load by enabling them to focus

only on icons deﬁned by the CJML concrete syn-

tax under the icon categories of Actors, Commu-

nication channels and Special symbols (address-

ing the ‘recognition rather than recall’ heuristic;

Nielsen, 2020). Naturally, users are still able to

create simple shapes, such as squares and circles,

by using the search function or the ‘+’ symbol on

the horizontal toolbar.

• Saving and Exporting Diagrams: Users can

save their diagrams in the native Draw.io, XML-

based format and can also export them in var-

ious image and document formats. The saving

and export functionalities are located under the

‘File’ drop-down menu (Figure 4). Diagrams.net

software, by default, enables saving diagrams lo-

IVAPP 2023 - 14th International Conference on Information Visualization Theory and Applications

206

Figure 5: Four screenshots presenting the UI of HORM-DT: a) the landing page, b) the ‘Templates menu’, c) the canvas and

d) the ‘Save as’ box.

cally and on connected, commercial cloud stor-

age providers, such as Google Drive and MS

OneDrive. Since these storage options belong

to commercial platforms (Von Der Assen et al.,

2022), which users may or may not use, their in-

tegration was excluded, and only saving/exporting

to the local drive was provided as an option; that

way, confusing users by suddenly introducing var-

ious storage options when saving/exporting could

be avoided (addressing the ‘consistency’-related

heuristic; Nielsen, 2020).

• Help and Guidance: The ‘Help’ dropdown menu

(Figure 4) was edited so that it provides users

guidance on the use of the tool and information

on the HORM framework. This change addressed

the ‘help and documentation’ heuristic (Nielsen,

2020).

3.3 Use

The basic functionalities of HORM-DT are presented

as follows:

• On the landing page, users can choose whether to

load a previously saved diagram or to create a new

one (Figure 5a).

• Upon making the choice of creating a new dia-

gram, users are asked to choose a template (Fig-

ure 5b). Templates can contain just the elements

of the HORM formalism or come with guidance

about what each symbol represents and how it is

used. Upon choosing a template, a canvas appears

on which users can work and create models using

icons, shapes, text and arrows (Figure 5c).

• When users want to save the model they have cre-

ated, there are two options. They can save the

model locally as a diagram ﬁle in the software’s

native XML-based format (Figure 5d), or they can

export it in the format of their choice (image or

document).

4 EVALUATION

4.1 Methodology

For the evaluation of HORM-DT, the focus was on the

tool’s perceived usability. Since the CJML formal-

ism and the CORAS framework have been validated

in a number of past user studies (Halvorsrud et al.,

2016a; Halvorsrud et al., 2021; Stølen, 2001; Stølen

et al., 2002; Dimitrakos et al., 2002; Raptis et al.,

2002), the main purpose of this work was to evalu-

ate the new UI elements and investigate the extent to

which HORM-DT (i.e., a diagramming tool based on

the Diagrams.net software) can facilitate user interac-

tion with the concrete syntax and the model creation

under the HORM-related formalism. A common way

of conducting the evaluation of modelling languages

and tools is to provide participants with textual de-

scriptions of scenarios/cases and ask of them to turn

them into models using the respective languages and

tools (Silva et al., 2018; Miranda et al., 2018). In this

work, and since the focus was exclusively on the tool,

the participants were provided with the visual rep-

resentations of cybersecurity-related user journeys,

The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for SME Cybersecurity Awareness

207

Figure 6: a) A HORM model of an online-service user journey with a problematic point/deviation, which had to be reproduced

by the study participants as the ﬁrst, introductory task. b) A HORM model of a phishing attack against an SME employee that

had to be reproduced by the study participants as the second, main task.

namely ready-made models, and asked to reproduce

them using HORM-DT. Therefore, the participants

were asked to use and evaluate the UI of HORM-DT

without needing to ‘speak’ HORM’s modelling lan-

guage.

The evaluation study took place between March

and June 2022 and followed a within-subject design.

Moreover, a within-subject comparison took place be-

tween the participants with prior experience with di-

agramming tools (hereafter called ‘experienced’) and

those without (‘inexperienced’).

4.2 Participants

The participants were recruited from the authors’ in-

stitutions, and the majority were students from the

University of Oslo. Prior knowledge of cybersecu-

rity and experience using diagramming tools were not

required. All the participants provided informed con-

sent to participate in the study.

4.3 Measures

The study deployed a mixed-methods research design

utilising qualitative and quantitative measures.

Demographic data were collected in the initial

stage of the study. A demographic questionnaire

collected information on age, gender, whether par-

ticipants had previous experience with diagramming

tools and, if so, which tool(s) they were.

To measure usability, the 10-item System Usabil-

ity Scale (SUS) questionnaire (Brooke, 2013) was

used. SUS is an instrument that allows usability prac-

titioners and researchers to measure the subjective

usability of products and services. SUS has been

utilised in several studies for measuring the perceived

usability of modelling languages and their compo-

nents (Eterovic et al., 2015; Miranda et al., 2018;

Nair et al., 2021; Silva et al., 2018; Aboussoror et al.,

2013). The 10-item questionnaire can be adminis-

tered quickly and easily, and it returns scores rang-

ing from 0 to 100. SUS scores can also be trans-

lated to adjective ratings, such as ‘worst imaginable’,

‘poor’, ‘OK’, ‘good’, ‘excellent’, ‘best imaginable’

and grade scales ranging from A to F (Bangor et al.,

2009). SUS has been shown both to be a reliable and

valid instrument that is robust with a small number

of participants and to have the distinct advantage of

being technology-agnostic, meaning it can be used to

evaluate a wide range of hardware and software sys-

tems (Brooke, 2013; Brooke, 1996; Tullis and Stet-

son, 2004; Kortum and Acemyan, 2013).

Next, a general-feedback, open-ended question-

naire was administered to collect the participants’

comments on HORM-DT. The participants were

asked the following two questions: i) what they liked

about their use of and experience with HORM-DT

and ii) what they did not like about it. Participants

were free to describe their experiences textually, and

there were no follow-up questions from the experi-

menter’s side.

The main task completion time was measured to

document the time needed for participants to repro-

duce a model for a cybersecurity-related user journey.

Finally, informal observations took place by the

experimenter that focused on modelling accuracy;

that is, the similarity between the ﬁnal user-submitted

models and the original models provided by the ex-

perimenter. The observations were transcribed in a

qualitative manner as high level descriptions of simi-

larities and differences.

4.4 Procedure

Due to COVID-19 restrictions, the entire evaluation

procedure took place online via the participants’ pre-

ferred video-conferencing platform (e.g., Zoom, MS

Teams or Skype), with them sharing their screens so

that the experimenter could observe their progress.

First, the participants were presented with an in-

troduction to the study, then they provided informed

consent and ﬁlled out a questionnaire on demograph-

ics and their experience with diagramming tools.

The experimenter then presented two tasks. The

ﬁrst was about reproducing a journey diagram model

of an online service user journey with a problematic

IVAPP 2023 - 14th International Conference on Information Visualization Theory and Applications

208

point/deviation (Figure 6a). It served as the introduc-

tory task, and so it was used to introduce participants

to the tool and become familiar with it by perform-

ing simple actions. For the second task, the partici-

pants had to reproduce a swimlane diagram model of

a phishing attack against an SME employee (Figure

6b). This was the main task of the evaluation study

and, based on pilot testing, its estimated completion

time was 25—45 minutes.

The models that had to be reproduced in the two

tasks were sent to the participants at the time of the

study via the video-conferencing platform as exported

images.

There was no time limit for the completion of the

tasks, and no external help was allowed for the con-

struction of the diagrams. At the end of each task,

the participants had to export their reproduced mod-

els in PNG image format and send them to the exper-

imenter via the video-conferencing platform. There

was no control or set threshold regarding the mod-

elling accuracy of the tasks (produced vs original) as

a requirement for completing the study.

Finally, participants completed the SUS and gen-

eral feedback questionnaires.

4.5 Statistical Analysis

All data were analysed using the IBM SPSS version

29 software. The signiﬁcance level was set at p <

0.05. Descriptive analysis was used to depict the de-

mographic data of the participants and to analyse the

SUS values. Welch’s t-test was used to detect dif-

ferences between the inexperienced and experienced

users’ SUS scores and main task completion times.

The data from the general feedback questions were

transcribed and then analysed through open and ax-

ial coding, which enabled core concepts, themes and

ideas to be identiﬁed. Two researchers coded the data

independently; the inter-rater reliability was assessed;

and any disagreements were identiﬁed, discussed and

settled.

5 RESULTS

Twenty-nine participants (N = 29, mean age: 27.76,

SD: 4.48, male/female: 17/12) evaluated HORM-DT.

Sixteen participants (N = 16, mean age: 26.38, SD:

3.56, male/female: 10/6) had never used diagram-

ming tools before (i.e., inexperienced users), and thir-

teen participants (N = 13, mean age: 29.46, SD: 5.03,

male/female: 7/6) had experience with diagramming

tools such as Draw.io/Diagrams.net, MS Visio, Miro

and the diagramming functions of MS Word and MS

PowerPoint (i.e., experienced users).

The HORM-DT interface scored a mean SUS

value of 80.69 (SD: 12.64). This value provided a

usability evaluation of between ‘good’ and ‘excel-

lent’, equivalent to a B grade (Brooke, 2013; Brooke,

1996). When the results were analysed and con-

trolled for participants’ previous experience with di-

agramming tools, the following results were identi-

ﬁed (Table 1). Experienced users (N = 13) awarded

a mean SUS value of 82.5 (SD: 14.43), and inexperi-

enced users (N = 16) awarded a mean SUS value of

79.22 (SD: 11.24). No statistically signiﬁcant differ-

ences were found between the experienced and inex-

perienced users’ SUS scores, based on Welch’s t-test:

t(22.4) = 0.67, p = 0.509.

Table 1: SUS results for HORM-DT.

Participants Mean SUS score

(SD)

All (N = 29) 80.69 (12.64)

– Experienced (N = 13) 82.5 (14.43)

– Inexperienced (N = 16) 79.22 (11.24)

The mean main task completion time (Table 2)

was 35.62 minutes (SD: 17.22); for experienced users

(N = 13), it was 29.38 minutes (SD: 13.2), and for

inexperienced users (N = 16), it was 40.69 minutes

(SD: 18.79). No statistically signiﬁcant differences

were found between the experienced and inexperi-

enced users’ main task completion time, based on the

Welch’s t-test: t(26.53) = −1.9, p = 0.069.

Table 2: Main task completion time results for HORM-DT.

Participants Mean main-task

completion time

(SD)

All (N = 29) 35.62 min. (17.22)

– Experienced (N = 13) 29.38 min. (13.2)

– Inexperienced (N = 16) 40.69 min. (18.79)

Table 3 presents the participants’ comments col-

lected from the two general feedback questions, to-

gether with the frequency of their occurrence. The

participants’ comments were further characterised as

positive (P) and negative (N).

Based on observations regarding modelling accu-

racy, all participants reproduced the diagrams with

very high accuracy, and only some typographical er-

rors and small positioning differences in lines and

shapes featured in their produced models.

The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for SME Cybersecurity Awareness

209

Table 3: Participants comments as collected from the two general-feedback questions.

Comment Count

P HORM-DT is simple and easy to use. 18

N It can be challenging to use the arrows to connect objects and form straight lines. 9

P Icons and graphical objects are clearly designed and categorised. 6

P Templates are useful and easily accessible. 4

P HORM-DT is suitable for novice users. 3

P It is easy to connect objects. 3

N When inserting text, the default font size is too small. 2

N HORM-DT does not contain all the libraries of Diagrams.net. 1

N Some icons are not immediately visible and need to be searched for. 1

6 DISCUSSION

In this work, HORM-DT was designed and devel-

oped to be a standalone, open and customised tool for

the application of extended CJML within the HORM

framework. The tool’s UI and functionality were

based on the Diagrams.net software and then signiﬁ-

cantly extended and redesigned to ﬁt the customised

needs of the modelling language domain. To that end,

a heuristic evaluation of the Diagrams.net UI took

place, and new icon libraries and templates were cre-

ated and new interaction steps regarding model devel-

opment were introduced, among other changes (Sec-

tion 3.2). The usability of the tool was central to the

study, paving the way for using HORM-DT when ex-

amining the effectiveness of the modelling language

for raising SME cybersecurity awareness.

Since HORM-DT featured satisfactory and good

usability values, based on the SUS score, it can be said

that the chosen design and development process con-

tributed to reaching the goal. The usability, simplicity

and user-friendliness of the tool were also praised by

the participants. The clearly designed icons, graphi-

cal objects and useful templates may have contributed

to that, based on the participants’ comments. On the

other hand, the drawing of lines was negatively rated,

since Diagrams.net features a very simplistic way of

connecting items with lines (by connecting predeﬁned

points); however, for new users, this might not have

been obvious and required a starting tutorial.

As for the comparison of the perceived usability

and main task completion time between experienced

and inexperienced users (Tables 1 and 2), there were

naturally differences; however, those differences were

not found to be statistically signiﬁcant. Therefore, it

may be said that HORM-DT can facilitate use even

by users that have no prior experience with diagram-

ming tools and fulﬁl the goal of enabling modelling

by users with diverse technical backgrounds (Section

1).

Finally, in this work, the decision was made to

use existing open-source code (i.e., Diagrams.net), in-

stead of starting from the ground up. This happened

for the following two reasons: i) Diagrams.net is a

widely used tool with a user-friendly UI, and it is

used frequently in the modelling-language domain (as

mentioned in Section 3.1), and ii) a green coding strat-

egy based on reusing existing code (Verdecchia et al.,

2021) was followed. In terms of the latter strategy,

this work alone may not have had an enormous en-

vironmental impact, but it has helped create a coding

attitude towards the environmental goal of reducing

software’s energy consumption, which can be applied

in future projects and contribute to raising awareness

about the topic.

6.1 Implications

In this work, the following elements can be gener-

alised and inform the designs of other practitioners

and researchers in the ﬁeld:

• A modelling tool for mapping cybersecurity-

related user journeys is openly deployed for use

by other practitioners and researchers in their

cybersecurity-related use cases. The tool features

satisfactory usability and can facilitate modelling

by users with diverse technical backgrounds.

Moreover, the tool’s code is openly distributed for

interested parties to further extend and build upon.

• An effective and resource-efﬁcient methodolog-

ical approach is introduced to measure the per-

ceived usability of modelling tools when the un-

derlying formalism remains the same, albeit with

an updated UI. The proposed methodology fol-

lows formalism-agnostic logic when the UI of a

modelling-language tool is updated so that i) the

evaluation results solely address and focus on the

updated UI elements and their usability perfor-

mance, and ii) resources such as time and cost are

saved during the evaluation process, since partic-

IVAPP 2023 - 14th International Conference on Information Visualization Theory and Applications

210

ipants are not required to become accustomed to

the tool’s formalism in order to use it for the pur-

poses of the study.

• An extension of the aforementioned methodologi-

cal approach is also introduced to measure the per-

ceived usability coming from users with diverse

attributes (in this case, prior experience with dia-

gramming tools). Therefore, in cases in which a

modelling language tool targets a heterogeneous

user group, a within-group comparative approach

based on the differentiating attribute can beneﬁt

the usability measurements and the conclusions

that come out of it.

6.2 Limitations

This evaluation study of HORM-DT had the follow-

ing limitations:

• COVID-19 restrictions affected the evaluation

process. Although the tasks were digital and

could take place remotely, the qualitative part of

the evaluation study might have been carried out

in a more effective way with physical attendance,

such as through personal interviews. The restric-

tions also affected the recruitment process, lead-

ing to convenience sampling and contributing to

the next point.

• Since the majority of the study participants were

students from the University of Oslo (i.e., con-

venience sampling), a participant sample of a

young age (mean age: 27.76, SD: 4.48) was re-

cruited. The study participants’ technical back-

ground and prior experience with diagramming

tools may have been affected by their young age.

• The participants, especially those without prior

experience with diagramming tools, may have

needed detailed guidance to become familiar with

drawing lines when using HORM-DT. Neverthe-

less, the logic behind drawing lines and connect-

ing elements became clear after a while, but a

more detailed starting tutorial could alleviate the

few negative experiences on that matter, as docu-

mented in Table 3.

7 CONCLUSION

In this work, HORM-DT was introduced as a tool

for modelling cybersecurity-related journeys to help

SMEs to raise cybersecurity awareness and take the

ﬁrst step towards deﬁning or updating their cyberse-

curity practices. The tool was examined in terms of

the usability its UI offers to users, producing satisfac-

tory and promising results. Apart from the actual tool,

which has been openly deployed online and its code

is freely accessible, the work also introduced a cost-

effective methodological approach for evaluating the

usability of modelling tools that have updated UIs but

still feature the same formalism. In the future, further

work will be conducted on i) developing a starting

tutorial (e.g., an introductory video) on how simple

HORM-DT diagrams can be made, with an empha-

sis on drawing lines and connecting objects; ii) ex-

ploring the effectiveness of the overall HORM mod-

elling language, as facilitated by HORM-DT, for rais-

ing SME cybersecurity awareness and involving var-

ious target groups, such as cybersecurity experts and

regular, non-technical employees; and iii) developing

a text editor as an extension of the tool that would en-

able users to textually describe their models, which

would be automatically translated to diagrams.

ACKNOWLEDGEMENTS

We would like to thank the IVAPP 2023 review-

ers for their constructive feedback. This research

is funded by the European Commission through the

CyberKit4SME project (www.cyberkit4sme.eu) un-

der Grant Agreement 883188. CyberKit4SME will

provide cybersecurity tools that help SMEs become

aware of, analyse, and manage cybersecurity and data

protection risks.

REFERENCES

Aboussoror, E. A., Ober, I., and Ober, I. (2013). Signiﬁ-

cantly increasing the usability of model analysis tools

through visual feedback. In International SDL Forum,

pages 107–123. Springer.

ardi, A. (2022). Relontouml model of the archaeological

ﬁndings. Production Systems and Information Engi-

neering, 10(1):64–71.

Arctic Wolf (2017). The state of mid-market

cybersecurity: Findings and implications.

https://2p167arhj4lo70dn1q26fm1c-wpengine.netdn

a-ssl.com/wp-content/uploads/AW Brief Midmarket

Cybersecurity Survey.pdf (Accessed 17 Nov 2022).

Bangor, A., Kortum, P., and Miller, J. (2009). Determin-

ing what individual SUS scores mean: Adding an

adjective rating scale. Journal of Usability Studies,

4(3):114–123.

Bellamy, R. K., Erickson, T., Fuller, B., Kellogg, W. A.,

Rosenbaum, R., Thomas, J. C., and Wolf, T. V. (2007).

Seeing is believing: Designing visualizations for man-

aging risk and compliance. IBM Systems Journal,

46(2):205–218.

The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for SME Cybersecurity Awareness

211

Benz, M. and Chatterjee, D. (2020). Calculated risk? A cy-

bersecurity evaluation tool for SMEs. Business Hori-

zons, 63:531–540.

Boletsis, C., Halvorsrud, R., Pickering, J. B., Phillips, S. C.,

and Surridge, M. (2021). Cybersecurity for SMEs:

Introducing the Human Element into Socio-technical

Cybersecurity Risk Assessment. In VISIGRAPP (3:

IVAPP), pages 266–274.

Brooke, J. (1996). SUS - A quick and dirty usability scale.

In Jordan, P., Thomas, B., Weerdmeester, B., and Mc-

Clelland, I., editors, Usability Evaluation in Industry,

pages 189–194. Taylor & Francis.

Brooke, J. (2013). SUS: A Retrospective. Journal of Us-

ability Studies, 8(2):29–40.

avez-Feria, S., Garc

ıa-Castro, R., and Poveda-Villal

on,

M. (2021). Converting UML-based ontology concep-

tualizations to OWL with Chowlk. In European Se-

mantic Web Conference, pages 44–48. Springer.

avez-Feria, S., Garc

ıa-Castro, R., and Poveda-Villal

on,

M. (2022). Chowlk: from UML-based ontology con-

ceptualizations to OWL. In European Semantic Web

Conference, pages 338–352. Springer.

Dimitrakos, T., Ritchie, B., Raptis, D., and Stølen, K.

(2002). Model based security risk analysis for web

applications: the CORAS approach. In EuroWeb 2002

Conference, pages 1–13.

Eterovic, T., Kaljic, E., Donko, D., Salihbegovic, A., and

Ribic, S. (2015). An Internet of Things visual do-

main speciﬁc modeling language based on UML.

In 2015 XXV International Conference on Informa-

tion, Communication and Automation Technologies

(ICAT), pages 1–5. IEEE.

Fair, N., Phillips, S., Erdogan, G., and Tverdal, S. (2022).

Information security & risk management: trustwor-

thiness and human interaction. In Tutorial proceed-

ings of the 16th International Conference on Research

Challenges in Information Science, pages 1–48.

RCIS. https://www.rcis-conf.com/rcis2022/ﬁles/T1-

NicholasFairEtAl.pdf.

Halvorsrud, R., Boletsis, C., and Garcia-Ceja, E. (2021).

Designing a modeling language for customer jour-

neys: Lessons learned from user involvement. In 2021

ACM/IEEE 24th International Conference on Model

Driven Engineering Languages and Systems (MOD-

ELS), pages 239–249. IEEE.

Halvorsrud, R., Haugstveit, I. M., and Pultier, A. (2016a).

Evaluation of a modelling language for customer jour-

neys. In 2016 IEEE Symposium on Visual Languages

and Human-Centric Computing (VL/HCC), pages 40–

48. IEEE.

Halvorsrud, R., Kvale, K., and Følstad, A. (2016b). Improv-

ing service quality through customer journey analysis.

Journal of Service Theory and Practice, 24(6):840–

867.

Halvorsrud, R., Lee, E., Haugstveit, I. M., and Følstad, A.

(2014). Components of a visual language for service

design. In ServDes. 2014 Service Future, Proceedings

of the fourth Service Design and Service Innovation

Conference, Lancaster University, United Kingdom,

9-11 April 2014.

Haugstveit, I. M., Halvorsrud, R., and Karahasanovic, A.

(2016). Supporting redesign of C2C services through

customer journey mapping. In Service Design Ge-

ographies. Proceedings of the ServDes. 2016 Confer-

ence, number 125, pages 215–227. Link

oping Univer-

sity Electronic Press.

Jaimez-Gonz

alez, C. and Mart

ınez-Samora, J. (2020).

DiagrammER: A Web Application to Support the

Teaching-Learning Process of Database Courses

Through the Creation of ER Diagrams. International

Journal of Emerging Technologies in Learning (iJET),

15(19):4–21.

JGraph (2022). Features of diagrams.net and draw.io.

https://diagrams.net/features (Accessed 17 Nov

2022).

Khan, M. I., Tanwar, S., and Rana, A. (2020). The Need

for Information Security Management for SMEs. In

2020 9th International Conference System Modeling

and Advancement in Research Trends (SMART), pages

328–332. IEEE.

Kortum, P. and Acemyan, C. Z. (2013). How low can you

go? Is the System Usability Scale range restricted?

Journal of Usability Studies, 9(1):14–24.

Kullman, K., Buchanan, L., Komlodi, A., and Engel, D.

(2020). Mental model mapping method for cyber-

security. In International Conference on Human-

Computer Interaction, pages 458–470. Springer.

Lund, M. S., Solhaug, B., and Stølen, K. (2011). Risk anal-

ysis of changing and evolving systems using CORAS.

In International School on Foundations of Security

Analysis and Design, pages 231–274. Springer.

Meshkat, L., Miller, R. L., Hillsgrove, C., and King, J.

(2020). Behavior modeling for cybersecurity. In

2020 Annual Reliability and Maintainability Sympo-

sium (RAMS), pages 1–7. IEEE.

Meszaros, J. and Buchalcevova, A. (2017). Introducing

OSSF: A framework for online service cybersecurity

risk management. computers & security, 65:300–313.

Miranda, T., Challenger, M., Tezel, B. T., Alaca, O. F.,

Bari

c, A., Amaral, V., Goul

ao, M., and Kardas, G.

(2018). Improving the usability of a mas dsml. In In-

ternational workshop on engineering multi-agent sys-

tems, pages 55–75. Springer.

Nair, A., Ning, X., and Hill, J. H. (2021). Using recom-

mender systems to improve proactive modeling. Soft-

ware and Systems Modeling, 20(4):1159–1181.

Nielsen, J. (2020). 10 Usability Heuristics for User Inter-

face Design . https://www.nngroup.com/articles/ten-

usability-heuristics/ (Accessed 17 Nov 2022).

Nurse, J. R., Creese, S., Goldsmith, M., and Lamberts, K.

(2011). Trustworthy and effective communication of

cybersecurity risks: A review. In 2011 1st Work-

shop on Socio-Technical Aspects in Security and Trust

(STAST), pages 60–68. IEEE.

Paulsen, C. (2016). Cybersecuring small businesses. Com-

puter, 49(8):92–97.

Ponsard, C. and Grandclaudon, J. (2019). Guidelines and

Tool Support for Building a Cybersecurity Aware-

ness Program for SMEs. In International Conference

IVAPP 2023 - 14th International Conference on Information Visualization Theory and Applications

212

on Information Systems Security and Privacy, pages

335–357. Springer.

Raptis, D., Dimitrakos, T., Gran, B. A., and Stølen, K.

(2002). The CORAS approach for model-based risk

management applied to e-commerce domain. In Ad-

vanced Communications and Multimedia Security,

pages 169–181. Springer.

Rauer, M. (2019). Draw.io diagramming in Con-

ﬂuence is currently the most successful app in

the Atlassian Marketplace. https://blog.seibert-

media.com/2019/04/18/draw-io-diagramming-in-

conﬂuence-is-currently-the-most-successful-app-

in-the-atlassian-marketplace/ (Accessed 17 Nov

2022).

Shojaifar, A. (2019). SMEs Conﬁdentiality Issues and

Adoption of Good Cybersecurity Practices. In IFIP

Summer School on Privacy and Identity Management,

pages 1–8.

Silva, J., Bari

sic, A., Amaral, V., Goul

ao, M., Tezel, B. T.,

Alaca, O. F., Challenger, M., and Kardas, G. (2018).

Comparing the usability of two multi-agents systems

DSLs: Sea ml++ and DSML4MAS study design. In

3rd International Workshop on Human Factors in

Modeling, volume 2245, pages 770–777.

Skubisz, C., Reimer, T., and Hoffrage, U. (2009). Com-

municating quantitative risk information. Annals

of the International Communication Association,

33(1):177–211.

Slovic, P. (1999). Trust, emotion, sex, politics, and science:

Surveying the risk-assessment battleﬁeld. Risk analy-

sis, 19(4):689–701.

Stickdorn, M., Hormess, M. E., Lawrence, A., and Schnei-

der, J. (2018). This is service design doing: applying

service design thinking in the real world. ” O’Reilly

Media, Inc.”.

Stølen, K. (2001). CORAS – A Framework for Risk Analy-

sis of Security Critical Systems. In supplement of the

2001 International Conference on Dependable Sys-

tems and Networks, pages D4-D11.

Stølen, K., den Braber, F., Dimitrakos, T., Fredriksen, R.,

Gran, B. A., Houmb, S.-H., Lund, M. S., Stamatiou,

Y., and Aagedal, J. (2002). Model-based risk assess-

ment – the CORAS approach. In iTrust Workshop.

Surridge, M., Meacham, K., Papay, J., Phillips, S. C., Pick-

ering, J. B., Shaﬁee, A., and Wilkinson, T. (2019).

Modelling compliance threats and security analysis of

cross border health data exchange. In International

Conference on Model and Data Engineering, pages

180–189. Springer.

Symantec (2019). Symantec 2019 internet security threat

report. https://docs.broadcom.com/doc/istr-24-2019-

en (Accessed 17 Nov 2022).

The National Center for the Middle Market (2016).

Cybersecurity and the middle market: The

importance of cybersecurity and how mid-

dle market companies manage cyber risks.

https://www.middlemarketcenter.org/Media/-

Documents/the-importance-of-cybersecurity-and-

how-middle-market-companeis-manage-cyber-

risks NCMM Cybersecurity Report FINAL.pdf

(Accessed 17 Nov 2022).

Tullis, T. S. and Stetson, J. N. (2004). A comparison

of questionnaires for assessing website usability. In

Proceedings of the Usability Professional Association

Conference, pages 1–12. Usability Professional Asso-

ciation.

Vakaliuk, T. A., Korotun, O. V., and Semerikov, S. O.

(2021). The selection of cloud services for ER-

diagrams construction in IT specialists databases

teaching. In CTE Workshop Proceedings, volume 8,

pages 384–397.

Verdecchia, R., Lago, P., Ebert, C., and De Vries, C.

(2021). Green IT and green software. IEEE Software,

38(6):7–15.

Von Der Assen, J., Franco, M. F., Killer, C., Scheid, E. J.,

and Stiller, B. (2022). CoReTM: An Approach En-

abling Cross-Functional Collaborative Threat Model-

ing. In 2022 IEEE International Conference on Cyber

Security and Resilience (CSR), pages 189–196. IEEE.

Vraalsen, F., Lund, M. S., Mahler, T., Parent, X., and

Stølen, K. (2005). Specifying legal risk scenarios us-

ing the CORAS threat modelling language. In Inter-

national Conference on Trust Management, pages 45–

60. Springer.

The HORM Diagramming Tool: A Domain-Speciﬁc Modelling Tool for SME Cybersecurity Awareness

213