Visualizing the Information Security Maturity Level of Public Cloud

Services Used by Public Administrations

Michael Diener

and Thomas Bolz

University of Regensburg, Regensburg, Germany

IU International University of Applied Sciences, Erfurt, Germany

Keywords:

Public Administration, Public Cloud, Visualization, Information Security Management, Security Audit.

Abstract:

The digitization of public administrations in Germany is making slow progress. At the same time, more and

more innovative IT solutions are available on the market for solving practical business problems, e.g. web-

based ﬁle sharing applications that are offered by external cloud service providers. Due to data protection

regulations and uncertainties regarding information security issues, the adoption and operation of public cloud

services within public administrations is a challenging task. As part of our research, we constructed a three-

phase process model that uses a web-based tool approach, in order to support chief information ofﬁcers to

manage security audits of various public cloud services that are used by different organizational units. To en-

sure the efﬁcient, transparent and comprehensive conduction of cloud security audits, we developed graphical

visualization components that illustrate the information security maturity level in relation to multiple security

requirements of the analyzed public cloud services. We have successfully evaluated our proposed tool visual-

ization under real conditions within a public administration. Furthermore, we discussed several use cases and

the user experience with different experts in this application domain.

1 INTRODUCTION

Cloud computing is a trending technology that is

nowadays increasingly adopted by government insti-

tutions (Mell et al., 2011). The usage of cloud tech-

nologies can both reduce ongoing costs of IT expen-

ditures and achieve efﬁciency advantages. Nowadays,

cloud-based IT services offer a wide range of tech-

nological options to solve organizational and techni-

cal problems efﬁciently e.g., AWS EC2, Cisco We-

bex Meetings or Google Workspace. Microsoft 365 is

also increasingly being discussed as a possible solu-

tion in public authorities, but in its current version it

still has numerous conﬂicts with European data pro-

tection regulations (Syynimaa and Viitanen, 2018).

In the recent past, public cloud services have

also been increasingly used by public administrations

(Nanos et al., 2019). There are many reasons for this.

On the one hand, software manufacturers are increas-

ingly tending to offer software products less or not at

all for on-premises installations to achieve economies

of scale (Armbrust et al., 2009). On the other hand,

innovative IT services can be adopted instantly by

means of cloud computing, for which previously own

high-performance data centers would have been re-

quired. In addition, the adoption of cloud services

can relieve the organization’s IT staff, which is usu-

ally necessary for the operation of inhouse IT.

Against the backdrop of the need to massively

drive forward the digitization of public administra-

tions in Europe (Braud et al., 2021; European Union,

2018), cloud solutions appear to be a suitable IT so-

lution (Zaharia-R

adulescu et al., 2017; Galletta et al.,

2017). For example, smart city projects require pow-

erful IT architectures that support collaboration with

various stakeholders (Ge and Buhnova, 2022). The

appropriate IT services can be adopted from the cloud

so that processing of real-time data becomes possible

(Su et al., 2022). In addition, cloud-based IT services

can signiﬁcantly improve the level of IT security, as

cloud security experts continuously ensure the imple-

mentation of the necessary security measures, thus

guaranteeing the protection of entrusted data (Henze

et al., 2020).

In most cases, public administrations process per-

sonal data in IT systems, sometimes even special cat-

egories of personal data (cf. Article 9 EU-GDPR).

Consequently, data protection and information se-

curity must be considered critically when it comes

to processing of sensitive or personal data in public

192

Diener, M. and Bolz, T.

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations.

DOI: 10.5220/0012637700003711

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 14th International Conference on Cloud Computing and Services Science (CLOSER 2024), pages 192-203

ISBN: 978-989-758-701-6; ISSN: 2184-5042

cloud environments (Rath et al., 2023; Sasubilli and

Venkateswarlu, 2021). Although information security

and data protection of cloud solutions are highly spec-

iﬁed in public tenders, managing the information se-

curity of external cloud services is still a major chal-

lenge for public administrations (Castro et al., 2019;

Nycz and Polkowski, 2015). Furthermore, newly de-

signed and secure federal cloud infrastructures (cf.

Bundescloud, Deutsche Verwaltungscloud etc.) are

currently not available for small and medium-sized lo-

cal authorities in Germany. This means that they have

to purchase secure public cloud services themselves,

which increases the risk of security gaps immensely.

The issue that the management of information se-

curity in public authorities is inadequate is shown by

the fact that there have been approximately 100 doc-

umented IT attacks on public authorities in Germany

between 2020 and 2024 (Lange, 2024). Not all of

them have been successful, but the sheer number of

attacks shows how dramatic the trend is among Ger-

man authorities.

As a result of the increasing adoption of public

cloud services in public administrations, it must be as-

sumed that there will be an expansion of data breaches

or cyberattacks. Suitable practical solutions for secur-

ing public cloud services in the area of public admin-

istrations are currently indispensable.

This paper proposes a possible technical solution

that can support responsible actors in public admin-

istrations to enhance the required information secu-

rity maturity level for used public cloud services by

offering advanced visualization techniques based on

realtime audit information.

The remainder of this paper is organized as fol-

lows: In the next section we describe practical prob-

lems of public administrations with respect to infor-

mation security management of public clouds using

the example of a medium-sized municipality. In the

related work part we describe how visualization grids

can be used to highlight anomalies in security con-

ﬁgurations (cf. section 3). In section 4, we propose

a process model that describes how tool-guided au-

dits with focus on public clouds in a municipality can

be implemented into information security processes.

Based on this, we present our developed prototype,

which provides visualization grids for highlighting

anomalies due to organizational and technical secu-

rity requirements within public clouds (cf. section

5). Following, we evaluate the proposed prototype in

section 6. Based on this, we discuss in section 7 the

evaluation results and the user experience of our de-

veloped prototype with experts from different public

administrations. In the last part of the paper, we sum-

marize the results of our work and provide an outlook.

2 BACKGROUND

One of the authors is chief information security ofﬁ-

cer (CISO) of a medium-sized municipal government

and has deep insights into information security pro-

cesses. Since the beginning of the Covid-19 pandemic

in spring 2020, observations have been ongoing re-

garding the adoption, the implementation, and the us-

age of public cloud services in its public administra-

tion.

2.1 Unclear Responsibilities and

Changing Structures

Basically, the expectations of implementing cross-

organizational IT solutions to drive digitization for-

ward, have increased dramatically. In particular,

the number of cloud applications has massively in-

creased, as less internal resources (e.g., IT tech-

nology, personnel) are necessary compared to on-

premises applications. In contrast, the responsibility

for managing external cloud services lies with those

departments that originally request the cloud service.

However, the problem is that employees in request-

ing departments often have little or no IT expertise,

that leads to serious gaps in information security pro-

cesses. To make matters worse, SaaS applications

in clouds are often multi-tenant and can be used in

various departments under different responsibilities.

This makes it increasingly difﬁcult to keep track of

the adopted cloud solutions within public administra-

tions. As a result of changes in departmental responsi-

bilities, it can happen that the organizational and tech-

nical administration of public clouds are not carried

out to the necessary degree as they are required by

security requirements of standards (e.g., ISO 27001,

BSI C5, CSA CCM, BSI IT-Grundschutz, CISIS12

etc.)

2.2 Security Audits of Public Clouds

Services

Today, many SaaS products do not offer an OpenID

interface, so that a single-sign-on (SSO) against a

user directory like Active Directory is technically un-

feasible. Since many different public cloud services

are adopted by public administrations, it is difﬁcult

to use standardized APIs. As a consequence, public

cloud services have their own identity management

that needs to be controlled manually.

If these manual identity checks are not carried out

carefully, serious security gaps can arise in identity

management processes. In a worst-case scenario, for-

mer employees could have full access to sensitive data

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations

193

in public clouds, which they should no longer have.

In addition, the activation of important security con-

ﬁgurations in each tenant of a public cloud service is

of particular importance. For example, the password

quality requirements must be correctly conﬁgured by

the responsible manager of a public cloud service.

In general, the problem is that such security

checks must be done manually under these circum-

stances and need to be done regularly for the identi-

ﬁed public cloud services. In this respect, the only op-

tion for CISOs might be to ask the responsible cloud

managers about the compliance of the deﬁned orga-

nizational security requirements. However, these or-

ganizational security requirements are extremely dif-

ﬁcult to implement in practice.

3 RELATED WORK

3.1 Visualization of Information

The possibilities of visualizing data relationships have

been researched for a long time and support a wide

variety of presentation techniques (Mazza, 2009). In-

creasingly, visualization techniques are being used to

support decisions in information security processes

by presenting complex relationships in simpliﬁed pre-

sentations (Yermalovich, 2020). As a consequence

it becomes possible to illustrate complex correlations

of business processes, system conﬁgurations etc. For

example, Colantonio et al. developed a grid based

visualization technique which has simpliﬁed the un-

derstanding of roles in social networks (Colantonio

et al., 2011). In their proposed approach, they visual-

ized existing access permissions between users and

objects using a two-dimensional grid (Meier et al.,

2013). With the help of visual grids, complex data

representations can be depicted graphically. Item se-

ries A is assigned to the x-axis, while item series B is

assigned to the y-axis (cf. ﬁgure 1).

Grid theorie

item A

item B

item C

item D

feature 1

feature 2

feature 3

feature 4

feature 5

y-axis

x-axis

item A1

item A2

item A3

item A4

item A

item B1

x x x

item B2

x x

item B3

item B4

x x

item

x-axis

y-axis

Figure 1: Grids are able to represent two-dimensional rela-

tions of data.

If there is any relationship between two items, it

can be easily presented by a simple cross. The state-

ment about the relationship between two character-

istics can then be depicted in a cell, e.g., by a nu-

merical value or a colored representation. For our

research we will adapt the principles of this visual-

ization technique. Heatmaps are sophisticated visu-

alizations with which multidimensional facts can be

presented. These are not required for our work.

3.2 Security Audits of Public Cloud

Services

The maturity level of information security can be de-

termined by adapting different methods (Proenc¸a and

Borbinha, 2018; Jaatun et al., 2017). A widespread

approach bases on questionnaires (Schmitz et al.,

2021). The status quo is documented in relation to

speciﬁc information security requirements (Parsons

et al., 2017). Depending on the design of the ques-

tions, an automated evaluation of the answers can

take place. Typically, information security manage-

ment tools (e.g., eramba

, verinice

) support CISOs

in managing technical and organizational security re-

quirements.

A highly simpliﬁed and ubiquitous approach for

auditing public cloud services is based on validity

checks of their underlying certiﬁcations (cf. ISO

27001). The problem with this audit approach might

be that it is not possible to validate organizational se-

curity processes at the cloud customer side (Di Giulio

et al., 2017). For example, the quality of identity man-

agement processes cannot be expressed by cloud cer-

tiﬁcates. Lins et al. are exploring the approach of

dynamic cloud certiﬁcation (Lins et al., 2019). By

applying this form of auditing, standardized measure-

ment indicators are regularly queried and evaluated,

e.g. the availability of the cloud over time. In this re-

spect, it is possible to monitor promised service level

agreements (SLAs) during operation (Stephanow and

Fallenbeck, 2015). However, by using this dynamic

approach it is also difﬁcult to automatically check se-

mantic relationships, for example, the allocation of

access rights to cloud users.

Recently published research has focused on meth-

ods with multi-dimensional certiﬁcation to handle

user’s requirements on the full public-cloud-services

life cycle (Anisetti et al., 2023).

https://www.eramba.org

https://verinice.com/en

CLOSER 2024 - 14th International Conference on Cloud Computing and Services Science

194

4 CONCEPTUAL DESIGN:

EFFICIENT CLOUD SECURITY

AUDITS IN PUBLIC

ADMINISTRATIONS

Our approach is based on the idea of conducting

organization-wide security audits in different depart-

ments which have adopted public cloud services. Se-

curity audits in public administrations must be sim-

ple and efﬁcient, so that regular repetitions are ac-

cepted by the involved actors. By iteratively applying

the principles of the design science research (DSR)

method (Peffers et al., 2007), we developed a three-

phase process model, with the goal of determining the

maturity level of information security:

• Phase 1: Inventorying Existing Public Clouds.

As a ﬁrst step, we conduct a inventory of all pub-

lic cloud services in each department. We also

determine who is responsible for the management

of the respective cloud. This step is performed for

each organizational unit of the public administra-

tion by asking skilled employees.

• Phase 2: Auditing the Public Cloud Security.

The used public cloud services are then examined

more precisely. It is important that cloud services

are audited per organizational unit, as different or-

ganizational conditions and settings can be given.

The answers of phase 1 serve as starting point.

• Phase 3: Evaluating the Cloud Information Se-

curity Maturity Level. Based on the questions

answered in phase 2, the maturity level of infor-

mation security is determined for each organiza-

tional unit for each adopted public cloud service.

In this step, deviations and anomalies are identi-

ﬁed by using graphical visualizations.

As part of the DSR design cycle, we discussed with

various stakeholders in six different public admin-

istrations, including chief information ofﬁcers, data

protection ofﬁcers and e-government managers. Ad-

ditionally, we discussed with several heads of various

departments of our own public administration. Dur-

ing this process, we were able to identify three differ-

ent roles that are mandatory to perform information

security audits of public cloud services within public

administrations successfully. Each of the actors in-

volved is taking on activities that are mapped to the

proposed three-phase process model. Figure 2 illus-

trates in BPMN 2.0 notation the holistic process of

security audits of public cloud services within a pub-

lic administration and the respective activities of the

actors involved.

In general, it is essential that all incoming and

Auditing public clouds in public administrations

tool-support

audit

repository

questionnaire

repository

Sending regular

"friendly reminder"

emails in case of

open questionnaires

CISO

Creating online-audit

with template

"cloud inventory"

Creating online-audit

with template

"cloud basics" for

cloud x AND

organizational unit y

manual start, if cloud

inventory is available for

Organization Unit y

Evaluation of the

information security

maturity level for all

public cloud services

Identification of

possible measurements

for organizational unit y

due to cloud x

HOU [Org.Unit y]

Answering the

questionnaire

"cloud inventory" for

Organization Unit y

Email notification

Submission

CPO [Org.Unit y]

Email notification

Answering the

questionnaire

"cloud basics" for

cloud x in org. unit y

Submission

Figure 2: Holistic process of cloud security audits within

public administrations.

outgoing audit information is stored within a central

repository. This ensures that audit data is captured in

a structured way and is reusable for different activi-

ties. In addition, it is important to involve the man-

agement of each organizational unit in the audit pro-

cess as early as possible. In this context, we explain

why security audits are necessary for public cloud ser-

vices and for what reason employees of a department

need to be involved. In terms of conducting cloud se-

curity audits in public administrations, we identiﬁed

the following actors and activities:

• CISOs are responsible for the central manage-

ment of all information security audit and are sig-

niﬁcantly involved in the activities of the three

phases. For this purpose, the respective heads of

the organizational units (HOUs) need to be iden-

tiﬁed and documented in phase 1.

• HOUs themselves determine for their organiza-

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations

195

tional unit which cloud services are being used

and which employees are responsible for their

management in each case. The answers are sub-

mitted to the CISO in a structured form. The

employees, that are responsible for administra-

tive cloud management activities are called Cloud

Product Owners (CPOs).

• CPOs answer speciﬁc audit questions that are re-

lated to the public cloud services for which they

are responsible. Moreover, in many cases they

are responsible to implement security measures in

their application ﬁeld. Due to the involvement of

HOUs in individual organizational units, a seam-

less relationship between CISOs and CPOs can be

easily established.

Overall, it becomes clear that a comprehensive tool

support is necessary to apply the proposed process

model in the domain of public administrations. Dur-

ing the DSR cycle, we conducted several interviews

thus we could identify three key requirements that are

essential for enhanced tool support:

1. It must be immediately apparent which organiza-

tional unit uses speciﬁc public cloud services.

2. In addition, the status of the initiated security au-

dits must be traceable.

3. It must be apparent to what extent speciﬁc secu-

rity requirements of public cloud services are im-

plemented.

4.1 Analysis of Existing Audit Software

In this context, we also looked at three different pro-

prietary security tools that are common and currently

used by CISOs in their public administrations.

It was noticeable that all three tools were used ex-

clusively by CISOs due to their enormous range of

functions. In all cases, user training would be neces-

sary to provide CPOs with the necessary basic under-

standing of how to use the questionnaires. In addition,

full licenses are always required for the use of online

questionnaires in the tools, meaning that the number

of cloud audits that can be carried out in parallel is

likely to be low.

None of the three tools examined have a work-

ﬂow engine that would support the integration of

HOUs. This results in additional coordination effort

for CISOs, as additional tools would have to be used

in the process model. In a highly dynamic cloud en-

vironment, this is a disadvantage for the analysis of

information security.

Furthermore, the CPOs’ responses in the ques-

tionnaires could not be evaluated automatically. This

additional activity would have to be carried out by

CISOs, which would likely mean that cloud audits

would not be repeated on a regular basis.

Two of the tools examined do not have a web-

based interface, which presents an additional difﬁ-

culty in large structures of public administrations.

One tool enables the use and customization of dash-

boards, but only data that is known can be visualized.

Missing data cannot be visualized as an anomaly in

this dashboard.

Altogether, we found that the process model we

proposed can only be supported to a limited extent by

established tools. A subsequent search on the internet

for cloud audit tools was also of little help in solving

the identiﬁed challenges in the public authority envi-

ronment.

5 PROTOTYPE: VISUALIZING

THE INFORMATION

SECURITY MATURITY LEVEL

OF PUBLIC CLOUDS

Following, we present the basic considerations of our

proposed tool which provides the proposed holistic

process for conducting decentralized security audits

of public cloud services within public administra-

tions. The development cycle took place over several

months since 2022 and, in accordance with the DSR

principles, required a mutual comparison with practi-

cal requirements and the scientiﬁc literature. Our goal

was to achieve a practical solution that is able to sup-

port the mentioned process model in order to deter-

mine the information security maturity level of public

cloud services.

During our research work, we have considered the

relevance and the rigor cycle in accordance with the

DSR principles. On the one hand, we searched the lit-

erature for suitable approaches, and on the other hand,

we always compared them with the actual problem

and discussed them with various actors involved in the

security audit process. During the expert discussions,

we identiﬁed the following essential requirements that

a novel tool approach must provide:

• Self-explanatory Graphical User Interface:

The web-based questionnaire had to be designed

in such a way that no further education and train-

ing of individual actors is required.

• Automatic Single Sign-On: Respondents do not

have to manually log in to our prototype. Instead,

user authentication uses a kerberos mechanism.

This keeps the barriers for answering question-

naires as low as possible.

CLOSER 2024 - 14th International Conference on Cloud Computing and Services Science

196

• Multiple-Choice Answer Options: In order to

be able to carry out the online questionnaires efﬁ-

ciently and in a standardized manner, descriptive

text answers should be avoided as far as possible.

Instead, answers are pre-qualiﬁed so that they can

later be analyzed automatically to determine the

information security maturity level.

• Automatic Storage of Answers: To avoid data

loss during the execution of more comprehensive

audits, the answers given should be stored tem-

porarily. This will signiﬁcantly increase the user

experience and acceptance of our tool during the

entire security audit.

• Delegation of the Questionnaire: Employees

who receive a web-based questionnaire should

themselves be able to forward them to more suit-

able persons. This should avoid unnecessary rout-

ing times.

5.1 Technical Speciﬁcations

Our prototype runs on-premises on a virtualized

Ubuntu Linux server (8 GB RAM, 30 GB disc space)

with access to internal network resources of our au-

thority. Access to this server is only possible from the

internal network for security reasons.

We have used Apache

web server, MySQL

database, as well as Laravel

and Laravel-Livewire

programming frameworks to build our prototype.

The access to the collected data and visualizations

in the tool is limited, depending on the user’s role. We

have currently implemented three roles: CISO, HOU

and CPO. The CISO role is the only one with admin-

istrative rights and full access to the data. Users who

are assigned the HOU or CPO roles only see the data

records released for them for an organizational unit

or for the assigned cloud assets. The answers of the

questionnaires are transmitted in encrypted form. The

database access is restricted so that only administra-

tors and service users can use the raw data.

5.2 Preparation of Cloud Audits

Before our proposed process model can be applied

for cloud security audits, the underlying questionnaire

must ﬁrst be modeled in our prototype. We focus

on the ISO 27001 standard to formulate appropriate

questions about cloud security requirements. In a

previous published research paper (Diener and Bolz,

https://httpd.apache.org

https://mysql.com

https://laravel.com

https://laravel-livewire.com

Figure 3: Predeﬁnition of one sentiment for one of several

answer-options to a single question.

2023), we have already successfully proven the uti-

lization of multiple-choice answers with associated

sentiment levels. Based on the concept of prede-

ﬁned sentiment levels, we can create precise state-

ments about the information security maturity level of

a speciﬁc public cloud service. Each cloud audit re-

lates to a single organizational unit. Thus, it is possi-

ble to identify possible lacks of security requirements

automatically, if answers with negative sentiments are

given by CPOs. In addition, it is possible for CISOs

to predeﬁne suitable security measures that can be au-

tomatically derived in case that negative sentiments

were identiﬁed. Figure 3 shows the question wizard

in our prototype that allows CISOs to qualify prede-

ﬁned multiple choice answer options with a machine

readable sentiment.

By submitting predeﬁned web questionnaires to

the appropriate actors, CISOs can initiate follow-up

security audits in different organizational units with

respect to phases 1 and 2 of our proposed process

model. For example, general questions about adopted

public cloud services are submitted to HOUs of or-

ganizational units. The given answers will be manu-

ally checked by the CISO. Based on the received re-

sults, phase 2 is initiated by the CISO for the identi-

ﬁed public cloud services. Consequently, responsible

CPOs will obtain automatically generated question-

naires with placeholders of public cloud services for

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations

197

which they are responsible. This means that the effort

required to provide individual questionnaires is very

low. At all times, the CISO maintains full control over

all running cloud security audits with the help of our

prototype.

5.3 Managing Security Audits

As the number of concurrent security audits for public

cloud services is growing fast in large organizations,

it becomes increasingly difﬁcult for CISOs to keep

track of all running activities. Although it is possible

to export metadata of audits for data analysis to ex-

ternal tools such as Microsoft Excel, enormous time

efforts will be associated. Moreover, there is no pro-

cessing of real-time data, as audit states can change

very quickly in heterogenous organizations.

Consequently, a graphical visualization of the cur-

rent state of security audits and the corresponding re-

lations between organizational units and public cloud

services is required. To solve this problem, we pro-

pose the usage of a two-dimensional visualization

grid that represents the relationships between orga-

nizational units and their adopted public cloud ser-

vices within a public administration (cf. ﬁgure 4). We

name this component organizational units assets grid

(OAG).

By using this visualization technique, CISOs ob-

tain a realtime overview about the adopted public

cloud services within their organization. Conversely,

it becomes obvious for which public cloud services

and organizational units no relationship is existing

(e.g., red font color). In such cases, our developed

graphical visualization supports CISOs to identify

possible missing correlations at a glance.

Additionally, the background color of single cells

provides information about the status of the intended

security audits. Marked cells in red mean that no audit

has yet been initiated for this relation. In such situa-

tions, it is highly likely that phase 1 of our process

model must be carried out ﬁrst. The orange color in-

dicates that a initiated security audit is still running.

Finally, green colored cells indicate that the audit for a

speciﬁc public cloud service in an organizational unit

has been ﬁnished and no further activities are required

by the involved actors. Marked cells in white signal

that there is no proven relation existing between a sin-

gle organizational unit and a public cloud service.

Moreover, we developed an additional visualiza-

tion grid for our prototype, supporting a simpliﬁed

visual determination of the maturity level of informa-

tion security for the adopted public cloud services.

For this purpose, the answers from the submitted

questionnaires are automatically evaluated. Based

on the predeﬁned sentiment level of each answer, a

graphical visualization is dynamically generated. In

the following section, we explain the concepts of this

novel functionality.

6 EVALUATION

In this section, we present the results of the evalua-

tion of the developed visualization grids that are im-

plemented in our prototype. For the evaluation we

have chosen a real use case from practice in our pub-

lic administration. We focus on auditing four different

public cloud services used by ﬁve different schools in

order to determine the maturity level of information

security.

Four of the ﬁve schools report directly to our Pub-

lic Administration e.g., three professional schools and

one high school. These organizational units must fol-

low the CISO’s instructions, such as participating in

the analysis of the status quo of their individual in-

formation security maturity level. One of the twenty

elementary schools in our city is also participating in

the evaluation but is not subject to instructions from

our Public Administration. This means that we have

selected an appealing and realistic scenario for our

evaluation, which we understand well. The validity

of the information presented in the visualization grid

can thus be checked easily and quickly.

The evaluation of the application of both visual-

ization grids is based on the process model we pre-

sented in section four. In phase 1, we used a web-

based questionnaire in our audit tool to determine

which public cloud services are adopted by the ﬁve

schools. Following the process model, we surveyed

the principals of the schools for phase 1. Responses

were available the next business day from all schools.

Once all responses were received, we were able to

evaluate the OAG visualization that we have devel-

oped.

6.1 OAG Visualization

The OAG provides an overview of the existing re-

lations between the audited organizational units and

their adopted public cloud services (cf. ﬁgure 4). For

a better data presentation in our visualization grid, the

ﬁlter was set to the asset type ”cloud”. By applying

the OAG visualization in our prototype, we were able

to identify the following key facts in phase 1:

• Very quickly, we found that Public Cloud 2 is

stored in our repository but is not used by any of

the schools surveyed. The OAG marks this fact

with a red font color.

CLOSER 2024 - 14th International Conference on Cloud Computing and Services Science

198

Figure 4: Visual managing support of cloud security audits within public administrations.

• Three of the twenty elementary schools and three

other high schools that do not report to our public

administration are already documented in our au-

dit tool. Six schools of the listed organizational

units do currently not have an assignment to a

public cloud service (cf. red font color).

• Immediately it becomes clear that ﬁve secondary

schools are already documented in our audit tool,

for which a relation to Public Cloud 1, Public

Cloud 3 and School Manager is existing. Since no

audit was allowed to be started for these schools

yet, the cells are red colored in each case.

• The OAG visualization shows that six audits are

currently still running (cf. orange cells) and six

audits have already been completed (cf. green

cells) with respect to the questionnaire of phase

Overall, we have found that the OAG visualization

is suitable for monitoring the status quo for several

security audits running in parallel. It is important to

use ﬁlters, otherwise the clarity suffers when different

asset types are displayed.

6.2 Visualizing the Information

Security Maturity Level

Based on phase 1, the subsequent phase 2 focuses

more detailed on ISO 27001 security requirements

for all public cloud services used by different orga-

nizational units (green marked). Therefore, the cloud

product managers (CPO) receives personalized ques-

tionnaires. Since high-quality data has already been

collected in phase 1, a targeted initiation of security

audits is possible. Similar to their superiors (HOUs),

the CPOs receive automatically generated emails with

a link that leads directly to the online questionnaire. A

total of 10 different CPOs received a questionnaire at

the ﬁve schools (cf. ﬁgure 5). The number is so high

because at each school exactly one teacher is respon-

sible for one single public cloud service. One excep-

tion is Primary school 01: the principal is responsible

for managing all cloud services herself.

The most important ﬁnding in the context of this

evaluation was that teachers at the schools were partly

unaware that they themselves were responsible for the

information security measures of the adopted public

cloud services. In some cases, the CPOs thought that

the IT department of the public administration or the

cloud service provider itself is responsible for manag-

ing the information security. In addition, there was a

lack of clear understanding of what activities informa-

tion security encompasses. In this respect, the struc-

tured questionnaire in our audit tool indirectly helped

to raise the awareness of information security activi-

ties among the CPOs involved.

After ﬁve working days, we evaluated the answers

of the submitted questionnaires using our designed se-

curity requirement grid (SRG). The SRG is structured

similarly to the OAG. We will brieﬂy describe the dif-

ferences here. In the SRG, the x-axis represents the

public cloud services used by an organizational unit.

In the ﬁrst line of the grid, these are summarized in

groups. For example, only two public clouds are as-

signed to High school II. The security requirements

checked in each case are listed on the y-axis in the

SRG. These are grouped according to the main re-

quirements. In the cells of the SRG visualization, ﬁve

different color types represent the state of the infor-

mation security maturity level. The colors correlate

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations

199

Figure 5: The SRG visualizes the information security maturity level of different public clouds.

with the sentiment of the associated responses of the

questionnaires. Positive and neutral sentiments are

represented in green color. Answers with negative

sentiment are marked in red. Remarkable sentiment

is visualized in yellow. Unclear answers from respon-

dents are colored orange. Unanswered questions are

shown in white.

In the course of phase 3 of our proposed process

model, we checked the relevance and correctness of

the information presented in the SRG visualization.

Overall, our questionnaire on cloud security consisted

of two groups of topics. Group one focused on the

status quo of identity management and group two on

the authorization concept based on it. In the follow-

ing, we describe the most important key ﬁndings of

the evaluation of the SRG visualization:

• The organizational units (cf. second row) high-

lighted in light green indicate that the associated

security audits have been completed. This level of

information is identical to the OAG visualization.

• Public Cloud 3, which is used in Primary school

1, has a very high maturity level in terms of in-

formation security. Public Cloud 1 used by Pro-

fessional school A has the worst maturity level

for information security. Three security require-

ments obtain a negative sentiment, e.g. the ques-

tion about the last check of user accounts. Of

particular concern in this context is the fact that

the access rights for creating new users are obvi-

ously incorrect. In addition, it is not clear to the

responsible CPO to what extent the permissions

for data export are set correctly. For the CISO

it becomes clear, that there is needed an urgent

improvement in this school with respect to Public

Cloud 1, for example by conducting some security

management trainings.

• Overall, it is noticeable that the questions about

access rights at the vocational schools were fre-

quently provided with unclear answers. This

could indicate that the responsible CPOs are also

insufﬁciently trained on this topic. A direct in-

quiry by the CISO will bring clarity.

• In addition, it is clear that numerous responses

from Professional school A and B as well as High

school II for Public Cloud 1 are currently missing.

Reminder emails could be sent automatically by

our prototype to the respective responsible CPOs

to speed up the audit process.

In total, we have not been able to identify any errors in

the representations of the status quo of the OAG and

SRG visualizations, displaying the information secu-

rity maturity level of a selected organizational units.

At the same time, the CISO gained valuable insights

in the status of the information security maturity level

of the public cloud services, that are used by organi-

zational units within the public administration.

CLOSER 2024 - 14th International Conference on Cloud Computing and Services Science

200

7 DISCUSSION

Based on the aforementioned evaluation results of our

prototype, we initiated a discussion about the ﬁndings

with HOUs and CPOs of our authority. Furthermore,

we discussed the results with ﬁve CISOs from differ-

ent public administrations in order to improve the pro-

posed prototype and to make scientiﬁc contributions

to user experience aspects.

Basically, the proposed tool support is simple

and self-explanatory. It is noticeable that there are

few icons on the GUI, so that CISOs are not over-

whelmed. Comprehensive training is therefore not re-

quired. The advantage is that CISOs can use the tool

to manage all security audits centrally, although the

use of public cloud services is decentralized and car-

ried out independently in the ofﬁces by various CPOs.

The integration of mail information has proven to be

very useful, as the responsible CPOs can be regularly

reminded of open audits using friendly-reminder.

The OAG visualization is useful compared to es-

tablished dashboards in ISMS tools, as it is possible

to see at a glance which cloud department relations

have not yet been audited. This signiﬁcantly im-

proves the level of information security in an author-

ity. It is not necessary to add ﬁlters to traditional re-

ports. In addition, this grid approach visualizes every

possible relationship between department and public

cloud service. However, the grid becomes very un-

wieldy as soon as many departments are entered in a

large authority.

Looking at the SRG visualization, it became clear

that it is very easy to determine the maturity level

of information security, as it is possible to see at a

glance which of the public cloud services used have

serious weaknesses in the security requirements based

on the colored markings. For example, cells marked

red in the SRG show that negative or even critical an-

swers were given by the CPOs to the questions asked.

This is very helpful for CISOs, as there is no need to

read comprehensive reports. Instead, the CISOs can

make direct enquiries to the CPOs. Compared to the

usage of traditional tools, it is necessary to manually

identify each organization-cloud relationship and then

deﬁne the corresponding security requirement ques-

tions. This is not necessary with the tool we presented

and was rated positively by the CISOs interviewed.

In general, it was positively noted that the question

texts can be easily individualized with our prototype

by integrating the product names of individual cloud

services. We use placeholders in the question texts

for this purpose (cf. ﬁgure 3). This increases the re-

spondents’ awareness of the audited context when

they have to deal with the confronting questions. As a

result, CPOs always have a concrete reference to the

currently audited public cloud service during the au-

dit. This is particularly useful if several public cloud

services are used within an organizational unit. This

fact was particularly conﬁrmed from the perspective

of CPOs.

CPOs themselves were also impressed by the fact

that login to the questionnaire is very low-threshold

thanks to SSO authentication via Kerberos ticket.

This eliminates time-consuming registration and

work with the audit questionnaire once or twice a

year.

The fact that the additional collected data from

the questionnaires is stored in a separate database was

seen as a negative factor. As most public administra-

tions already use tools for information security man-

agement, inconsistencies may arise between the two

systems. For this reason, it was suggested that an

API should be developed to enable data exchange

between our prototype and existing security tools.

8 CONCLUSION

In this work, we extended our web-based audit

tool with two visualization grids, supporting CISOs

in public administrations with realtime information

about the maturity level of the information security

with respect to the adopted public cloud services

within different organizational units. The develop-

ment of the visualization grids took place in several

cycles by adapting the DSR process with different in-

ternal and external stakeholders. The evaluation of

the developed OAG and SRG visualizations was done

by investigating a practical example within our public

administration. Additionally, we discussed the evalu-

ation results with CISOs from different authorities in

order to achieve deep feedbacks to our research.

In general, our research has shown that tradi-

tional software products for security audits have

reached their limits when it comes to auditing de-

centralized public cloud services in public admin-

istrations. Innovative and lightweight concepts are

essential in this domain to ensure the information se-

curity of public cloud services in public administra-

tions in the future. In particular, the knowledge carri-

ers in the respective departments must be intensively

involved in both the security processes and the use of

audit tools.

Moreover, we were able to show that it is possi-

ble to implement more efﬁcient and transparent secu-

rity audit processes by using novel visualization tech-

niques. With the help of our enhanced tool, holistic

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations

201

information security processes in public administra-

tions can be improved, so that the management of de-

centralized public cloud services becomes possible.

This research work has found out three key ﬁndings:

1. The proposed OAG visualization provides a sim-

ple overview of the identiﬁed public cloud ser-

vices and existing departments in an organization.

A relation between them is described by using

colored cells. CISOs can easily use this feature

to obtain a quick overview about the status of var-

ious security audits of different assets within an

organization. Traditional reports and dashboards

have the problem that anomalies can only be dis-

played to a limited extent.

2. While designing the procedure for our three-phase

process model, we were able to identify three im-

portant actors for conducting security audits of

public cloud services within public administra-

tions. The CISO himself is the main actor and

needs to be able to coordinate security audits by

using novel visualization techniques. HOUs of or-

ganizational units need be involved immediately

and in an easy way, to obtain meta information

about responsibilities with respect to adopted pub-

lic cloud services. The people actually responsi-

ble for public clouds (= CPOs) must be directly

involved in security audits, as a public adminis-

tration’s IT department is often not responsible for

supporting decentralized public cloud services.

3. In practice, similar public cloud services are used

by different departments within an organization.

Therefore, the relationships between organiza-

tional units and the adopted public cloud services

must be documented at regular intervals. New au-

dits can then be generated on the basis of visual-

ized anomalies (cf. Figure 4). As a consequence

it is possible to derive the maturity level of infor-

mation security with respect to concrete security

requirements (cf. Figure 5) by using visualization

techniques.

In future research we want to achieve further improve-

ments in this research ﬁeld. For example, the problem

of potential shadow IT needs to be investigated. In

this context, we want to make scientiﬁc efforts to train

HOUs to identify public cloud services in their de-

partments. If a new, previously undocumented cloud

has been identiﬁed, the further ISMS process should

be carried out in a lightweight manner with the help

of our tool.

In addition, we want to ﬁnd out how our web-

based audit tool can be combined with security aware-

ness methods. We have noticed during our security

audits that despite previous e-learning trainings on se-

curity awareness, there is still a lack of understand-

ing with regard to security measures within public

cloud services. Moreover, we would like to evalu-

ate the practical applicability of our developed tool

under real conditions in another authority. We want

to understand to what extent visualizations can help

CISOs to improve the maturity level of information

security. For this reason, we will attempt to carry out

a quantitative analysis in collaboration with another

authority over a longer period of time in which met-

rics are examined in order to be able to make state-

ments about the effectiveness of our prototype. How-

ever, this project is associated with major hurdles in

the authorities’ environment.

There is also the problem that CPOs may give in-

correct answers in the questionnaires because they do

not understand the context properly. In this respect,

we need to make further observations to assess the

extent to which qualitatively complete and correct an-

swers to questions are reported back.

Taking everything into account, we are making the

public administration sector a bit more secure, and we

are helping to drive forward the urgently needed im-

plementation of digitization in this environment.

REFERENCES

Anisetti, M., Ardagna, C. A., and Bena, N. (2023). Multi-

dimensional certiﬁcation of modern distributed sys-

tems. IEEE Transactions on Services Computing,

16(3):1999–2012.

Armbrust, M., Fox, A., Grifﬁth, R., Joseph, A. D., Katz,

R. H., Konwinski, A., Lee, G., Patterson, D. A.,

Rabkin, A., Stoica, I., et al. (2009). Above the clouds:

A berkeley view of cloud computing.

Braud, A., Fromentoux, G., Radier, B., and Le Grand, O.

(2021). The road to European digital sovereignty with

Gaia-X and IDSA. IEEE Network, pages 4–5.

Castro, K., Macedo, G. R., Araujo, A. P., and de Carvalho,

L. R. (2019). Cloud. jus: Architecture for provision-

ing infrastructure as a service in the government sec-

tor. In Proc. of the 9th Int. Conf. on Cloud Computing

and Service Science (CLOSER), pages 412–421.

Colantonio, A., Di Pietro, R., Ocello, A., and Verde, N. V.

(2011). Visual role mining: A picture is worth a thou-

sand roles. IEEE Transactions on Knowledge and

Data Engineering, 24(6):1120–1133.

Di Giulio, C., Sprabery, R., Kamhoua, C., Kwiat, K., Camp-

bell, R. H., and Bashir, M. N. (2017). Cloud standards

in comparison: Are new security frameworks improv-

ing cloud security? In Proceedings of the 10th Int.

Conf. on Cloud Computing (CLOUD), pages 50–57.

IEEE.

Diener, M. and Bolz, T. (2023). Cloud inspector: A tool-

based approach for public administrations to establish

information security processes towards public clouds.

CLOSER 2024 - 14th International Conference on Cloud Computing and Services Science

202

In Proc. of the 9th Int. Conf. on Information Systems

Security and Privacy (ICISSP), pages 543–551.

European Union (2018). Regulation (EU) 2018/1724 of the

European Parliament and of the Council of 2 October

2018 establishing a single digital gateway to provide

access to information, to procedures and to assistance

and problem-solving services and amending Regula-

tion (EU) No 1024/2012.

Galletta, A., Ardo, O., Celesti, A., Kissa, P., and Villari, M.

(2017). A recommendation-based approach for cloud

service brokerage: A case study in public administra-

tion. In Proc. of the 3rd Int. Conf. on Collaboration

and Internet Computing (CIC), pages 227–234. IEEE.

Ge, M. and Buhnova, B. (2022). Disda: Digital service

design architecture for smart city ecosystems. In Proc.

of the 12th Int. Conf. on Cloud Computing and Service

Science (CLOSER), pages 207–214.

Henze, M., Matzutt, R., Hiller, J., M

uhmer, E., Ziegeldorf,

J. H., van der Giet, J., and Wehrle, K. (2020). Com-

plying with data handling requirements in cloud stor-

age systems. IEEE Transactions on Cloud Computing,

10(3):1661–1674.

Jaatun, M. G., Tøndel, I. A., Moe, N. B., Cruzes, D. S.,

Bernsmed, K., and Haugset, B. (2017). Accountabil-

ity requirements for the cloud. In Proc. of the 8th Int.

Conf. on Cloud Computing Technology and Science

(CloudCom), pages 375–382. IEEE.

Lange, J. (2024). Kommunaler Notbetrieb: IT-

Sicherheitsvorf

alle in Kommunalverwaltungen.

https://kommunaler-notbetrieb.de.

Lins, S., Schneider, S., Szefer, J., Ibraheem, S., and Sun-

yaev, A. (2019). Designing monitoring systems for

continuous certiﬁcation of cloud services: deriving

meta-requirements and design guidelines. Commu-

nications of the Association for Information Systems,

pages 460–510.

Mazza, R. (2009). Introduction to information visualiza-

tion. Springer Science & Business Media.

Meier, S., Fuchs, L., and Pernul, G. (2013). Managing the

access grid-a process view to minimize insider misuse

risks. In Proc. of the 11th Int. Conf. on Wirtschaftsin-

formatik (WI2013).

Mell, P., Grance, T., et al. (2011). The NIST deﬁnition of

cloud computing.

Nanos, I., Manthou, V., and Androutsou, E. (2019). Cloud

computing adoption decision in e-government. In

Operational Research in the Digital Era–ICT Chal-

lenges: 6th International Symposium and 28th Na-

tional Conference on Operational Research, Thessa-

loniki, Greece, June 2017, pages 125–145. Springer.

Nycz, M. and Polkowski, Z. (2015). Cloud computing in

government units. In Proc. of the 5th Int. Conf. on Ad-

vanced Computing & Communication Technologies,

pages 513–520. IEEE.

Parsons, K., Calic, D., Pattinson, M., Butavicius, M., Mc-

Cormac, A., and Zwaans, T. (2017). The human as-

pects of information security questionnaire (hais-q):

two further validation studies. Computers & Security,

66:40–51.

Peffers, K., Tuunanen, T., Rothenberger, M. A., and Chat-

terjee, S. (2007). A design science research method-

ology for information systems research. Journal of

management information systems, 24(3):45–77.

Proenc¸a, D. and Borbinha, J. (2018). Information secu-

rity management systems-a maturity model based on

iso/iec 27001. In Proc. of the 21st Int. Conf. of

Business Information Systems (BIS), pages 102–114.

Springer.

Rath, M., Keller, L., and Spies, A. (2023). Sovereign

clouds—an overview of the current privacy challenges

associated with the use of us cloud services, and how

sovereign clouds can address these challenges. Com-

puter Law Review International, 24(3):78–84.

Sasubilli, M. K. and Venkateswarlu, R. (2021). Cloud com-

puting security challenges, threats and vulnerabilities.

In Proc. of the 6th Int. Conf. on Inventive Computation

Technologies (ICICT), pages 476–480. IEEE.

Schmitz, C., Schmid, M., Harborth, D., and Pape, S.

(2021). Maturity level assessments of information se-

curity controls: An empirical analysis of practition-

ers assessment capabilities. Computers & Security,

108:102306.

Stephanow, P. and Fallenbeck, N. (2015). Towards contin-

uous certiﬁcation of Infrastructure-as-a-Service using

low-level metrics. In Proc. of the 12th Int. Conf. on

Ubiquitous Intelligence and Computing (UbiComp),

pages 1485–1492. IEEE.

Su, P., Chen, Y., and Lu, M. (2022). Smart city information

processing under internet of things and cloud com-

puting. The Journal of Supercomputing, pages 3676–

3695.

Syynimaa, N. and Viitanen, T. (2018). Is my ofﬁce 365

gdpr compliant?: Security issues in authentication and

administration. In International Conference on En-

terprise Information Systems. SCITEPRESS Science

And Technology Publications.

Yermalovich, P. (2020). Dashboard visualization techniques

in information security. In Proc. of the 7th Int. Sympo-

sium on Networks, Computers and Communications

(ISNCC), pages 1–6.

Zaharia-R

adulescu, A.-M., Radu, I., et al. (2017). Cloud

computing and public administration: approaches in

several european countries. In Proc. of the Int. Conf.

on Business Excellence, volume 11, pages 739–749.

Sciendo.

Visualizing the Information Security Maturity Level of Public Cloud Services Used by Public Administrations

203