Jesus Luna, Hamza Ghani, Daniel Germanus and Neeraj Suri
Department of Computer Science, Technische Universit¨at Darmstadt, Hochschulstr. 10, 64289 Darmstadt, Germany
Cloud dependability, Cloud security, Security compliance, Security measurements, Security metrics.
Cloud computing is redefining the on-demand usage of remotely-located, and highly available computing
resources to the user. Unfortunately, while the many economic and technological advantages are apparent, the
migration of key sector applications to the Cloud has been limited due to a major show-stopper: the paucity of
quantifiable metrics to evaluate the tradeoffs (features, problems and the economics) of security. Despite the
obvious value ofmetrics in different scenarios to evaluate such tradeoffs, a formal and standard-based approach
for the addressing of security metrics in the Cloud is a much harder and very much an open issue. This paper
presents our views on the importance and challenges for developing a security metrics framework for the
Cloud, also taking into account our ongoing research with organizations like the Cloud Security Alliance and
European projects like ABC4Trust, CoMiFin and INSPIRE. This paper also introduces the basic building
blocks of a proposed security metrics framework for elements such as a Cloud provider’s security assessment,
taking into account the different service and deployment models of the Cloud.
The Cloud just as defined in (Mell and Grance, 2009),
has increasingly become a computing/communication
paradigm that seems to have the potential to change
the way we consider systems and services. Thanks
to the rapid provisioning of computational resources
taking place in the Cloud with minimal management
effort or service provider interaction, now we are
forced to rethink about the core Information Techno-
logy (IT) elements of data.
For Small and Medium Enterprises SMEs
and sectors like eHealth and eGoverment the advan-
tages of using the Cloud are clear, unfortunately as
also highlighted by ENISA — the European Network
and Information Security Agency (ENISA, 2011)
in their report (Catteddu and Hogben, 2009) the Cloud
also conveys serious security and privacy issues that
nowadays represent major “show-stoppers” for its
The importance of creating secure and trusted
Cloud services has resulted in a central question: how
to objectively and quantitatively measure the security
of a Cloud service provider?
In other IT ecosystems, (e.g. critical infrastruc-
tures) well designed security metrics have provenuse-
ful not only in helping formally understand the secu-
rity guarantees provided by a system, but also raising
awareness about its vulnerabilities and even assessing
the effectiveness of the different security mechanisms
being implemented. Unfortunately due to the Cloud’s
special characteristics, at the state of the art, there are
just a few efforts aimed at using a framework or com-
mon set of objectives and, quantitative security met-
rics for the Cloud.
The main contributions of this paper are (i) a
scenario-driven approach to obtain a set of common
requirements for designing Cloud security metrics
(Section 2), (ii) an analysis of the state of the art re-
lated with the use of security metrics in the Cloud
(Section 3) and, (iii) a presentation of our initial
research results aimed to create a security metrics
framework as an essential milestone required to build
trust in Cloud environments (Section 4). Finally in
Section 5 this paper presents our conclusions and fu-
ture work.
In this section we motivate the creation of a frame-
work for Cloud security metrics, by presenting some
scenarios where such basic building blocks are re-
quired in order to deploy the full potential of Cloud
computing whilst guaranteeing its security.
Luna J., Ghani H., Germanus D. and Suri N..
DOI: 10.5220/0003446902450250
In Proceedings of the International Conference on Security and Cryptography (SECRYPT-2011), pages 245-250
ISBN: 978-989-8425-71-3
2011 SCITEPRESS (Science and Technology Publications, Lda.)
2.1 Cloud Security Metrics Scenarios
The four scenarios presented next have been inspired
from our ongoing collaboration with the Cloud Secu-
rity Alliance — CSA (CSA, 2011).
1. Security Compliance and Dependability: The
term compliance is closely related to the no-
tion of measurements and metrics. For Cloud
providers security compliance can become dif-
ficult to demonstrate. From our perspective, a
well designed security metric should allow Cloud
providers to quantify and objectively demonstrate
their security compliance with some specific set
of requirements (Travis and Annie, 2008) for ex-
ample a Digital Forensic’s readiness policy (Tan,
2. Cloud Federations: The current proliferation and
diversity of Cloud providers has resulted in the
idea of creating Cloud Federations” (Rochw-
erger et al., 2010)), where users might be able
to compose complex workflows by combining the
capabilities of different providers while avoid-
ing dependency on one particular vendor (lock-in
risk as defined by ENISA (Catteddu and Hogben,
2009)). Just as in computational Grid environ-
ments, the creation of Cloud federations depends
on the correct use of objective security metrics
((IGTF, 2011) and (Casola et al., 2010)).
3. Dark Clouds: The infinite availability of computa-
tional resources provided by the Cloud has caught
the attention of a wide range of cybercriminals
willing to use it for their purposes (Antonopou-
los, 2011) and (Samson, 2011)) Our belief is that
quantitative,run-time security metrics can be used
by Cloud providers in order to build architectures
able to monitor and detect potential abuses or,
cyberattacks targeting or even originating inside
their systems.
2.2 Summary of Requirements
From the analysis of different scenarios presented in
Subsection 2.1, the research introduced in this paper
proposes to classify their requirements in three differ-
ent classes:
1. Taxonomies: There is a need for a taxonomy or hi-
erarchical classification, of the different elements
that model the security behavior of the Cloud ser-
vice. Taxonomies are the first step in designing
flexible and interoperable security metrics (Sed-
digh et al., 2004).
2. Metrics: This is the set of security metrics devel-
oped from the proposed taxonomy. The presented
scenarios require comprehensive (from the Cloud
service-level to underlying algorithms), quantita-
tive and objective metrics.
3. Reference architectures: The basic building
blocks required to implement and deploy the pro-
posed set of security metrics. Monitoring the ful-
fillment of an expected security level can be inte-
grated as a functionality of the proposed architec-
This common set of requirements is being used to
propose the Cloud security metrics framework intro-
duced in Section 4.
Next, we survey and analyze the state of the art re-
lated with Cloud security metrics, mapped to the three
groups of requirements proposed in Section 2.
3.1 Taxonomies
One of the first taxonomies tackling Cloud security
can be found on ENISAs report (Trimintzios, 2011),
where a risk-driven approach is proposed by the au-
thors. This taxonomy focuses on risks-based con-
siderations and associates qualitative scores to them,
moreover it also introduces a set of vulnerabilities
and affected assets that can be used to develop spe-
cific metrics for the Cloud. The work of (Grobauer
and Walloschek, 2010) is complementary to ENISAs
report, where the authors further elaborate about the
need for measuring a Cloud provider’s security level
trough a vulnerability-based approach. Their major
contribution is an overviewof Cloud-specific vulnera-
bilities, that can be further organized into a taxonomy
for Infrastructure as a Service models (IaaS).
Based on his previous research on security met-
rics taxonomies, Savola (Savola et al., 2010) uses a
threat-based approach to propose a high level taxon-
omy and associated metrics for measuring the Cloud’s
security, privacy and trustworthiness. The proposed
taxonomy contributes to the state of the art with the
inclusion of a new taxonomy class focused on the
Cloud’s privacy features. The Cloud Security Al-
liance’s (CSA) Common Assurance Maturity Model
(CAMM) and Cloud Controls Matrix Work Group,
are the leading initiatives of industrial Cloud secu-
rity metrics research. CAMM (CAMM, 2010) is an
ongoing industrial project that aims to create a frame-
work to attest the information assurance maturity of
a Cloud provider. In order to fulfill its goal CAMM
proposes a set of controls based on ENISAs taxon-
omy (Catteddu et al., 2009), the Cloud Control Matrix
SECRYPT 2011 - International Conference on Security and Cryptography
from the CSA (CCM, 2011), and existing standards
such as ISO 27001 (ISO27001, 2005). CAMM is an
ongoing initiative that has not proposed any new high-
level taxonomy or metrics so far. The CSA also pro-
motes the Cloud Controls Matrix (CSA CCM (CCM,
2011)), which is based on (Brunette et al., 2009) and
proposes a set of questions providing fundamental
security requirements to guide Cloud vendors, and
Cloud customers in assessing the overall security risk
of a Cloud provider. The CSA CCM seeks to create
both, a Cloud security metrics taxonomy and a set of
associated security measures. The CCM taxonomy is
derived from (Brunette et al., 2009), and despite its
usefulness it turns out to be challenging with regard
to the derivation of quantitative and objective metrics
from it.
Despite not being focused on the Cloud, there are
two security taxonomies worth to mention due to their
broad community use: the National Institute of Secu-
rity and Standards’ (NIST) taxonomy (Chew et al.,
2008), and the one contributed by the Center for In-
ternet Security (CIS) in (Center for Internet Security,
2010)). Both are quite similar about defined cate-
gories and proposed set of metric definitions. Due
to their flexibility, our belief is that the metrics pro-
posed in both documents can be also applied to the
Cloud via taxonomies like e.g. the one from ENISA
(Trimintzios, 2011).
3.2 Metrics
One of the few works focused on quantitatively eval-
uating the security of a pure” IaaS Cloud has been
presented in (Arshad et al., 2010), where the authors
introduce the idea of integrating security metrics into
an IaaS scheduler. Unfortunately, no further details
are given about the architecture or policies used by
the proposed security evaluation system. The Com-
mon Assurance Maturity Model (CAMM) (CAMM,
2010) explores metrics and measurements by propos-
ing to quantify the level of assessment required to
achievegreater confidence. CAMM considers two ba-
sic principles: (i) objective metrics can be used to ob-
tain scores, and (ii) scores from different components
that can be composed to model the security level of a
Cloud provider (Hogben, 2011). At the time of writ-
ing this paper, CAMM has not released further infor-
mation about the proposed metrics. The CSA Met-
rics Work Group complements the CSA CCM (CCM,
2011), by developing the security metrics needed to
evaluate CCM’s requirements. The CSA Metrics WG
has created a template that characterizes each metric
with attributes, and also has proposed their first 10
metrics covering approximately 25 of CCM’s control
areas. From our perspective this is a useful work in
progress, but that still needs to be complemented with
the formal models in order to achieve required fea-
tures like the composability of two or more metrics.
Our research group is collaborating with CSA Met-
rics Work Group in order to achieve these goals.
In (Catteddu et al., 2011) ENISA analyzes the
risks associated with the use of Cloud computing for
eGoverment. This report proposes a set of security
and resilience parameters that can be evaluated in
order to compare different Cloud service providers.
The proposed parameters are divided into high-level
categories (preparedness, service delivery, response
and recovery and, legal and regulatory compliance),
but unfortunately some of these are qualitative (e.g.
tolerance to malicious attacks). It is also worth to
mention the security metrics contributions made by
(Wang, 2005), NIST (Chew et al., 2008) and CIS
(Center for Internet Security, 2010) in particular with
the definition of a flexible metric “template” that al-
lows for creating more specific metrics that are ob-
jective and quantitative. A missing point with these
metrics (apart from not having a focus on Clouds), is
the lack of a set of rules or “algebra” that allows to
model complex Cloud services (e.g. Federations).
3.3 Reference Architectures
Reference architectures and technologies enabling the
use of security metrics in the Cloud are still on a very
early stage, however the most representative effort is
the CloudAudit API (CloudAudit, 2011), that aims to
give more “transparency” to Cloud providers by cre-
ating a common interface and namespace that allows
them to automate the audit, assertion, assessment, and
assurance of their environments. The CloudAudit API
can be used to automatically retrieve and transport
attributes from the provider, therefore enabling cus-
tomers to perform on-the-fly security measurements.
CloudAudit will be an essential piece of the frame-
work proposed by our research, because it has been
designed in such a way that it can be used with new
taxonomies and metrics.
For the ongoing research presented in this paper,
it is also worth to mention three EU-funded projects
that are developingreference architectures that use se-
curity metrics in order to improve the security, pri-
vacy and resilience of IT infrastructures. The first
project is INSPIRE ((D’Antonio et al., 2008), (IN-
SPIRE, 2011)) an EC funded research project whose
name stands for “INcreasing Security and Protection
through Infrastructure REsilience”, with a focus on
Supervisory Control and Data Acquisition (SCADA)
systems. Within the INSPIRE project, an overlay ap-
proach was taken to propose an architecture to mon-
itor and react to perturbations in the communication
layer of the SCADA network. Secondly, the CoMiFin
Communication Middleware for Monitoring Fi-
nancial Critical Infrastructure (CoMiFin, 2011)
project takes an approach similar to INSPIRE and
provides an overlay architecture for financial institu-
tions for sharing security relevant information such
as alerts about cyberattacks and other threats. The
CoMiFin middleware is capable of collaborative cy-
berattack detection stemming from patterns that a sin-
gle financial institution is unable to monitor. A met-
rics monitoring framework has been developed within
this project (Ghani et al., 2010) in order to calculate
the security metrics and monitor compliance with se-
curity requirements. Finally, it is worth to mention
the recently started project ABC4Trust (ABC4Trust,
2011), which aims (among other goals) to estab-
lish a comparison framework and associated architec-
ture for the so-called anonymous credentials (Chaum,
1985). We hope that the security metrics architecture
to be developed in ABC4Trust, can also be applied to
Cloud services because of the approach being taken
(service-level metrics, technology-neutral).
3.4 Summary of Research Challenges
The state of the art presented in this section contains
a common set of research challenges, to be taken into
account for developing the proposed security metrics
1. Taxonomies: It is necessary to “adapt” well-
known taxonomies like the one from CIS (Center
for Internet Security, 2010) to model the Cloud’s
unique features. The taxonomy should be flexible
enough (represent the Cloud’s security, privacy or
risk) and able to cope with the Cloud’s service ori-
ented nature.
2. Metrics: There is the need for researching formal
models and algebras for using quantitative met-
rics. Also both, the creation of pragmatical mea-
surement methodologies and the use of predic-
tion capabilities based on historical data, should
be further explored.
3. Reference architectures: Research should focus
on proposing non-intrusive, scalable, interopera-
ble and comprehensive security metric architec-
tures (from services to algorithms). These should
support the automatic monitoring of static and
dynamic measurements, while considering the
Cloud provider’s “opacity”.
Taking into account the security requirements and
challenges from Sections 2 and 3, this section intro-
duces our ongoing research towards creating a secu-
rity metrics framework for the Cloud. The proposed
framework is composed of the three building blocks
introduced in Section 2, and shown in Figure 1, and
takes as a starting point the Cloud’s service oriented
perspective also known as the SPI model
, in order to
have a multi-layer, comprehensive metric that consid-
ers interfaces, network infrastructures, and algorithms
at the further end. Depending on the scenario require-
ments it might be possible to adopt a taxonomy fo-
cused on either security, privacy or risks associated
with the Cloud service. A security-oriented taxonomy
might be useful for compliance scenarios (where a
baseline security level exists), whereas a risk-oriented
one could be more suitable for the dark-cloud sce-
nario presented in Section 2.
Figure 1: Basic building blocks of the proposed security
metrics framework for the Cloud.
The metrics used by the proposed framework
should be objective and quantitative to promote au-
tomatization and composition to model more com-
plex services. Chosen metrics should be able to give
an answer to security provisions like “What is the
SPI stands for Software as a Service — SaaS —, Plat-
form as a Service PaaS — and Infrastructure as a Service
— IaaS —
SECRYPT 2011 - International Conference on Security and Cryptography
security level of the authentication mechanism used
by the IaaS’ management interface?”. For security-
oriented taxonomies our current research is taking an
approach like the one used by the Reference Evalu-
ation Methodology (REM, (Casola et al., 2007) and
(Casola et al., 2005)), so it might be possible to for-
mally compose the security levels of different Cloud
services. In previous works we have used this ap-
proach for quantitatively measuring the security of
complex Grid infrastructures i.e. in (Luna et al., 2008)
and (Luna et al., 2010), nevertheless we are also con-
sidering other formal methodologies like the one pro-
posed in (Schryen et al., 2011). The final building
block of the proposed taxonomy is the reference ar-
chitecture comprehending measurement techniques,
monitoring network and reporting mechanisms. At
this early stage of our research, the reference archi-
tecture is being planned as an non-intrusive overlay
network just as the one we proposed in (Ghani et al.,
2010). Despite its final shape, the reference archi-
tecture should be able to integrate a set of monitors
able to alert if the Security Level Agreement is vio-
lated. In the architecture, the process of reporting to
external entities (like Third Party Auditors) may be
realized via mechanisms like CloudAudit (Section 3),
which are flexible enough to represent risks, security
or privacy measurements depending on the taxonomy
that was chosen.
In this position paper we have presented several sce-
narios to introduce our views on the importance of
creating a security metrics framework for the Cloud.
To contribute towards the development of such a
framework, we analyzed the features and challenges
of relevant related work in this area. This paper also
introduced the initial research results of our proposed
security metrics framework for the Cloud, which aims
to improve tasks like compliance evaluation or de-
pendability assessment. Our goal is to create an open,
flexible and technology-agnostic framework able to
be extended through the integration of new security
metrics that might be developed for specific scenar-
We have identified a set of research challenges re-
lated with the formal aspects of Cloud security met-
rics that will be part of our future work, in particu-
lar the composition of different security parameters
which allows for the computation of an overall secu-
rity level (like e.g. in (Casola et al., 2007)). Further-
more, the reference architecture proposed in this posi-
tion paper will explore some of our experiences with
projects like INSPIRE, CoMiFin and ABC4Trust (re-
viewed in Section 3) in order to develop an architec-
ture able to enforce security level agreements in the
In order to obtain community feedback about the
proposed framework, we have begun to collaborate
with groups like the CSAs Security Metrics WG. The
resulting framework should be able to model existing
use cases, like the governmental Clouds analyzed in
(Catteddu et al., 2011).
Finally, as a proof of concept study we are plan-
ning to develop an architecture that integrates the
proposed framework into a Cloud Federation’s data
storage broker, in order to perform data allocation
based on the evaluation of a predefined Security Level
Agreement. We hope that this work will aid to
show the tradeoffs between security and performance,
therefore supporting the decision making process by
providing a metrology basis for the quantitative as-
sessment of different security attributes.
Research supported in part by EC FP7 IP
ABC4Trust (2011). ABC4Trust FP7. Online:
Antonopoulos, A. (2011). Dark cloud computing. Online:
Arshad, J., Townend, P., and Xu, J. (2010). Quantifi-
cation of Security for Compute Intensive Workloads
in Clouds. In Parallel and Distributed Systems (IC-
PADS), 2009 15th International Conference on, pages
479–486. IEEE.
Brunette, G., Mogull, R., et al. (2009). Security Guid-
ance for Critical Areas of Focus in Cloud Com-
puting V2. 1. CSA (Cloud Security Alliance),
USA. Online: http://www. cloudsecurityalliance.
org/guidance/csaguide. v2, 1.
CAMM (2010). Common Assurance Maturity Model. On-
line: http://common-assurance.com/.
Casola, V., Luna, J., Manso, O., Mazzocca, N., Medina,
M., and Rak, M. (2007). Interoperable grid pkis
among untrusted domains: An architectural proposal.
In C´erin, C. and Li, K., editors, GPC, volume 4459
of Lecture Notes in Computer Science, pages 39–51.
Casola, V., Preziosi, R., Rak, M., and Troiano, L. (2005). A
Reference Model for Security Level Evaluation: Pol-
icy and Fuzzy Techniques. J. UCS, 11(1):150–174.
Casola, V., Rak, M., and Villano, U. (2010). Identity Feder-
ation in Cloud Computing. In Sixth International Con-
ference on Information Assurance and Security (IAS),
pages 253–259. IEEE.
Catteddu, D. et al. (2011). Security & Resilience in Govern-
mental Clouds. European Network and Information
Security Agency (ENISA).
Catteddu, D. and Hogben, G. (2009). Cloud Computing
Risk Assessment. European Network and Information
Security Agency (ENISA).
Catteddu, D., Hogben, G., et al. (2009). Cloud Computing
Information Assurance Framework. European Net-
work and Information Security Agency (ENISA).
CCM (2011). Cloud Control Matrix. Online:
Center for Internet Security (2010). The CIS security met-
rics. Technical Report 28, Center for Internet Security.
Chaum, D. (1985). Security without identification, card
computers to make big brother obsolete. Original
Version appeared in: Communications of the ACM,
Chew, E., Swanson, M., Stine, K., Bartol, N., Brown, A.,
and Robinson, W. (2008). Performance measurement
guide for information security. Technical Report July,
National Institute of Standards and Technology.
CloudAudit (2011). CloudAudit. Online:
CoMiFin (2011). Communication Middleware for Mon-
itoring Financial Critical Infrastructure. Online:
CSA (2011). Cloud Security Alliance. Online:
D’Antonio, S., Romano, L., Khelil, A., and Suri, N. (2008).
INcreasing Security and Protection through Infras-
tructure REsilience: the INSPIRE Project. In Pro-
ceedings of The 3rd International Workshop on Criti-
cal Information Infrastructures Security (CRITIS’08).
ENISA (2011). European Network and Information Secu-
rity Agency. Online: http://www.enisa.europa.eu.
Ghani, H., Khelil, A., Suri, N., Csertn, G., Gnczy, L., Ur-
banics, G., and Clarke, J. (2010). Assessing the Secu-
rity of Internet Connected Critical Infrastructures (The
CoMiFin Project Approach). In Proceedings of the
Workshop on Security of the Internet of Things (Se-
cIoT 2010).
Grobauer, B. and Walloschek, T. (2010). Understanding
cloud-computing vulnerabilities. IEEE Security and
Privacy, pages 1–14.
Hogben, G. (2011). ENISA Cloud Computing Strat-
egy. Online: http://www.terena.org/activities/tf-
IGTF (2011). The International Grid Trust Federation. On-
line: http://www.igtf.net/.
INSPIRE (2011). INcreasing Security and Protec-
tion through Infrastructure REsilience. Online:
ISO27001 (2005). Information Security Man-
agement System (ISMS) standard. Online:
Luna, J., Dikaiakos, M. D., Marazakis, M., and Kyprianou,
T. (2010). Data-centric privacy protocol for intensive
care grids. IEEE Transactions on Information Tech-
nology in Biomedicine, 14(6):1327–1337.
Luna, J., Flouris, M., Marazakis, M., and Bilas, A. (2008).
Providing security to the Desktop Data Grid. pages
Mell, P. and Grance, T. (2009). The NIST Definition of
Cloud Computing. National Institute of Standards and
Technology (NIST).
Rochwerger, B., Breitgand, D., Levy, E., Galis, A., Nagin,
K., Llorente, I., Montero, R., Wolfsthal, Y., Elmroth,
E., and Caceres, J. (2010). The Reservoir Model and
Architecture for Open Federated Cloud Computing.
IBM Journal of Research and Development, 53(4):4.
Samson, T. (2011). Amazon EC2 Enables
Brute-force Attacks on the Cheap. Online:
Savola, R., Juhola, A., and Uusitalo, I. (2010). Towards
Wider Cloud Service Applicability by Security, Pri-
vacy and Trust Measurements. In 4th International
Conference on Application of Information and Com-
munication Technologies (AICT), pages 1–6. IEEE.
Schryen, G., Volkamer, M., Ries, S., and Habib, S. (2011).
A formal approach towards measuring trust in dis-
tributed systems. In ACM Symp. on Applied Comput-
ing, pages 1739–1745.
Seddigh, N., Pieda, P., Matrawy, A., Nandy, B., Lam-
badaris, J., and Hatfield, A. (2004). Current trends and
advances in information assurance metrics. In Pro-
ceeding of the Second Annual Conference on Privacy,
Security and Trust, pages 197–205.
Tan, J. (2001). Forensic Readiness. Technical report,
@Stake Organization.
Travis, D. and Annie, I. (2008). Analyzing Regulatory
Rules for Privacy and Security Requirements. IEEE
Trans. Software Eng., 34(1):5–20.
Trimintzios, P. (2011). Survey on Resilience Metrics.
European Network and Information Security Agency
Wang, J. (2005). Information Security Models and Met-
rics. In Guimar˜aes, M., editor, ACM Southeast Re-
gional Conference, volume 2, pages 178–184. ACM.
SECRYPT 2011 - International Conference on Security and Cryptography