A COMPARISON OF SECURITY

SAFEGUARD SELECTION METHODS

Thomas Neubauer

Secure Business Austria, Favoritenstrasse 16, 1040 Vienna, Austria

Keywords:

Security, Risk Management, Economics, Decision Support.

Abstract:

IT security incidents pose a major threat to the efﬁcient execution of corporate strategies and business pro-

cesses. Although companies generally spend a lot of money on security companies are often not aware of

their spending on security and even more important if these investments into security are effective. This paper

provides decision makers with an overview of decision support techniques, describes pros and cons of these

methodologies.

1 INTRODUCTION

Companies are often not aware of their spending on

security and even more important if the investments

into security are effective. The deﬁnition of security

safeguards is often a result of current needs or inﬂu-

enced by security problems that may go public. When

seeking to select the most appropriate set of measures

and, thus, the right level of security investments, de-

cision makers are challenged by (i) having to concen-

trate resources on value-generating and supplemen-

tary business processes while (ii) having to consider

multiple strategic objectives that are often conﬂicting

as well as (iii) the cost-efﬁcient usage of the available

resources and interdependencies between the systems

and (iv) a variety of potential technologies and po-

tential systems. As a consequence security decisions

provide only punctual solutions and are made without

considering the costs and beneﬁts of introducing the-

ses measures. Accordingly, a variety of approaches

have been introduced that aim to support decision

makers in identifying the “right” investment candi-

dates. This paper provides decision makers with an

overview of common methods for the evaluation and

selection of security safeguards and describes pros

and cons of these methodologies. One major focus of

this evaluation lies on identifying the method’s capa-

bilities in (i) considering business processes for align-

ing expenditure to actual business needs and (ii) inte-

grating multiple objectives in order to properly con-

sider ﬁnancial, technical and/or further types of ob-

jectives. Thereby this paper supports decision makers

in their decision which methods to choose when hav-

ing to evaluate security investments.

2 COMPARISON

The Analytic Hierarchy Process (AHP) developed by

Saaty (Saaty, 1980) is a tool for solving multicriteria

decision making problems and is based on the prin-

ciples of hierarchy, pairwise comparison, and weight

synthesizing for prioritizing criteria and the evalua-

tion of alternatives. Speciﬁcally, the process consists

of the following steps: Structuring a Hierarchy, Pri-

oritizing the Criteria, Evaluating the Alternatives, and

Calculating the Global Priorities. The Analytic Hier-

archy Process (AHP) is a widespread and easy to use

decision support tool for evaluating different alterna-

tives that can be applied to solve security safeguard

selection problems. Its strength is the analysis of the

alternatives’ properties in different categories, or in

other words, the evaluation of alternatives with re-

spect to multiple objectives. The pairwise comparison

technique however requires (i) the direct comparabil-

ity of alternatives and (ii) may result in a major ef-

fort. As security safeguards are not limited to techni-

cal solutions but also include organizational measures

and operational procedures, comparing them directly

may be problematic, e.g., comparing a packet ﬁlter-

ing ﬁrewall with a ﬁre extinguisher. Therefore, it may

be beneﬁcial not to use the AHP as a standalone tool

for solving information security-related problems, but

320

Neubauer T. (2009).

A COMPARISON OF SECURITY SAFEGUARD SELECTION METHODS.

In Proceedings of the 11th International Conference on Enterprise Information Systems - Information Systems Analysis and Speciﬁcation, pages

320-323

DOI: 10.5220/0001860503200323

 SciTePress

to integrate it with other methods. Regarding the ef-

fort, Maiden (Maiden and Ncube, 1998) illustrates

this fact by means of a case study of a project with

about 130 requirements: because it would have re-

quired an estimated 42,000+ individual paired com-

parison scores, applying AHP was impossible in this

and similar cases due to the time constraint involved.

Another framework that takes multiple objectives

into consideration when determining suitable secu-

rity safeguards is the cost/beneﬁt-based Security At-

tribute Evaluation Method (SAEM) (Butler, 2002). As

the risk assessment process is tasked with prioritiz-

ing threats, a beneﬁt assessment determines the safe-

guard effectiveness, and a cost analysis determines

the expenses associated with the security measures.

The SAEM process involves the following four steps:

Risk Assessment, Beneﬁt Analysis, Coverage Analy-

sis, and Security Technology Tradeoff Analysis. Un-

like AHP, it was developed speciﬁcally for solving in-

formation security evaluation problems and therefore

particularly addresses security-related concepts such

as threats and safeguard effectiveness. Multiobjectiv-

ity is considered in the multiattribute risk assessment

where threats are ranked according to their likeli-

hood of occurrence and impact on attack outcome at-

tributes. The resulting threat index values and the ef-

fectiveness values of the security technologies under

consideration are then used to calculate their risk re-

duction impact. The coverage analysis ensures that no

security gap is overlooked when arranging the safe-

guard portfolio, and the tradeoff analysis compares

the risk reduction impact and other beneﬁts of secu-

rity measures with their costs like implementation or

maintenance costs. The SAEM approach is a very de-

tailed and structured process to evaluate information

security and safeguards. The risk assessment pro-

cess ensures that speciﬁc threats are addressed, and

the beneﬁt and tradeoff analysis consider the multiob-

jectivity of threat consequences and safeguard effec-

tiveness. The normalization of threat and safeguard

values also allows the concurrent usage of qualitative

and quantitative data. But, as the SAEM method is

quite detailed and extensive, it is also rather complex

to conduct. Each phase requires relatively much work

and, without automation of certain steps, this work

can be quite tedious. Although this method allows the

deﬁnition of multiple objectives, the outcome is still

aggregated into a single scalar value used for the eval-

uation, i.e., the threats and safeguards cannot be eval-

uated subject to the attributes ’independently’. Also,

SAEM does not consider the business processes and

the safeguards’ inﬂuence on them.

The Central Computer and Telecommunications

Agency (CCTA) Risk Analysis and Management

Method (CRAMM) (InsightConsulting, 2007) is a

commercial qualitative risk analysis methodology de-

veloped by the UK government’s Central Computer

and Telecommunications Agency in full compliance

with BS7799. CRAMM is divided into the three

stages Asset Identiﬁcation and Valuation, Threat and

Vulnerability Assessment, and Countermeasure Se-

lection and Recommendation. The CRAMM pro-

cess includes the following steps: Assets, Threats,

Vulnerabilities, Risks, Countermeasures, Implemen-

tation, and Audit. The frameworkis intended for large

governmental and commercial organizations. It pro-

vides a structural method to identify relevant assets

(arranged into asset groups), possible vulnerabilities,

and threats, to combine them to risks that are mea-

sured according to the assets’ values and vulnerability

and threat levels, and to recommend suitable counter-

measures. The process is considered as rather com-

plex and considerable experience is required in order

to produce meaningful and correct results. There-

fore, organizations often rely on external qualiﬁed

CRAMM practitioners to conduct the analysis in-

stead of letting internal analysts undergo the exten-

sive training to gain the necessary expertise. This re-

duces the organization’s internal staff’s involvement

in the assessment phase and therefore also does not

improve their insight into security matters. The out-

come is often quite extensive and a full review may

last up to several months. The framework neither of-

fers the ﬂexibility to customize it to the organizations

characteristics, nor does it provide an evaluation of

business process related issues. Grouping the assets

into asset groups may also prove to be unfavorable,

as all assets have their own properties and security re-

quirements. And ﬁnally, CRAMM does not calculate

economic indicators, such as implementation costs of

the recommended safeguards and whether they ﬁt into

the security budget.

Operationally Critical Threat, Asset, and Vulner-

ability Evaluation (OCTAVE) (CERT, 2007) is a risk-

based strategic assessment and planning technique for

security developed by the Carnegie Mellon Univer-

sity. Essentially, the OCTAVE framework is a set

of criteria containing guidelines and requirements for

implementing process steps, instead of pre-speciﬁed

techniques. These criteria must be fulﬁlled in or-

der to correctly implement the OCTAVE framework.

Unlike other frameworks, the OCTAVE approach fo-

cuses on organizational risks instead of technological

ones and is structured into three phases: Build Asset-

based Threat Proﬁles, Identify Infrastructure Vulner-

abilities, and Develop Security Strategy and Plans.

OCTAVE is quite different to CRAMM: On the one

hand, the actual OCTAVE framework does not pro-

A COMPARISON OF SECURITY SAFEGUARD SELECTION METHODS

321

vide a step by step procedure as CRAMM, but a set

of criteria that has to be met to conform to the OC-

TAVE methodology. This provides great adaptability

and more ﬂexibility than CRAMM to address speciﬁc

organizational needs. For easier access, three speciﬁc

application methods have been developed to choose

from, suited for different sizes of organizations. On

the other hand, it is self-directed, meaning that the

OCTAVE methodology has to be wholly exercisable

by the organization’s internal staff in a workshop en-

vironment, thus not relying on external experts. Fi-

nally, OCTAVE is focused on organizational risks,

whereas CRAMM concentrates more on technical is-

sues, and OCTAVE generally does not take threat

likelihood into consideration (except for OCTAVE S

which provides basic means for including threat like-

lihood in the evaluation).

While the risk assessment methodologies evaluate

security technologies on their effectiveness to miti-

gate risks, the POSeM framework deals with informa-

tion security in a different way. The Process-Oriented

Security Model (POSeM) framework developed at the

University of Z¨ı¿

rich by R¨ı¿

hrig (R¨ohrig, 2002) is

a methodology to deﬁne security requirements and to

derive security measures by using process models as

the basis for the analysis. In this proposal, the four

security objectives conﬁdentiality, integrity, availabil-

ity, and accountability are used to measure the secu-

rity levels of each process component (actor, artifact,

activity), and suitable security measures are derived

via rule bases. It takes into consideration that as-

sets do not generate business value themselves, but

participate in business processes that produce utility,

and therefore the methodologies rely on business pro-

cesses as the basis for their analysis. It also ignores

speciﬁc harmful events (e.g., threats), but concentrate

on eliciting security requirements and deriving appro-

priate security measures. The POSeM approach con-

sists of ﬁve steps: Deﬁnition of General Security Ob-

jectives, SEPL Model, Consistency Analysis, Deriva-

tion of Generic Security Measures, and an optional

Implementation Phase. The strengths of the POSeM

approach lie in the usage of business process mod-

els as the basis for the security evaluation and the

deﬁnition of organization speciﬁc rule sets for the

consistency checks and safeguard derivation. Using

process models seems to be a logical step in today’s

process-centered world. As processes are continually

improved (or completely restructured with BPR tech-

niques), the security status should be evaluated and

improved in line with the business processes. Relying

on the well established CIA properties also enhances

the insight into security-related matters of processes,

especially by assigning them to the individual process

components (participant, data, activity). By deﬁning

organization-speciﬁc rules, the particularities of the

organization and its main processes can be taken into

consideration, thus reaching a high level of adaptabil-

ity of the POSeM process. These rules can be spec-

iﬁed once and stored for further uses, which signiﬁ-

cantly reduces the amount of time needed for the (re-

)evaluation. And the formal description methods of

SPEL, SMDL, and SCRL allow a high degree of au-

tomation, thus further reducing the workload. As em-

phasized by R¨ı¿

hrig, this framework is mainly in-

tended to elicit the requirements for safeguards, but

not to decide which speciﬁc safeguards to choose. As

a matter of fact, it is not suited as a standalone deci-

sion making method. The outcome of the evaluation

is solely a list that is suitable for implementing the

required security levels of the process components.

This is underpinned by the fact that the economical

factors of safeguards, namely their costs in monetary

or time units, are completely neglected and only the

technical aspect of security measures are evaluated.

Furthermore, POSeM ignores any speciﬁc negative

factors inﬂuencing business processes such as threats

and vulnerabilities, which are integral parts of risk as-

sessment practices. No harmful events (such as a viral

infection of the information system) are considered

and therefore no safeguards can be deﬁned to counter

that speciﬁc problem. This is a major disadvantage of

a framework that is speciﬁcally designed to be a secu-

rity evaluation process.

The CORAS Framework is a tool-supported and

model-based risk analysis methodology, the result

of the EU-funded CORAS project. The frame-

work is founded on four pillars: Risk Documen-

tation Framework, Risk Management Process, Inte-

grated Risk Management Process and System Devel-

opment Process, and Platform for Tool Inclusion. The

CORAS framework that also applies UML for mod-

eling security-related issues, is centered around a tra-

ditional risk assessment process with asset, vulner-

ability, threat identiﬁcation and evaluation and safe-

guard derivation. In order to model the risk entities, a

UML proﬁle has been developed. Unlike the other

methods, CORAS incorporates techniques of other

frameworks to realize its risk identiﬁcation process

(e.g., HAZard and OPerability study (HazOp) and

Fault Tree Analysis). This ensures a thorough analy-

sis of the problem, but also requires participants pro-

ﬁcient with these techniques to pick the most appro-

priate. The main pillar of interest, the risk manage-

ment process, is based on the Australian/New Zealand

Standard AS/NZS 4360:1999: Risk Management and

ISO/IEC 17799: 2000 Information technology - Code

of practice for information security management. In

ICEIS 2009 - International Conference on Enterprise Information Systems

322

contrast to specifying its own methods, the CORAS

risk management process relies on techniques of other

frameworks for each of the steps including HAZard

and OPerability study (HazOp), Fault Tree Analy-

sis (FTA), Failure Mode and Effect Criticality Analy-

sis (FMECA), Markov analysis, and the CCTA Risk

Analysis and Management Methodology (CRAMM).

To provide a framework for modeling all risk-related

aspects, a UML proﬁle was developed to act as a

graphical reference and communication method be-

tween the different stakeholders. The risk manage-

ment process consists of the following steps: Es-

tablish Context, Identify Risks, Analyze Risks, Risk

Evaluation, and Risk Treatment. The CORAS risk

management process represents a holistic framework

for the evaluation of information security of different

application areas. It inherits all strengths and weak-

nesses of the assessment methods it incorporates, and

the graphical models are used for describing the tar-

get system, its context, and all security features, and

therefore provide a valuable insight into the subject

and facilitates communication between the stakehold-

ers. The combination of different analysis methods

also reduces the individual weaknesses and therefore

enhances the overall quality of the risk assessment

outcome. This integration also poses a considerable

drawback of the CORAS approach. As it is rec-

ommended to rely on multiple methods in the same

process step to get a more complete result, this also

means an increased demand for time. Generally, the

CORAS methodology is very time consuming, and

the participants need experience in the multiple meth-

ods to be able to select and apply them efﬁciently.

3 CONCLUSIONS

Today many companies are not aware of their spend-

ing on security and if their investments into secu-

rity are effective. Decision makers are increasingly

challenged by having to deﬁne an optimal set of

security safeguards in line with the corporate busi-

ness processes as well as multiple strategic objec-

tives. This paper gave an overview of common meth-

ods for the evaluation of security safeguards identi-

ﬁed the method’s capabilities in considering business

processes for aligning expenditure to actual business

needs and integrating multiple objectives. This com-

parison reveals that all methodologies focus on cer-

tain aspects of information security only and neglect

others, thus not being able to provide a complete secu-

rity evaluation of a business process and multiobjec-

tive safeguard selection. The risk-based approaches

take assets and threats into account and therefore con-

sider speciﬁc risks in the analysis, but neglect busi-

ness processes. POSeM in turn is process-based,

but ignores speciﬁc threats and risks. None of these

frameworks consider multiple criteria when evalu-

ating the safeguards, except for AHP and SAEM.

But AHP is not speciﬁcally developed to deal with

security-related issues, thus lacking information secu-

rity speciﬁc functionality. And SAEM can be consid-

ered as a hybrid, including methods for multiobjective

optimization and risk determination, but not as thor-

ough as the other risk-based methods, and also lacks

business process support.

REFERENCES

Butler, S. A. (2002). Security attribute evaluation method:

a cost-beneﬁt approach. In ICSE ’02: Proceedings of

the 24th International Conference on Software Engi-

neering, pages 232–240, New York, NY, USA. ACM

Press.

CERT (2007). Octave. Online at

(http://www.cert.org/octave/index.html.)

InsightConsulting (Access in May 2007). Cramm. Online

at (http://www.cramm.com).

Maiden, N. A. and Ncube, C. (1998). Acquiring COTS

software selection requirements. IEEE Software,

15(2):46–56.

R¨ohrig, S. (2002). Using process models to analyze health

care security requirements. In International Con-

ference Advances in Infrastructure for e-Business, e-

Education, e-Science, and e-Medicine on the Internet,

Italy.

Saaty, T. L. (1980). The Analytic Hierarchy Process.

McGraw-Hill.

A COMPARISON OF SECURITY SAFEGUARD SELECTION METHODS

323