CLOUD COMPUTING FRAMEWORK
A Framework to Evaluate Cloud Computing for IT Service Providers
Rebecca Bulander and Julian Dengler
Pforzheim University, University of Applied Sciences, Tiefenbronner Str. 65, 75175 Pforzheim, Germany
Keywords: Cloud computing, Framework, Cost analysis, Risk analysis, Evaluation criteria, Service provider.
Abstract: Cloud Computing will be one of the most important technology triggers in the information technology for
the next two to five years. Thus companies which provide IT solutions have to evaluate Cloud Computing in
the context of their customer-driven IT projects. For this evaluation of Cloud Computing there is a need for
a comprehensive framework to analyse the project scenario, the requirements of costumers and business
scenario of the IT provider. Therefore this paper provides an approach of a comprehensive evaluation model
of Cloud Computing for a prospective Cloud Computing provider and includes all necessary aspects of
customer-driven IT projects.
1 INTRODUCTION
In the recent past the quality and utilization of Cloud
Computing has rapidly increased. According to
Gartner Research’s Hypecyle of 2010 (Gartner Inc,
2010) Cloud Computing will be one of the most
important technology triggers in the next two to five
years. Thus for companies wanting to provide Cloud
Computing services within their customer-driven
projects it is a main interest to evaluate Cloud
Computing for their own business model.
These companies have to compare the benefits,
costs and risks of Cloud Computing in each IT
project. Therefore they need a comprehensive
evaluation framework which compares their current
business model and the customer-driven IT projects
with the requirements of Cloud Computing. In
addition to these challenges in these IT projects is a
need to estimate the profitability of Cloud
Computing solution and compare it with the present
project solution e.g., hosting services. Within the
evaluation a prospective Cloud Computing provider
has to include several aspects and requirements of
the customer-driven IT projects and the present
business model to make a decision, e.g., the risks
undertaken or the fit to the business IT alignment. A
proper framework will lead the management of a
company according to their requirements to the most
profitable decision and give an overview of the most
expected costs, risks and business alignment of
Cloud Computing within an IT project.
The main objective of this paper is to provide a
framework to evaluate Cloud Computing in the
context of a customer-driven IT project. This is
motivated by the fact that all current frameworks in
literature only provide some aspects of a compre-
hensive analysis for a prospective Cloud Computing
provider.
This framework allows companies to specify
their requirements of a customer IT project and to
analyse and compare the IT project with different
project solutions. This practical approach compares
schematically the different solution scenarios and
gives exact answers about costs, risks and security
issues as well as profit.
2 LITERATURE REVIEW
With the rise of Cloud Computing several evaluation
frameworks have been developed. Some of these are
only partly suitable in a scenario of Cloud
Computing and have a different focus in their
evaluation. Therefore in the next section we point
out some important analysis approaches and
compare these with the requirements for a holistic
Cloud Computing project.
2.1 Models for Cost Analysis
in Cloud Computing
The decision model of Henneberger et al. (Henne-
131
Bulander R. and Dengler J..
CLOUD COMPUTING FRAMEWORK - A Framework to Evaluate Cloud Computing for IT Service Providers.
DOI: 10.5220/0003459501310137
In Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER-2011), pages 131-137
ISBN: 978-989-8425-52-2
Copyright
c
 2011 SCITEPRESS (Science and Technology Publications, Lda.)

berger et al. 2010) offers the opportunity to make a
valid decision for adopting Cloud Computing in
strategic applications in three steps. Using this
model a company is able to evaluate the risks and
benefits for Cloud Computing in a specific scenario
while selecting specific Cloud services. Kram
(Kram, 2010) developed a framework, which leads a
company in six steps to a detailed opportunity and
risk analysis for different sourcing strategies. The
framework identifies and measures the costs,
benefits and risks for an analysis. In a concluding
report different sourcing options are compared. The
Cloud Computing ROI (return on investment) model
of O’Neil and Dubowski (O’Neil, Dubowski, 2010)
evaluates Cloud Computing in a specific business
case. Therefore they identify the business critical
demands and measure the influence of Cloud
Computing and the ROI within a business case.
Matros et al. (Matros, et al. 2009) provide a very
formal model for a make-or-buy decision in the
context of Cloud Computing. Within this model the
costs for an on-premise solution are compared with
the cost of an off-premise solution. The model
considers the utilization of the resources but not the
initial hardware cost. The evaluation model of
Klems et al. (Klems, et al. (2009) provides an
abstract approach to measuring Cloud Computing in
a business case of a company. In the analysis costs
and benefits of a reference model of different Cloud
Computing scenarios can be compared.
Besides these models mentioned above there are
other evaluation models which do not focus on
Cloud Computing. However for the holistic ap-
proach of this paper they provide some relevant
aspects. Therefore we point out some of these mo-
dels. The IT sourcing model of Lacity et al. (Lacity,
et al. 1996) compares different solutions of sourcing
(e. g. Cloud Computing) and allows a company to
detect the most promising solution for the strategic
alignment of a company. The model of Martens
et al. (Martens, et al. 2010) measures the level of
maturity of Cloud services offered by companies.
The model analyses the services offered hosted by a
Cloud Computing provider. Dias de Assunção et al.
(2009) developed an evaluation model to estimate
different performance strategies in the context of the
Cloud service IaaS (Infrastructure as a Service). The
model of Kondo et al. for analysis (Kondo, et al.
2009) compares the cost benefits of Cloud Com-
puting versus desktop grids. Both models, the one of
Dias de Assunção et al. and the one of Kondo et al.,
allow companies to compare the different options
within a specific business case. Opitz et al. (Opitz, et
al. 2008) analyses in their model the specific cost
structure inside Grid Computing. In this model all
the costs of resources needed from different
providers are evaluated.
The models of Friedrich (Friedrich, 2008),
Koomey (Koomey, 2007), Yao (Yao, 2004), Chang
(Chang 2010), Jayatilaka et al. (Jayatilaka, et al.
2003), Eren & Yates (Eren, Yates, 2010) or Xin et
al. (Xin, Levina, 2008) can be seen analogically in
the line of the models presented above and don’t
provide any new perception; therefore we won’t
focus on them any further.
2.2 Evaluation of cost analysis models
For a comprehensive analysis of Cloud Com-
puting a service provider needs a framework which
includes a lot of criteria. Besides costs, benefits and
risks there are other important parameters like
strategy, security and usability. The most important
evaluation criteria are the view as a prospective
service provider instead of the view of a customer
and the option to compare different solutions within
a business case. Table 1 shows an overview of the
models presented and their included evaluation
criteria.
Table 1: Evaluation of approaches in literature.
Criteria
Evaluation model
Costs Benefits Risks Security Stra
t
egy
Provider view Usability Business case
Henneber
g
er et al.     *
Kram
    
*
O’Neil and Dubowski
   
*
Matros et al.
  
Klems et al.
   
Lacity et al.
    
Martens et al.
*
  
Dias de Assunção et al.  
Kondo et al.
   
Opitz et al.
  
*possible, but not part of the model
To get such an approach the models of Dias de
Assunção and Martens are not completely suitable
because the cost and benefit analysis is missing. The
models of Henneberger, Kram and Martens only
include a risk analysis of Cloud Computing, but no
security analysis. The models of Kondo and Opitz
are also not suitable for the required approach in this
paper, because the evaluation of risks, security and
strategy are missing.
The main concern within all models reviewed is
the lack of security analysis and the evaluation view
over all. A prospective Cloud Computing provider
needs a framework with a specific view of Cloud
CLOSER 2011 - International Conference on Cloud Computing and Services Science
132

Computing. No reviewed model includes a view of a
service provider; all models evaluate Cloud
Computing from the point of the view of a Cloud
Computing user.
Thus a new framework is necessary to offer a
complete framework for evaluating Cloud
Computing with all its aspects from a Cloud service
provider view.
3 CLOUD COMPUTING
FUNDAMENTALS
To develop a comprehensive framework several
technical aspects about Cloud Computing are given.
3.1 Cloud Architecture
The Cloud Computing architecture is based on four
layers, which are shown in figure 1.
Figure 1: Cloud architecture.
The physical infrastructure (server, network
and storage nodes) is the basis of Cloud Computing
and part of a data center in general. They are
connected to the internet and have their own security
and failover concept. (Baun et al., 2010, p. 7;
Dodani, 2009)
The virtual infrastructure provides the physical
hardware within different virtual resources. There-
fore several virtualisation manufacturers offer virtu-
alisation solutions.
The hypervisor of the virtual resources sup-
ports several operating systems and applications.
The user can install these applications in the virtual
environment and use them directly as physical
resources. (Baun et al., 2010, p. 25)
The access to the Cloud environment is possi-
ble via an interface. This interface can be a web
browser or a specific Application Programming In-
terface (API). (Stanoevska-Slabeva, Wozniak, 2010,
p. 50)
Over all architectural layers of Cloud Computing
there is a need for service management. A provider
needs specific business processes which provide
automatic and dynamic resources. Basic services
such as backup restore and lifecycle management are
the main requirements for a functional model.
(Dodani, 2009)
3.2 Cloud Services Models
To provide Cloud Computing within IT projects a
provider has to use one of three fundamental types
of Cloud service which based on the Cloud archi-
tecture shown in section 3.1 (Mell, Grance, 2009;
Stanoevska-Slabeva, Wozniak, 2010, p. 47):
Software as a Service (SaaS): The provider
delivers a complete infrastructure with preinstalled
software and the user gains access over an API or
web interface. Users can only use the software for
their requirements and cannot change the config-
uration of the applications.
Platform as a Service (PaaS): The user gains
access to a complete development environment and
tools. The user is allowed to change the config-
uration of the applications (platform) but not of the
basic infrastructure.
Infrastructure as a Service (IaaS): The user gets
access to the complete virtual infrastructure of the
provider and can install operating systems and
applications to his demands. The user has the control
over these applications but not over the basic hard-
ware resources of the Cloud.
3.3 Cloud Deployment Models
In general there are different Cloud types to provide
Cloud services within customer-driven projects.
(Mell, Grance 2009; Stanoevska-Slabeva, Wozniak,
2010, p. 47)
A Private Cloud provides the Cloud infra-
structure exclusively to one specific organisation.
The Cloud can be managed by this organisation or
third party and may exist on- or off-premise.
The Community Cloud is a Cloud infrastructure
shared by several organisations and supports a
specific community with equal concerns. Like the
Private Cloud it can be managed by the organisation
or third party and may exist on- or off-premise.
The Public Cloud provides the Cloud services to
the general public and is managed by an organi-
sation selling Cloud services.
The Hybrid Cloud combines two or more
Clouds and provides a standardized or proprietary
technology that enables data and application por-
tability but remains a unique entity.
CLOUD COMPUTING FRAMEWORK - A Framework to Evaluate Cloud Computing for IT Service Providers
133

4 FRAMEWORK APPROACH
To evaluate Cloud Computing the following
framework approach features a comprehensive
model from the point of view of a Cloud Computing
provider. The framework can be structured into the
three phases business scenario, analysis and evalu-
ation (see figure 2).
4.1 Overview
Figure 2: Overview of the framework to evaluate Cloud
Computing.
4.2 Business Scenario
Within the business scenario all essential parameters
to provide Cloud Computing are identified.
Therefore the model of Klems et al. (Klems, et al.
2009) delivers basic ideas such as the definition of a
reference model. The analysis of the project
scenario, the project demands and the Cloud services
delivered lead to the alternative solutions scenarios.
Like within the models of Klems et al. (Klems, et al.
2009) and Matros et al. (Matros, et al. 2009) it is
necessary to analyse the project relating to its
demands in the context of the project scope and
different solutions. To compare it with the previous
project solution of the company the framework
needs a defined reference model, like the hosting at
the data center of the customer.
4.3 Analysis Phase
After setting the project scenario and scope there are
several analyses necessary, e.g., benefit, cost, risk,
security and accounting analysis, which are
explained below. The final evaluations of the results
are part of the next phase. The combination and
enhancements of different analyses and approaches
from the models of Henneberger et al. (Henneberger,
et al. 2010), O’Neil et al. (O’Neil, Dubowski, 2010),
Kram (Kram, 2010) and Klems et al. (Klems, et al.
2009) allow this comprehensive step.
Cost Analysis. In the cost analysis the cost of each
alternative scenario in the complete project period
are measured and summed up. Therefore table 2
shows possible cost drivers within a project period.
The sum of all costs over the complete project
period is the total cost for the respective alternative
scenarios.
Table 2: Cost driver.
Cost driver Amount
Hardware costs
Server, network, infrastructure, maintenance
Complete project
Software costs
Operating system, applications, database,
maintenance
Complete project
Communication cost
Delivery, utilisation
Project quota
Operational cost
Power, cooling, fire security
Project quota
Administration costs
Imputed rent, security, facility management,
access control
Project quota
Personal cost
Installation (hard-/software), migration,
monitoring, assistance
Project quota
Development costs Complete project
Training costs Complete project
Support/Service Project quota
General administration costs Project quota
External costs
Development, service
Complete project
Benefit Analysis. For each alternative scenario the
expected benefit has to be identified. Therefore table
3 shows possible benefit categories for each
scenario. For each category the benefit is identified
and measured on a monetary basis.
CLOSER 2011 - International Conference on Cloud Computing and Services Science
134

Accounting Analysis. Additional to the benefit
analysis the account analysis identifies und measures
the accounting opportunities for the customer over
the complete project period. Based on the project
scenario there are three general types of location for
the Cloud and possible accounting parameters. Table
4 shows possible parameters similar to the pricing
system of the Amazon Elastic Compute Cloud
(Amazon EC2) (Amazon 2010).
Table 3: Benefit analysis.
Benefit Description
Processes
Improvements of current and new business
processes (through standardisation or
efficiency)
Finance
Reduction of investment costs
Product and
service
Improvements of the product and service
quality, competitive advantages
Relation to
customer
and supplier
Improvements in relation to customer and
supplier , cost reduction through better
connection
Table 4: Accounting analysis.
Cloud location Accounting parameters
Customer
Maintenance (per period)
Monitoring (per period)
Support (per period)
Provider/
Third party
Processing power (per hour)
Storage (per GB)
Virtual server instances (per instance)
Data transmission (per GB)
Risk Analysis. Every project or investment has its
own risks and the according impact. With the risk
analysis they are be identified and analysed.
Therefore every risk and the incidence rate are
calculated and the possible loss documented. Table 5
shows possible risk categories within an IT project.
Table 5: Risk analysis.
Risk category Risks
Organisation
Changes in organisational structure or processes
Management
Lack of support by the management
Market
Dependence on third party, changes in market
conditions
Project
management
Lack of requirements, wrong project scope,
budget or delays, lack of responsibilities or
communications
Staff
Lack of know-how, shortage of staff
Technology
Delivery bottleneck, development problems
Customer
Cancellation of a contract, changes in
requirements or project scope, insolvency
Security Analysis. In addition to the risk analysis a
security analysis is also required. For each
alternative scenario and through the specific contract
with a customer all possible security issues (e. g.
data privacy) have to be identified and documented.
4.4 Evaluation Phase
All identified parameters of the analysis phase are
evaluated in this phase. With such an evaluation the
management can assess the different solutions. The
following approach is based on the models of Kram
(Kram, 2010), Klems et al. (Klems, et al. 2009) and
Lacity et al. (Lacity, et al. 1996) and continued.
Investment Evaluation. The investment evaluation
helps companies to compare the total costs of each
solution scenario over the complete project period.
Therefore all costs of the cost analysis for each
scenario (alternative or reference scenario) are
summed up and compared.
ROI Evaluation. The ROI evaluation is the central
evaluation of this framework. For each scenario the
costs are summed up as total costs for the complete
project period. These costs are the total cost of
investment and the ratio to the sum of all monetary
benefit. This ratio is the ROI of each solution. In this
case the financial benefit is the combination of the
results of the benefit and account analysis. This
evaluation allows a conclusion of the used capital
and its cash flow. The ROI evaluation gives an exact
key figure for the profitability of each scenario and
an ROI of more than 100 percent is cost-covering.
Risk Evaluation. The risks detected in risk analysis
are compared with the incidence rate and the
business influence into a rating table. According to
their priority they are monetarily assessed. The
framework provides an overview for each scenario
with all risks and their financial business influence
on the project and the company.
Security Evaluation. Like to the risk evaluation the
security issues are ranked with their relevance and
compared with a monetary value. The monetary
values are the cost of security violations, sabotage or
contract penalties.
Scenario Evaluation. The final evaluation is the
comparison of the different solution scenarios and
the reference model. All perceptions of the
evaluation phase are summed up in the final step to
lead the company management to the most profitable
decision in a concrete IT project scenario.
CLOUD COMPUTING FRAMEWORK - A Framework to Evaluate Cloud Computing for IT Service Providers
135

5 CONCLUSIONS
AND FURTHER STEPS
This approach provides a framework to evaluate
Cloud Computing from the view of a company
which wants to provide Cloud services to their
customer. The framework is based on previous
approaches from literature but enhances these with
further Cloud Computing specific aspects e.g., a
business IT alignment fit or data privacy. For an
evaluation with the business model of an IT provider
there are several analyses required e.g., a business
strategy analysis.
Further steps for the development of this
framework in practical use are:
The framework has to include analyses to
measure the business alignment and strategic impact
of each Cloud solution within the project scenario.
This analysis can be part of a strategic phase and
contains the identification of the business strategy
and objectives of the IT provider. Beside this, the
framework has to include detailed practical in-
structions and criteria for a company. In further steps
this developed guideline helps IT provider to do the
evaluation with a standardised approach to measure
the value of Cloud Computing within a specific
customer-driven project.
Finally the framework approach of this paper has
to be evaluated in an actual business scenario with
specific customer demands and business require-
ments. With the findings of this practical
implementation of this framework the approach can
be adjusted to the common business scenario of IT
providers.
REFERENCES
Amazon Inc. (2010). Amazon EC2 Pricing, Website:
http://aws.amazon.com/ec2/pricing/. Accessed at. 12th
Jan. 2011. Update: 7 December 2010.
Dias de Assunção, M. et al. (2009). Evaluating the cost-
benefit of using cloud computing to extend the
capacity of clusters. In: Proceedings of the
International Symposium on High Performance
Distributed Computing, pp. 141-150.
Baun, C. et al. (2010). Cloud Computing: Webbasierte
dynamische IT-Services. Springer, Berlin, Germany.
Chang, V. (2010): Investigating the Cloud Computing
Business Framework - Modelling and Benchmarking
of Financial Assets and job submissions in Clouds. In:
UK e-Science All Hands Meeting 2010, Research
Clouds: Hype or Reality Workshop, September 2010.
Dodani, M. (2009). The Silver Lining of Cloud Computing.
In: Journal of Object Technology, Vol. 8, No. 2,
March 2009, pp. 29-38.
Eren, U., Yates, D. J. (2010). Enterprise Fraud
Management Using Cloud Computing: A Cost-Benefit
Analysis Framework. In: Eighteenth European
Conference on Information Systems.
Friedrich, P. (2008). Allgemeine TCO-Betrachtungen zum
aktuellen IBM-Mainframe System z10. In: e-Journal of
Practical Business Research, Vol. 6 (December 2008).
Gartner Inc (2010). Gartner's 2010 Hype Cycle Special
Report Evaluates Maturity of 1,800 Technologies,
Stamford, USA, 2010. Website:
http://www.gartner.com/it/page.jsp?id=1447613.
Accessed at. 22th Dec. 2010. Update: 7 October 2010.
Henneberger, M. et al. (2010). Ein Entscheidungsmodell
für den Einsatz von Cloud Computing in
Unternehmen. In: HMD - Praxis der Wirtschafts-
informatik, Vol. 275 (October 2010), pp. 79-84.
Jayatilaka, B.et al. (2003). Determinants of ASP choice:
an integrated perspective. In: European Journal of
Information Systems, Vol. 13, pp. 210 - 224.
Klems, M. et al. (2009). Do Clouds Compute? – A Frame-
work for Estimating the Value of Cloud Computing.
In: Business Information Processing, Vol. 22, January
2009, pp. 110-123.
Kondo, D. et al. (2009). Cost-Benefit Analysis of Cloud
Computing versus Desktop Grids. In: IEEE
International Symposium on Parallel & Distributed
Processing, May 2009.
Koomey, J. (2007). A simple Model for Determing True
Total Cost of Ownership for Data Centers, Uptime
Institute, Santa Fe, USA.
Kram, M. (2010). Konzeption eines Bewertungsmodells
zur Evaluierung von Cloud-Sourcing-/Utility-
Sourcing-Strategien, Stuttgart Media University,
Stuttgart, Germany.
Lacity, M. C. et al. (1996). The Value of Selective IT
Sourcing. In: Sloan Management Review, No. 37,
April 1996, pp. 13 - 25.
Matros, R. et al. (2009). Make-or-Buy im Cloud
Computing – Ein entscheidungsorientiertes Modell für
den Bezug von Amazon Web Services. In: Bayreuther
Arbeitspapiere zur Wirtschaftsinformatik, No. 45
(March 2009).
Martens, B. et al. (2010). Datenbank und Reifegradmodell
für die Auswahl und Bewertung von Cloud Computing
Services. In: HMD - Praxis der Wirtschaftsinformatik,
Vol. 275 (October 2010), pp. 52-61.
Mell, P.; Grance, T. (2009). Definition of Cloud
Computing. Version 15, National Institute of
Standards and Technology, Gaithersburgg, USA.
O’Neil, M; Dubowski, S. (2010). All About Cloud ROI. In:
IT Market Dynamics.
Opitz, A. et al. (2008). What Does Grid Computing Cost?
In: Journal of Grid Computing, Vol. 6, No. 4
(December 2008), pp. 385-397.
Stanoevska-Slabeva, K.; Wozniak, T. (2010). Grid and
Cloud Computing: A Business Perspective on Tech-
neology and Applications. Springer, Berlin, Germany.
CLOSER 2011 - International Conference on Cloud Computing and Services Science
136

Xin, M.; Levina, N. (2008). Software-as-a-Service Model:
Elaborating Client-Side Adoption Factors. In:
Proceedings of the 29th International Conference on
Information Systems.
Yao, Y. (2004). An integrative model of clients' decision
to adopt an Application Service Provider, Louisiana
State University, Baton Rouge, USA.
CLOUD COMPUTING FRAMEWORK - A Framework to Evaluate Cloud Computing for IT Service Providers
137