EVALUATING THE SERVICE LEVEL AGREEMENTS OF NDT

UNDER WS-AGREEMENT

An Empirical Analysis

Marcos Palacios

, Laura Moreno

, María José Escalona

and Mercedes Ruiz

University of Oviedo, Oviedo, Spain

IWT2 Group, University of Seville, Seville, Spain

University of Cádiz, Cádiz, Spain

Keywords: Service-Level Agreements, Software Testing, Web Engineering.

Abstract: Service-Level Agreements (SLA) is one of the well-known mechanisms to manage the services offered by a

provider. In software engineering, it is usually taken as the starting point to define the collaboration between

the development team and clients. More concretely, an SLA is a part of a service contract where the level of

service is formally defined. In this document, both providers and clients purchase commitments that rule the

working guidelines in a software project. However, despite its relevance, sometimes it is difficult or too

subjective to assure that providers and clients are following SLA. This paper proposes a research approach

based on the standard WS-Agreement and model-based paradigms that offers a mechanism for SLA testing

and presents an initial evaluation of this approach in a Web methodology named NDT (Navigational

Development Techniques). The paper refers to some practical experiences so as to evaluate the quality of

the results, as well as introduces a new research line as a future work.

1 INTRODUCTION

In the scope of software engineering, a Service

Level Agreement (SLA) is a contract or technical

document signed by both the service provider and

the user. Basically, this document states all the

conditions to be fulfilled regarding the provision of

the services. Furthermore, the SLA can also specify

the set of actions (penalties or rewards) that must be

applied if the guaranteed conditions are not satisfied.

In order to make a decision about the fulfilment

of the SLA conditions, it is necessary to gather

information about the service provision. This task

may be performed by means of a set of monitors in

charge of obtaining relevant information according

to the SLA terms. However, the evaluation of SLAs

is not a trivial task and may result very difficult or

too subjective to be carried out. One of the main

reasons is that SLAs are not always defined

following a systematic and standard specification

but they are usually elaborated in an unstructured

format or even through natural language.

This paper presents a research approach that

allows specifying SLAs in a systematic way using

model-based paradigms. The approach is aligned

with the application of a Web methodology named

NDT (Navigational Development Techniques) that

is currently used in a large number of companies in

Spain. In this work we compare NDT solution with

the standard WS-Agreement with the aim at

identifying their common characteristics and

differences. The approach is assessed by means of

two empirical examples extracted from real projects

developed by the Regional Cultural Ministry in

Andalusia, Spain.

The paper is organized as follows. Section 2

outlines the basic concepts about WS-Agreement

and a general vision of NDT methodology. Section 3

describes the NDT solution to manage SLAs by

analyzing empirical examples in comparison with

WS-Agreement. Finally, conclusions and future

work are presented in Section 4.

2 BACKGROUND

This section introduces the basic concepts about

WS-Agreement language as well as a general vision

of NDT methodology.

246

Palacios M., Moreno L., José Escalona M. and Ruiz M..

EVALUATING THE SERVICE LEVEL AGREEMENTS OF NDT UNDER WS-AGREEMENT - An Empirical Analysis.

DOI: 10.5220/0003906602460250

In Proceedings of the 8th International Conference on Web Information Systems and Technologies (WEBIST-2012), pages 246-250

ISBN: 978-989-8565-08-2

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

2.1 WS-Agreement

WS-Agreement is a recommendation of the Open

Grid Forum (OGF), which offers an XML-based

structure to define SLA documents. The

specification of an SLA in WS-Agreement is

composed of the following elements: name, context

and Terms. Although there are different types of

terms, Guarantee Terms are the most important

elements of the SLA and they list the obligations

that have to be fulfilled by a specific obligated party

(service provider or service consumer). A guarantee

term includes a list of services the term applies to

(Service Scope), a Qualifying Condition (QC) which

is an assertion under which the term is applied, a

Service Level Objective (SLO) which is the

condition that has to be guaranteed and, finally, a

Business Value List (BVL) which specifies a set of

business values related to the agreement description

(i.e., penalties, rewards, importance).

2.2 NDT – Navigational Development

Techniques

NDT (Navigational Development Techniques) is a

methodological approach based on the model-driven

paradigm. NDT begins defining requirements phase

formal metamodels and a set of transformations to

generate analysis phase models. Requirements

engineering phase begins with the definition of the

objectives of the system. In this context, NDT can be

understood as an objective-guided methodology.

Once the objectives are defined, NDT proposes

the capture and definition of all the requirements in

the system. In the analysis phase, NDT works in a

similar manner, representing a content model, a

navigation model and an abstract interface model.

All of them are also defined in a set of metamodels.

After analyzing both the requirements

metamodel and the analysis metamodel, this

approach establishes a series of relationships among

all the artefacts from the different models.

In recent years the initial approach of the NDT

methodology (Escalona and Aragon, 2008) has

evolved and nowadays it offers full support for the

entire life cycle. It covers the stages of viability

study, requirements, analysis, design, construction

and implementation, as well as maintenance and

testing phases during software development. Thus,

NDT is compatible with a set of processes such as

project management or safety and quality assurance.

Currently, a large number of companies in Spain are

working with NDT in software development due to a

set of free tools grouped in NDT-Suite (NDTQ-

Framework 2011).

3 EVALUATION OF SLA OF NDT

In the maintenance phase, NDT supports SLAs in its

lifecycle. In this Section, we present the solution

offered by NDT and we compare this solution with

the standard WS-Agreement.

3.1 SLA in NDT

The current solution of NDT to support SLAs is

offered in two ways to the development team. The

first one, more classical, is a word pattern that has to

be completed by the development team and accepted

by the client. The index of this NDT SLA Template

is organized in the following sections (Figure

1).

Figure 1: NDT SLA Template Index.

1. Introduction: defines business objectives,

scope of services, details of the organizational

structure and relationships between the

organization and service providers.

2. Project description: includes information

about the technical infrastructure and

architecture, obligations, terms of license or

responsibilities among others.

3, 4 and 5: Annex, Glossary and Bibliography.

This model means a good solution for development

teams although it poses limitations (difficulty to be

tested and maintained). Regarding this pattern and

the fields defined in the model, a metamodel to

include SLA in NDT was developed. Thus, by

means of elements of UML, a development team can

develop an integrated SLA in the rest of the system.

3.2 Empirical Examples

With the aim at comparing NDT approach with WS-

EVALUATINGTHESERVICELEVELAGREEMENTSOFNDTUNDERWS-AGREEMENT-AnEmpiricalAnalysis

247

Agreement standard, we assessed two empirical

examples of SLAs obtained from two real projects

carried out by the Culture Government in Andalusia.

The first example is named Archiva and

consists in a report to the Steering Committee on the

proposed maintenance of the application. It analyzes

the current status of the project and the incidents

response time, studying the reasons that increase

these times and setting the necessary measures to

correct them. Then, the project indicators will be

graphically represented according to open and close

incidents. The report must be based on SLA

indicators so as to obtain the list of incidents that do

not fulfil such indicator and the reasons. Another

indicator is the impact analysis, which allows us to

be informed of the state of the system, showing the

set of indicators that have improved and those that

must be improved. Finally, an analysis of the whole

project must be done, showing a matrix with the

current problems of the project and identifying in

each case the impact of the lack of resolution as well

as the necessary actions to solve it.

The second example involves the project for the

evolution of Mosaico, which describes the

operational, management and relationships models

as well as the quality parameters from the catalogue

governing during the full life of the project. This

plan should help to increase efficiency in the

information systems life cycle management and to

facilitate continuous improvement of the quality of

the TIC services provided. The description of the

project presents the services, the operating model

which describes the details of the service requests

and the information that must be provided at each

stage when processing a request for service. Once

the volume of service and required effort have been

decided, the indicators that determines whether it is

necessary to establish specific actions to stabilize the

situation or start the modification process should be

set. Finally, operational tools are listed and the

metrics and indicators per service are established. It

will also establish periodicity for SLAs

measurement with SLA indicators and the basic

documents that must be enclosed for the committees.

3.3 SLA vs WS-Agreement

This section studies the traceability between the

SLA template provided in NDT and WS-Agreement.

The objective is twofold: identify the common

characteristics of both specifications and describe a

reasonable mapping between the elements of each

template. In Table 1 the elements of the NDT SLA

Template are listed in rows while WS-Agreement

elements are represented in columns. We use () in

a cell to show that these characteristics represent the

same information in both specifications. We use ()

to indicate that the information within the NDT SLA

Template is not explicitly specified in WS-

Agreement, but should be reasonably included in the

corresponding WS-Agreement element. In the same

way, we use () to highlight that the information

specified in the WS-Agreement element can be

mapped to the section of NDT SLA Template

represented in that row. We use () if the

information contained in WS-Agreement and NDT

SLA Template is not the same, but it can be

bidirectionally mapped from both elements. Finally,

the cross-lined cells points out the elements of both

specifications that do not have similar information,

so they could not be directly mapped.

Both specifications share identical information

in different parts of their structure. Firstly, the whole

definition of the services is described in the Project

Service Types section of NDT SLA Template and

also in the SDTs of WS-Agreement through a

specific DSL. Furthermore, both specifications allow

identifying the set of measurable properties with

their corresponding metrics.

Probably, the most important part of the SLA

comprises the specified conditions to be fulfilled by

the stakeholders signing the agreement. This

information is stated in the Service Guarantee of the

NDT SLA Template and the GTs section of WS-

Agreement. Such sections indicate who is

responsible for satisfying the conditions and the

thresholds that have to be honoured.

The fulfilment or violation of the SLA terms

may lead stakeholders to further consequences.

Hence, a specific policy must be agreed, depending

whether the objectives have been met or not. This

information may appear both in the Penalties and

Rewards Planning of NDT SLA Template and in the

BVL of a GT in WS-Agreement.

Generally, each agreement lasts a specific

period of time. However, WS-Agreement only

includes the expiry date of the SLA, whereas the

NDT SLA Template indicates both the initial and

final dates of validity of the SLA.

In addition to the specific guarantees of the

agreement, different information about the business

objectives, stakeholder’s organization or the

relationship between service providers and

consumers, among others, may be specified in the

Introduction section of NDT SLA Template and the

Context section of WS-Agreement. Furthermore, a

description of the project or system can be given

both in the Project Service Types of NDT SLA

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

248

Table 1: Traceability between NDT SLA Template and WS-Agreement specification.

Agreement ID

Name

Context

Terms

Term Compositors

Service

Description

Terms

Service

References

Service

Properties

Guarantee Terms

Name

Obligated

Service Name

QC SLO BVL

AgreementInitiator

AgreementResponder

Provider

Expiration Date

TemplateID & Name

Any Attribute

Name

Service Name

Full Description

Name

Service Name

Representation

Name

Service Name

Variable Name

Metric

Location

Condition

Name

Target

Custom Value

Importance

Penalty

Reward

Preference

Custom Business Value

Introduction







 

Project Description





Project Service Types

     

Service Properties Types

 

Metrics

  

Respons.

Service

Types

Responsibility

   

QoS Thresholds

     

Compliance

Monitoring

Reports

Format

Addressee

Planning

Problems

Managem.

Procedures

Scalability

Severity Levels

 

Resource Planning

Incent. and Penalt. Plan

 

Changes

Manag.

Planning

Validity Period



Review Proc.

Improvement

Procedures

Annex, Glossary and Ref.

Template and the Service Description Terms of WS-

Agreements, here again through a specific DSL.

4 CONCLUSIONS

This paper presents an approach that allows

specifying SLAs in a systematic way through a

model-based paradigm. The solution is applied in

the context of a Web methodology named NDT,

which is currently in use in several companies in

Spain. The evaluation of the solution is performed

by two empirical examples of SLAs extracted from

real projects developed by the Culture Government

in Andalusia. This evaluation has allowed us to

identify similarities and differences between both

specifications helping to address our future work.

With the aim at improving the profile of NDT,

it should be integrated with the types of problems

and tests, the business objectives and the scope of

the contracted services as well as the organizational

structure and its relationships. This would follow the

standard and improve the traceability of the system.

ACKNOWLEDGEMENTS

This work has been funded by the Department of

Education, Science and Innovation (Spain) and

ERDF funds within the National Program for

Research, Development and Innovation, project

PROTES (TIN2010-20057-C03), National Network

Quality Assurance in Practise, CaSA (TIN2010-

12312-E), FICYT (Government of the Principality

of Asturias) Grant BP09-075 and NDTQ-Framework

project of the Junta de Andalucía, Spain (TIC-5789).

EVALUATINGTHESERVICELEVELAGREEMENTSOFNDTUNDERWS-AGREEMENT-AnEmpiricalAnalysis

249

REFERENCES

Escalona, M. J., Aragón, G., 2008. NDT a Model-Driven

Approach for Web Requirements. In: IEEE

Transactions on Software Engineering. vol. 34 (3), pp

377-390.

NDTQ-Frawework. Available in www.iwt2.org/iwt2/ndt-

qframework.php. Last accessed in February 2012.

WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies

250