INTEGRATION METHOD AMONG BSC, CMMI AND SIX

SIGMA USING GQM TO SUPPORT MEASUREMENT

DEFINITION (MIBCIS)

Leonardo Romeu, Jorge Audy and Andressa Covatti

School of Computer Science, Pontifical Catholic University of Rio Grande do Sul

Porto Alegre, Rio Grande do Sul, Brazil

Keywords: Software Process Improvement, CMMI, GQM, Six Sigma.

Abstract: The software quality area has presented various studies and surveys in different fronts, either about products

or processes. There are many initiatives in the area of software process improvement, which might be more

than often conflicting in an organization. If we observe some of the existing models and methodologies in

the market, the CMMI Model and the Six Sigma Methodology stand head and shoulders above the rest for

being complemented. While CMMI focuses on organization and on process management and Six Sigma has

its focus on the client and on the financial results, both highlight the importance of the data produced for

decision making. This study presents a method for the integrated implementation of the CMMI Model and

the Six Sigma Methodology for programs of process improvement, having as a backup measurement and

assessment techniques such as the Balanced Scorecard (BSC) and the Goal-Question-Metric (GQM).

1 INTRODUCTION

Quality is a concern even bigger in all types of

organizations, in the software organizations this

concern is intensified, once that the level of

information systems in all sections grows. During

the last years, there has been a great interest by the

software engineering community in general, for

process improvement as an agent that provides

product improvement, taking into account that, by

understanding and improving the development

processes it is possible to improve the product

quality and cut down on expenses. (Sommerville,

2003)

The major concern with the software process is

straight related with the necessity to understand,

evaluate, learn, improve, plan, verify and mainly

control software engineers’s job. To achieve this

goal it is needed to, document the facts, define,

measure, analise, evaluate, control and change the

development processes.

These controls could help the organization to

reduce terms, supply trustable budgets, speed their

processes of development and will have primacy of

the market (Rocha, Maldonado and Weber, 2001).

Balanced Scorecard (BSC) is a businees

performance evaluation known by the industry as a

measure of health of the organizations. The BSC can

also be described as a bunch of actions derived from

the organizational strategy that are selected to

represent a tool that will be used by the leadears to

communicate either external or internal in the

organization (Goethert and Fisher, 2003).

The Capability Maturity Model Integration

(CMMI) is a model very well spread and used in the

world to implement processes improvement in

software companies. (Chrissis, Konrad and Shrum,

2003).

The methodology Six Sigma is at its turn know

to be used in the industry to improve the

development processes of products. It is based on

statistics to measure and prove the improvement of

the products and processes.

These initiatives BSC, CMMI and Six Sigma,

might be called as complements, in the moment that

we use the CMMI to implement development

processes and Six Sigma as the official methodology

to implement continuous process improvement.

Connected to this, the utilization of BSC to support

the improvement program, to be adjusted to

171

Romeu L., Audy J. and Covatti A. (2009).

INTEGRATION METHOD AMONG BSC, CMMI AND SIX SIGMA USING GQM TO SUPPORT MEASUREMENT DEFINITION (MIBCIS).

In Proceedings of the 11th International Conference on Enterprise Information Systems - Databases and Information Systems Integration, pages

171-176

DOI: 10.5220/0002003901710176

 SciTePress

organization’s strategic objectives and the GQM

approach to define efficient measures.

This paper aims to introduce a method that

proposes connections between models and

methodologies above mentioned and to describe a

case study performed to validate the method

proposed.

This paper has the following structure: section 2

presents the theoretical base; section 3 describes

related studies; section 4 describes the proposed

method; section 5 discuss the case study; section 6

presents the conclusions, future studies and the

research limitations.

2 THEORETICAL BASE

2.1 CMMI

CMMI is an evolution to the model Capability

Maturity Model (CMM) (Kulpa, 2003). It was

created to help the organizations in maturity or

capabilities evaluations of process areas, set

improvement priorities and to implement these

improvements. To attend these objectives in the

several areas either the systems engineering as well

as software engineering, CMMI version 1.1 is

composed by the following models:

• SE-CMM (Systems Engineering CMM);

• SA-CMM (Software Acquisition CMM);

• IPD-CMM (Integrated Product

Development Team Model);

• SECAM (System Engineering Capability

Assessment Model);

• SECM (Systems Engineering Capability

Model);

CMMI is formed by the integration of the models

above and it covers products, development and

maintenance areas. Nowadays four areas are covered

by CMMI v1.1: Systems Engineering, Software

Engineering, Integrated Product and Process

Development and Suppliers (Chrissis, Konrad,

Shrum, 2003).

2.2 Six Sigma

The Six Sigma methodology can be considered as a

quality management philosophy. It is a methodology

used to manage business and processes improvement

and has its focus on the customer. It uses data to lead

and implement solutions that are found from causal

analysis of a certain problem.

The method uses statistics to measure and

analyse organization’s processes. Statistically

talking, reach the Six Sigma means that, the process

or product is being done or manufactured practically

with no defects. The Six Sigma represents 3.4

defects found per millions of opportunities, which

represents a potentiality of about 99.9996% of

efficiency. It is known that nowadays the majority of

implemented processes operate between 3 and 4

sigma, that means, close to 93.3% to 99.4% of

efficiency. (Hayes, 2005).

Six Sigma, apart from the quality models, is a

methodology oriented to business that uses a process

improvement approach with several aspects like cost

reduction and profit increase. Its main principle is

increasing customer’s satisfaction based on the

defects reduction. Six Sigma methodology, when

used by the organizations, enables that the team

responsible for implementing improvement, to

identify the process and the actions correspondent

that will be affected. This flexibility is the part that

allows Six Sigma and its joint of tools to be easily

integrated to process improvement models already

existent.

2.3 Balanced Scorecard

The organizations face several obstacles when they

develop a measuring system that is able to measure

the right things at the right time. What is needed is a

system that balances the historical accuracy of

financial numbers with the future performance

objectives as well as it helps the organization to

implement new strategies.

BSC is a tool that meets both purposes, the

approach developed by Robert Kaplan and David

Norton as a result of a research on the 90’s intended

to prove that only financial indicators were not

efficient to the global organizations. Throughout

researches made in many companies, they created

the so called Balanced Scorecard, which apart from

the financial indicators also concerns with

customer’s problems, internal processes and learning

and growth. (Niven, 2002).

2.4 GQM

The Goal Question Metric approach is based on the

presuming that, for an organization to implement a

measurement process in an effective way, first they

must specify their objectives and projects. From this

point, data for each objective must be planned so

that they can be set in an operational manner.

Finally, provides a structure to interpret the collected

ICEIS 2009 - International Conference on Enterprise Information Systems

172

data that would meet the measurement objectives

previously defined. (Basili, 1994)

GQM helps to define and gather objectives to

process models, products and quality perspectives

based on specific needs of projects and organizations

through measurement program.

Figure 1: GQM structure.

3 RELATED STUDIES

Recent studies prove the concerns to integrate

improvement efforts, in order to obtain

organizational growth in all areas using different

models, methods and approaches. (Rocha,

Maldonado and Weber, 2001).

According to Vasques, Balanced Scorecard,

CMMI and Six Sigma are strongly related because,

the first deals with de organization’s strategies,

directions that it is going to take and its priorities;

the second one provides to the organizations a guide

of management by processes and the third provides a

process improvement methodology already rested

and proved worldwide. (Vasques, 2005).

Pickeril says that there is an interesting

relationship between the improvement approach

used by CMMI, the IDEAL model, developed by

SEI, and Six Sigma methodology. IDEAL

improvement life-cycle contemplates five steps to

process improvement: Initiating, Diagnosing,

Establishing, Acting and Leveraging and it might be

used to implement process improvement along with

Six Sigma. (Pickerill, 2005).

As per Siviy, Six Sigma can be used as a support

tool to implement CMMI in software organizations.

Teams that are used to the Six Sigma methodology

feel more comfortable and contribute on the

implementation of a quality model like CMMI, and

besides that, these teams will not be resistant on the

implementation of measures and controls in the

proposed processes. (Siviy, 2004).

4 PROPOSED METHOD

After the studies of several models, methodologies,

and methods it becomes clear that the efforts to

processes implementation and continuous

improvement programs at the software organizations

quality area need to be adjusted to the strategic plans

of the organization. This basis it will be possible to

help on the achievement of goals and objectives

proposed to them. The method is divided basically in

three levels: Organizational, Projects and Systems

and Continuous improvement.

Figure 2: Method basic configuration.

As we can note in the figure 2 that the GQM

approach is part of all levels of the model supporting

measures definition and interpretation.

To represent the method MIBCIS it was chosen

UML through activities diagrams. Some adjustments

and use of stereotypes are necessary to better specify

the method.

The activities proposed in the method MIBCIS

does not happen in sequence, some are periodical

and some are by events. The strategic plans, for

instance, are done/revised periodically. On the other

hand, continuous improvement happens by events,

the improvements or new processes implementation

is an activity usually triggered from improvement

opportunities or requests of change. Besides the

activities described on the model through UML

activity diagrams, a series of document templates

have been developed as work products for each

activity.

4.1 Organizational Level

On the organizational level the organization’s

strategic objectives are evaluated. The business

processes passes by an analysis, and based on that,

BSC approach is used to define performance

indicators that will support decisions that are

supposed to be taken based on that data.

The organization’s strategic objectives and

planning are used to construct the strategic maps

INTEGRATION METHOD AMONG BSC, CMMI AND SIX SIGMA USING GQM TO SUPPORT MEASUREMENT

DEFINITION (MIBCIS)

173

which are part of the existing work products of this

stage. At this point it is also created the so-called

document Strategic Objectives, where they are

organized within BSC perspectives.

Each strategic objective must be worked and

detailed afterwards to generate the performance

indicators definition. The objectives detailing is

done in a document called strategic maps.

Based on the information of these documents,

finally it is defined the performance indicators. The

performance indicators provide management

information to support decision making process. To

help these performance indicators to be composed

there is a necessity to define and specify good

measures. MIBCIS proposes the GQM approach to

be used to set the measures.

4.2 Projects and Systems Level

In this level, the method MIBCIS suggests to use the

reference model CMMI to define software

development processes and management processes.

The reference model guides the organization in the

direction of saying what should be done to achieve

quality projects but not how to do it. How to do will

be set by the organization itself that will generate a

framework of processes from the good practices

suggested by CMMI.

To help in the construction of new processes the

method also proposes the usage of a methodology

called Design for Six Sigma (DFSS) that implements

a development life-cycle, specifically to attend the

necessities of creation of new processes, products

and services.

This combination of Six Sigma to implement

new processes using CMMI as a reference model

turned to be able through the mapping of DFSS to

IDEAL.

Figure 3: DMADV.

By using DMADV life-cycle, which can be seen

on Figure 3 above, along with the IDEAL model the

new processes will be set. The process commences

through the Define phase, as its name says it deals

with customer’s needs definition, be it internal or

external. In the Measure phase these needs are

specified on a measurable way. It can be achieved

by studying competitors and/or partners results, and

to do so, benchmark technique can be used. At the

Analyse phase, possible solutions must be

considered to support customer’s requirements. On

this phase, besides solutions’ analysis, one of them

will be chosen to be developed. In the Design phase

the previously chosen solution is then fully

developed and implanted. At last, in the Verify

phase it is necessary to identify if the new process

requirements are in compliance with the

benchmarks, until the execution allows the

organization to have an own base of comparison.

After the defining the new process, the

organization must use its GQM program to define

measures to manage it quantitatively.

Eventually, when the new process is ready, it

will be part of the organization’s process framework.

4.3 Continuous Improvement Level

The Continuous improvement level is responsible to

identify improvement opportunities and to develop

solutions to enhance the processes, products and

services quality.

The improvement opportunities identification is

done through performance indicators data analysis.

All the generated organization’s measurement data

will be submitted to analysis by the process

improvement group that will be formed by a

multidiscipline team that are able to evaluate the

data over several perspectives.

After this analysis the results will be re-checked

and if they are satisfactory, the data analysis is

concluded. If the analysis results are not acceptable

the problem causes need to be identified and the

group might take two different decisions: either it is

necessary to create a process improvement project or

it is necessary to create a new process to give

solution of the existing problems. If the analysis

results points to the creation of a new process, so the

methodology Design for Six Sigma will be used as

per previously described. If the analysis results

suggests that there is no need of a new process, but

changing one of the existing processes, so the

methodology DMAIC will be used.

The method DMAIC of the Six Sigma

methodology can be mapped to the process

improvement activities proposed by the IDEAL

model and it is formed by five phases as shown

below:

Figure 4: DMAIC.

ICEIS 2009 - International Conference on Enterprise Information Systems

174

5 CASE STUDY

The case study was applied on a large scale software

development unit. The organization owns offices in

more than 34 countries worldwide. According to the

information supplied by the organization, it has

more than 55 thousand employees all over the world.

The unit where the study was carried out is located

in Porto Alegre, Rio Grande do Sul State, Brazil.

There are more than 400 people working on projects

that meet the companies’ IT area in any region of the

world. The biggest interaction is with the

headquarters, located in Austin, Texas, USA, which

is responsible for the demand of projects. This

Brazil’s unit is certified on SW-CMM level 2

January/2003.

5.1 Organizational Level

Concerning this level of the model, the same brings

meaningful contributions setting the flows for usage

of the Balanced Scorecard. This level of the model

does the alignment of the program of quality with

the organization’s strategic objectives in a clear and

efficient way.

Through the case study it was noted that this

level of the method presents signs of possibility of

generalization, once that, the studied organization

does not uses the technique BSC. This basis the

performance indicators were mapped from the

Hoshin Plan, used by the organization to the BSC

proposed in the method. This mapping was done

through Strategic Objectives and Strategic Maps

documents. The sources of the data were the Hoshin

Plans of the IT, PMO and GPQT (Global Process,

Quality and Tools) areas of the studied organization.

5.2 Projects and Systems Level

The method adopts cycles well spread to the creation

of processes; this basis the application of this level

was easy and direct. But it was noted that the

method is not efficient when it comes to defining

execution roles to the creation of process’ life-

cycles. This facility however, does not mean that the

method can help in a practical way the organizations

which are commencing the definition of its own

processes’ framework once the studied organization

has already been working this way.

5.3 Continuous Improvement Level

Though the analysis flow of measures and indicators

the proposed method contributes straight with this

area. The organization which was object of studies

had all the information needed to this level, although

they were not well organized in a way that

everybody could have access. The improvement

projects, that use a specific program called BPI,

could be perfectly mapped to the proposed methods,

because they meet all Six Sigma requirements.

6 CONCLUSIONS

We can note that the integration between continuous

improvement initiatives used in one organization can

help the implementation of a quality program. The

different efforts that arise with the objective to

improve products, processes e services quality need

to be connected. These actions need to have as a

main focus on the business process improvement of

each organization.

Along with BSC, software measures were the

quantitative part of the indicators, it means, what we

want to measure. Those measures will naturally be

aligned to the strategic objectives of the organization

once they will be created from performance

indicators’ definitions. This procedure eliminates

efforts to collect and work information that are not

import for the organization.

After it is given a definition to what type of

information it is considered important to the

organization it is necessary to define which process

will give origin to the data that will feed the

indicators.

Once this process is defined, the indicators will

give to the organization the possibility to realize and

choose improvement fields based on trustable

information. Using the methodology Six Sigma, it is

possible to make an analysis of the processes, work

the causes that affect them and control the process.

REFERENCES

Basili, V. R. The Goal Question Metric Approach.

Available at:

ftp://ftp.cs.umd.edu/pub/sel/papers/gqm.pdf

Chrissis, Mary Beth; Konrad, Mike; Shrum, Sandy.

CMMI Guidelines for Process Integration and Product

Improvement. Addison-Wesley, 2003, 664pp.

Goethert, Wolfhart; Fisher Matt. Deriving Enterprise-

Based Measures Using the Balance Scorecard and

Goal-Driven Measurement Techniques. Institute

Carnegie Mellon University. October 2003.

Kulpa, Mararet K. Interpreting the CMMI: a process

improvement approach. Estados Unidos: Auerbach,

2003, 1º Edição, 414 pp.

INTEGRATION METHOD AMONG BSC, CMMI AND SIX SIGMA USING GQM TO SUPPORT MEASUREMENT

DEFINITION (MIBCIS)

175

Niven, P.R. Balance Scorecard step-by-step. John Wile &

Sons. 2002.

Pickerill Jay. Implementing the CMMI in a Six Sigma

World. Institute Carnegie Mellon University.

Available at:

http://www.sei.cmu.edu/cmmi/adoption/pdf/pickerill.p

df.

Pressman, Roger S. Engenharia de Software. São Paulo,

McGraw-Hill,, 2001, 5ª Edição.

Rocha, Ana R.; Maldonado, José; Weber, Kival.

Qualidade de Software - Teoria e Prática. São Paulo,

Prentice Hall, 2001, 303pp.

Siviy, Jeannine M; Forrester, Eileen C., Using Six Sigma

to Accelerate the Adoption of CMMI for Optimal

Results. Institute Carnegie Mellon University.

Available at:

http://www.sei.cmu.edu/cmmi/adoption/comparisons.h

tml. 2004

Sommerville, Ian. Engenharia de Software. São Paulo,

Addison Wesley, 2003, 592 pp.

Vasques, Renato. What does "High Maturity" Really

Mean? An Integrated View from Best practices: BSC,

CMMI and Six Sigma, SEPGLA, México, 2005.

ICEIS 2009 - International Conference on Enterprise Information Systems

176