Towards a Model for Managing

Success Factors in Software

Process Improvement

Joseph Trienekens

Eindhoven University of Technology Den Dolech 2

5600 MB Eindhoven The Netherlands

Abstract: Existing software engineering and management literature argue for

many different ways of looking at success factors that influence Software

Process Improvement (SPI). This study investigates success factors identified in

a number of research projects and proposes a model for managing these factors

in a structured way during different phases of an SPI program. It also aspires to

describe a model that reflects the cyclical influences of factors in a continuously

progressing SPI program.

1 Introduction

Many factors seem to influence the effectiveness of Software Process Improvement

(SPI) activities. These factors can be of different types, such as organizational (e.g.

commitment of management), human (e.g. resistance), technical (e.g. lack of tools),

financial (restricted budget). Over the years various papers have been written about

success factors of organizational change in SPI.

Although many articles look at factors that drive success in Software Process

Improvement activities, they look at them from different perspectives and with

different definitions [2]. Some articles look at the factors as prerequisites “written in

stone”, while other articles describe them as factors that are possible to influence

through various management actions. There is consequently a form of chaos prevalent

in the area, and a lack of overview of what drives success in organizational change

under SPI. The opportunity to try to create an overview of this field is the area of

research that this article will focus on. The purpose of this study is to enhance the

understanding of SPI implementation success by investigating various sets of factors

found in literature that are argued to have an impact on the outcome of organizational

change. Many research projects have dealt with this topic but there is not a high

degree of similarity between all these studies [3].

For this paper we selected four articles [1], [4], [5], [6], looking at various aspects

of successful SPI. The articles are based on case studies as well as a large number of

surveys. The surveys in these articles also contain interviews of managers and

practitioners. Several factors have been found and these factors can be used as

guideline in the SPI program.

This paper is divided in two parts. Firstly we compare and conclude to the success

Trienekens J. (2004).

Towards a Model for Managing Success Factors in Software Process Improvement.

In Proceedings of the 1st International Workshop on Software Audits and Metrics, pages 12-21

DOI: 10.5220/0002661200120021

 SciTePress

factors that are found in surveys and related literature. In the second part we present a

model for SPI success factors that reflects the cyclical influences of factors in a

continuously progressing SPI program.

2 Approach

Articles have been selected that reflected upon different aspects of drivers for success

when implementing Software Process Improvement. The paper collection is based on

literature, articles, surveys and experiences from different scientific databases. The

four papers that have been analyzed address respectively success factors:

• that affect organizational change [6].

• on different levels of maturity [4].

• in large and small organizations [1].

• that affect software processes [5].

This paper will address first the key findings of the different articles and the most

important coherences as well as contrasts between the articles. Secondly, a model will

be introduced for managing success factors during SPI implementation. The paper’s

conclusions can be considered as suggestions for companies to decide whether or not

an organization is ready to embark on a journey of Software Process Improvement,

and if not, what criteria must be improved in the organization if it should have any

significant change of becoming successful in the improvement process.

3 Investigating SPI success factors

3.1 Success factors that affect organizational change

In [6] ten factors are described, which can either positively or negatively affect

organizational change in Software Process Improvement (SPI). The article is based on

surveys in 56 software organizations that use an ISO 9000 quality system or the

Capability Maturity Model (CMM) as a basis for process improvement. A ranking of

the ten factors was created based on the frequency of them being mentioned in

published experience reports and case studies. The factors ranked in order of

decreasing importance are as follows:

Management Commitment and Support

This factor concerns the degree to which management in all company levels

sponsor the change.

Staff Involvement

Staff involvement addresses the amount of staff members that participate in

improvement activities.

Providing Enhanced Understanding

The degree to which knowledge of current software processes is acquired and

diffused across the organization is addressed by this factor.

Tailoring Improvement Initiatives

This factor is based on the need to customize and adapt a SPI program for local

needs as well for the departments’ own strengths and weaknesses.

Managing the Improvement Project

This concerns how effectively the improvement efforts are planned and controlled.

Change Agents and Opinion Leaders

This factor addresses the people that initiate and support the improvement

activities at corporate level, and those who are opinion leaders at the local level.

Stabilizing Changed Processes

Once improvement has taken place, this factor describes the organization’s ability

to maintain the improved state and not fall back into old habits.

Encouraging Communication and Collaboration

This factor concerns how communication efforts between different teams and

departments are important for SPI activities.

Setting Relevant and Realistic Objectives

The subject is how goal setting must both be realistic and for the overall benefit of

the organization.

Unfreezing the Organization

The lowest ranked factor concerns the inner resistance in an organization that SPI

activities must face before embarking on a SPI program.

As a conclusion the paper states that neither ISO 9000 nor CMM in itself provide

adequate support on how to implement the elements of an SPI program. Further it is

recommended that the success factors should be used as guidelines to prepare an

organization for a Software Process Improvement initiative.

3.2 Success factors on different levels of maturity

Many researchers have studied factors that influence Software Process Improvement

(SPI) positively or negatively. However not much attention has been given to the

impact of success factors at different levels of maturity. In [4] 16 factors have been

identified, respectively “inspections”; “reviews”; “standards and procedures”;

“project post mortems”; “metrics”; “risk assessment”; “estimating tools”;

“automation”; “training and mentoring”; “reward schemes”; “internal leadership”;

“internal process ownership”; “executive support”; “experienced staff”; “external

consultants”; and “stringent controls”. The factors “reviews”, “standards and

procedures”, “training and mentoring” and “experienced staff” have, according to the

authors, a major impact on SPI success. For two factors, “estimating tools” and

“reward schemes”, the majority opinion was that these factors do not have an impact

on SPI success. Two factors “reviews” and “training and mentoring” in both lower

and higher mature companies have a major impact on SPI success. For differences in

factors relating to SPI success, only the factor “training and mentoring” was

mentioned.

To summarize, the most prominent factor is “training and mentoring”.

Respondents think that “reviews”, “standards and procedures”, “training and

mentoring” and “experienced staff” have a major impact on SPI success. “reviews”

and “standards and procedures” are related to process. “Training and mentoring” and

“experienced staff” are related to skills. Therefore, the authors suppose that an

important element of SPI is process improvement in conjunction with skill

development.

3.3 Success factors in small and large organizations

Presumably there are differences between small and large organizations regarding the

way they implement software process improvement (SPI) programs in order to

advance their businesses. However in [1] it is shown that there is no difference in the

level of success between small and large software organizations. More important is

that large successful and small successful organizations differ fundamentally in their

approach to SPI, especially with respect to participation and the preferred mode of

learning. In small successful organizations the employers participate and explore new

knowledge more than in large successful organizations. The main difference between

small and large software organizations is the way in which they react to unstable and

changing stimulus situations. Small software organizations in turbulent environments

require learning strategies that are more closely aligned with explorative behavior. So,

this kind of SPI strategy is based on improvisation. Large software organizations

relied on learning from experience to prepare future rather than exploring new

possibilities. They kept doing what they did well, rather than risk failure. But on the

other hand, software businesses must be able to turn unexpected problems and failure

into learning opportunities; successful SPI requires tolerance for failure.

The size of the organization does not limit its potential for SPI success. This

means that software companies of any size can advance their businesses by practicing

a critical set of SPI elements. Small companies can and do implement SPI elements as

effectively as large organizations and achieve high performance. Large successful

software organizations emphasize exploitation of their best practices through formal

procedures, process models, guidelines, rules and checklists in order to manage and

improve their software process. Small successful organizations pay more attention on

exploring new possibilities, making the most of diversity and creativity of the human

resources involved in software process. To be successful, formal processes must be

supplemented with informal, inter-personal coordination about practice.

3.4 Success factors affecting software processes

Even though numerous studies have been done there is still a great deal of variability

in the success of SPI programs and often it is so that specific success factors don’t

offer an explanation. The aim of the research in [5] is to get insight into the factors

that practitioners think to affect software processes. This study includes many surveys

of practitioners’ opinions and the investigation was primarily focused on the research

question: What factors, as identified by case studies, affect or don’t affect software

processes?

According this study two main reasons exist which relate to variability of success

with SPI. These main reasons are the factors that affect software processes and the

research strategies that are used to investigate these factors. The importance for seven

factors that affect software processes was investigated across a survey study and

multiple cases study. These seven factors are respectively: “executive support”,

“experienced staff”, “internal process ownership”, “metrics”, “procedures”, “reviews”

and “training”. In accordance with the study there were two factors, respectively

“reward schemes” and “estimating tools”, which were shown not to be relevant to

SPI. Four factors were identified by the survey, but not by the case studies. These

factors are: “inspections”, “internal leadership”, “standards” and “mentoring”. In the

next section we use Table 1 to compare dimensions and viewpoints on success

factors for SPI.

Table 1: Comparing dimensions and viewpoints on success factors for SPI

Stelzer and

Mellis (1999)

Rainer and Hall

(2002)

Dybå (2003) Rainer and

Hall (2003)

Problem Area

Backgrounds

Neither CMM

nor ISO 9000

provides

support on

how to

implement

organizational

change in SPI

There was not

yet any study

performed about

the impact of

factors at

different levels of

maturity.

There has been

no attempt to

verify whether

small and large

organizations

implement SPI

programs

differently in

order to

advance their

businesses

Despite of

many studies,

there is still a

great

variability in

the success of

SPI programs,

whose reasons

cannot be

explained

Article

Objectives

Give help on

how to

implement

SPI though

identifying

success

factors,

comparing to

research and

examining the

generality of

the factors

Determine

factors that have

an impact either

positively or

negatively on

SPI of low and

high maturity

companies

Investigate

whether an

organization’s

size affects its

SPI

implementation

strategy and

the degree of

SPI success

Get insight into

the factors that

practitioners

think affect

software

processes

Consider

research

strategies used

to study these

factors

Research

Methodologies

Factors were

identified by

the authors

Empirical

research of

factors

mentioned in

case studies

for ranking

purposes

Interviews

with German

software

managers

16 factors were

identified.

Through the use

of a

questionnaire, the

impact of each

factor was

determined.

Empirical

research

through the use

of a

questionnaire

sent to 120

software and

quality

managers in 55

companies

Empirical

research

though usage

of case studies

and interviews

with

practitioners

Usage of

quantitative

and qualitative

analyses

Results of the

Articles

Ten factors

that affect

organizational

change in SPI

Factors are

the same for

both CMM

and ISO 9000

Similarities in

factors

between all

industries

Factors that have

a major impact

and factors

having no impact

on SPI were

found.

Four broader

themes emerge:

people, process,

skills and

leadership.

The size of an

organization

does not limit

its potential for

success

Small should

capitalize on

employee

participation

and exploration

of new

knowledge

Factors that

contribute to

successful SPI

implementation

A multi-

strategy

approach is

necessary for

an improved

understanding

of SPI

Application of

the Results

Although

factors may

seem self-

evident, they

can be used

as a guideline

before

implementing

SPI

Offer

recommendations

to practitioners of

different maturity

level companies

on SPI.

Provide new

understandings

and enhance

relative

strengths for

managers of

software

companies

Give

practitioners a

set of success

factors they

can use when

implementing

an SPI program

3.5 Discussion on the investigated success factors

Table 1 indicates the main points of the articles for the aspects: problem area

backgrounds, article objectives, research methodologies, results of articles, and

possible application of the results. In the next subchapters a discussion of the

differences and similarities is provided for each of the aspects of the articles.

3.5.1 Differences in problem area backgrounds. As can be seen the backgrounds

for the articles differ substantially. In [6] research has as starting point an idea that

neither CMM nor ISO 9000 provides enough support on how to implement

organizational change in SPI. In [4] a lack of description is identified of what drives

success within SPI for different levels of maturity of the organization. In [5] the

authors argue that there still is too much variability in the success of an SPI program

that cannot be accounted for. In [1] a totally different approach is followed by making

a distinction between success factors for large and for small organizations.

3.5.2 Differences in objectives. In [6] the objectives were to investigate how to

implement SPI though dentifying success factors. In [4] research strategies are

considered, which can be used to study these factors. In the research they determined

factors that have an impact either positively or negatively on SPI implementation

success of companies with both low and high levels of maturity. The authors also in

their second article [5] want to get insight into factors, which practitioners “in real

life” think to affect software processes. In [1] the author seeks to investigate among

others whether an organization’s size affects its SPI implementation strategy and the

degree of SPI success.

3.5.3 Differences in research methodologies. In [6] the authors identified factors in

their survey by themselves. They used empirical research of factors mentioned in case

studies for ranking purposes. They also interviewed German software managers for

background information. In [4] a questionnaire has been used to provide the

researchers with data on factors influencing SPI implementation success. The authors

identified 16 factors based on previous research. The impact of each factor was

determined from the results of the questionnaires. Their second article [5] used

quantitative and qualitative analyses, and they compared results of the case studies

with the results of the surveys.

3.5.4 Differences in research results. In [6] the research resulted in 10 factors that

affect organizational change in SPI. The factors found were the same for SPI

programs based on both CMM and ISO 9000. They also noticed that there are

similarities of factors in all industries. In [4] the authors found factors that have either

a major impact or no impact on SPI implementation. In their second article [5], they

found many factors, some of these were based on survey and case studies and others

were not. They found two main reasons that exist which relate to variability of

success with SPI, namely: factors that affect software processes and the research

strategies that are used to investigate these factors. They got 26 factors as results that

potentially affect SPI. They also noticed that a multi-strategy approach is necessary

for an improved understanding of SPI. In [1] the author investigated organizations and

found as results that the size of an organization does not limit its potential for success.

Small organization should capitalize on employee participation and exploration of

new knowledge.

3.5.5 Application of the results.

In [6] the authors underline that although factors

may seem self-evident, they can be used as a guideline before implementing SPI. In

[4] also recommendations are offered to practitioners of different maturity level

companies on SPI. Based on [5] practitioners are able to get a set of success factors

they can use when implementing an SPI program. In [1] the author also offers new

understandings and enhance relative strengths for managers of software companies.

4 A model for managing success factors in software process

improvement

This section will present a model for managing factors that influence the outcome of

SPI efforts. First of all, the factors presented in the different articles have a tendency

to be based on different viewpoints. Articles have looked upon the success factors as

organizational prerequisites, which, if not fulfilled, make the SPI program pointless.

Other articles have looked at the factors that can actually be influenced by the

management. Therefore a combination of these two main dimensions is suggested,

and the model will present the factors grouped into different “areas of influence”. The

graphical representation of the model is depicted in Figure 1.

Many of the factors influence the SPI program efforts in such a way that the

output of the SPI program in its turn will influence the factors again. This argues in

favor of a cyclical model, where the initial factors need to provide the SPI efforts with

the necessary momentum to succeed. In the following sections, the various factors

found under each area of influence will be described and discussed. Since this is a

model of drivers of SPI program success, a network structure has been created to

explain influences rather than by providing a checklist of factors to check against the

organization in focus.

Figure 1: The cyclical model for managing success factors in an SPI program

The arrows represent influence between different stages both in requirements of

what has to be done before taking on the next step, but also in what conditions the

later steps have to be adapted to.

4.1 Organizational prerequisites

The organizational prerequisites are a set of factors that are very difficult to manage

directly for the firm and are also not subject to change as of influence from previous

cycles in this model. Some of these factors can make embarking on a SPI program

extremely difficult. An example of such a factor is “experienced staff” by Rainer et al,

see [4], [5].

Other factors might merely affect in what way management tackles a situation. An

example here is “the size of the organization”, by Dybå [1], which affects what

strategy to use and what organizational strengths to build the processes on.

Organizational

prerequisites

Employee

coaching

Providing

organizational

infrastructure

Directly

managing the

SPI program

Setup stage

Execution

Results of the

SPI program

Feed-back

4.2 Employee coaching

Factors found under the domain of employee coaching are “staff involvement”,

“change agents and opinion leaders”, “unfreezing the organization” and “setting

relevant and realistic objectives”, see e.g. [6]. Another factor is “training and

mentoring”, introduced by Rainer et al [4], [5].. The reason behind this is that their

main goal is to give the employees the right momentum to embark on a SPI program.

4.3 Providing organizational infrastructure

These factors include “providing enhanced understanding”, “tailoring improvement

initiatives”, and “encouraging communication and collaboration”, see e.g. [6]. Other

factors are “reviews”, and “standards and procedures”, in [4], “internal process

ownership”, “metrics” and “procedures”, in [5]. The factors all provide the main

prerequisites to be performed for the employees to be able to take on the SPI initiative

and perform the improvement tasks well.

4.4 Directly managing theSPI program

The most important factor is “management commitment and support”, which in its

turn affects all the other domains more or less directly. This factor is emphasized in

multiple reports [6], [5]. The outcome of the previous steps of the setup phase also

results in the factors of how specifically to “manage the improvement project” and

how to “stabilize the changed process”, see e.g. [6].

4.5 Making use of the results of the SPI program

The outcome of the SPI program can be measured in various ways, and these results

can directly provide a guideline on both what factors to try to improve, but also

indirectly affect such things as “employee commitment”, since, if the program proves

successful, this is likely to increase the initial support given. Although the factors in

the setup stage have to be influenced prior to embarking on a SPI program, they

remain important after the program has started. The cyclical model constantly

revolves and there is a continuous flow of influences between the stages.

5 Conclusions

Because empirical studies on success factors and their results tend to flow together

and overlap, representative articles have been selected for discussion. These articles

have been discussed from different dimensions and viewpoints. Subsequently a

cyclical model for managing success factors influencing a software process

improvement has been presented. This model provides a structured way to handle

success factors while also dealing with the relationships between these factors. In this

model, four sets of factors have emerged, respectively Organizational Prerequisites,

Employee Coaching, Providing Organizational Infrastructure and Directly Managing

the Software Process Improvement Program. Next steps in our research will focus on

the validation of the presented cyclical model for success factors in practice, in

particular regarding the management of SPI programs and the development of

strategies for SPI implementation.

References

1. Dybå, T.: Factors of Software Process Improvement Success in Small and Large

Organizations: An Empirical Study in the Scandinavian Context, ESEC/FSE

(2003)

2. El-Emam, K., Goldenson, D., McCurley, J., Herbsleb, J.: Modeling the

Likelihood of Software Process Improvement: An Exploratory Study. Empirical

Software Engineering (2001)

3. Gandi, N., Hartikainen, V., Kuningas, I., Nilsson A.: Success Factors of

Organizational Change in Software Process Improvement, Internal Research

Report TU Eindhoven, The Netherlands (2003)

4. Rainer, A., Hall T.: Key success factors for implementing software process

improvement: a maturity-based analysis, The Journal of Systems and Software

(2002)

5. Rainer, A., Hall T.: A quantitative and qualitative analysis of factors affecting

software processes, The Journal of Systems and Software (2003)

6. Stelzer D., Mellis W.: Success Factors of Organizational Change in Software

Process Improvement, Software Process Improvement and Practice (1999)