Towards an Approach for Integrating Risk Management Practices into

Agile Contexts

Fernando Vedoin Garcia, Jean Carlo Rossa Hauck, Fernanda Narloch Rizzo Hahn and

Raul Sidnei Wazlawick

Department of Informatics and Statistics, Federal University of Santa Catarina, Florian

opolis, Brazil

Keywords:

Risks, Agile, Practices, Management.

Abstract:

Risk management has been perceived as important to assist in the delivery of software products that meet

the expected scope, schedule, and costs. In traditional plan-driven software development, risk management

includes identifying, analyzing, planning responses, monitoring, and controlling risks. In agile methods, how-

ever, risks are typically managed implicitly, through practices and values that tend to keep risks under control.

However, in many contexts, such as software development for highly regulated domains, such as health and

ﬁnance, this implicit risk management may not be enough. This paper presents an action research on the in-

troduction of explicit risk management practices in a healthcare software development organization that uses

agile methods. Based on identiﬁed risk management practices used in agile contexts, the research intervention

is planned, applied in two teams during three iterations and the collected data are evaluated and interpreted.

The results of the study raise evidence that the agile risk management practices adopted were effective in

identifying and mitigating risks and improving project results, without negatively impacting the team’s pro-

ductivity or the agile values adopted by the organization. Also as a result of the application of the study, a

possible approach to introduce risk management practices in agile contexts emerges.

1 INTRODUCTION

Risk management in software development projects

has historically been perceived as important, as soft-

ware projects are subject to various risks related to

scope, stakeholder expectations, estimates, technol-

ogy evolution and deadlines (Islam et al., 2014), (Ab-

delrafe et al., 2016).

Risk management is a well-established process

in the traditional plan-oriented software development

methods. These methods have adopted risk manage-

ment practices, such as risk identiﬁcation, qualita-

tive and quantitative analysis, response planning and

risk monitoring and controlling (PMI, 2021), (Inter-

national Organization for Standardization, 2009), (In-

ternational Organization for Standardization, 2010).

In agile methods, however, risk management is

usually treated implicitly since typical agile methods’

practices, such as small increments, work visibility

and expectations management, tend to optimize pre-

dictability and, thus, mitigating risks (Concha et al.,

2007), (Nyfjord and Kajko-Mattsson, 2007).

However, the absence of explicit risk management

practices has often led agile projects to fail (Dhir

et al., 2019), (Tam et al., 2020), (Tanner et al., 2014),

especially in certain software development scenarios

where the typical implicit risk management strategy

of the agile methods has not been enough to keep risks

under control (Hauck and Vieira, 2021) (Shrivastava

and Rathod, 2015), (Nelson et al., 2008).

This need to improve agile risk management, leav-

ing traditional and cumbersome processes aside and

without losing agile principles, has attracted recent

attention of researchers and practitioners, leading to

different approaches to adopting risk management

practices in agile methods (Afshari and Gandomani,

2022), (Ylimannela, 2012), (Godse and Rajiv, 2021).

This same need for risk management has been ob-

served in a software development organization that

uses agile methods and develops products for the

healthcare and educational domains. The organiza-

tion has more than 180 employees, serving 2.500

municipalities with more than 75.000 direct users in

Brazil. Due to the domains of application, that are

subject to possible risks with very high human im-

pact, two of the authors of this study, who work at the

organization, have perceived the need to integrate ex-

plicit risk management practices into the agile meth-

Garcia, F., Hauck, J., Hahn, F. and Wazlawick, R.

Towards an Approach for Integrating Risk Management Practices into Agile Contexts.

DOI: 10.5220/0012790200003753

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 19th International Conference on Software Technologies (ICSOFT 2024), pages 46-58

ISBN: 978-989-758-706-1; ISSN: 2184-2833

ods used in the organization.

Thus, our research question is ”How to integrate

explicit risk management practices in a software de-

velopment organization that uses agile methods?”. In

an initial investigation through a Systematic Mapping

Study (SLM) (Garcia et al., 2022), we did not ﬁnd

any speciﬁc guide that establishes an approach for in-

tegrating risk management practices into agile meth-

ods.

Based on the results of the previous SLM, we car-

ried out an action research following the approaches

proposed by (Petersen et al., 2014) and (Avison et al.,

1999). Initially, a diagnosis of the organization is car-

ried out, then the action planning is performed fol-

lowed by the design of a possible solution. The inter-

vention is then applied in the organization, data are

collected and analyzed and the lessons learned are

documented.

The main contributions of this study are twofold.

For researchers, we present an action research, report-

ing and discussing the main results and ﬁnally gener-

alizing the steps in an initial proposal of an approach,

contributing to the recent research on the adoption of

risk management with agile methods. For practition-

ers, we present a practical application that can serve

as inspiration for how to apply explicit risk manage-

ment practices in those scenarios where the implicit

risk management of the agile methods may not have

been enough.

The rest of this paper is organized as follows: the

next section presents related works (2), followed by a

section on methods (3), the action research planning

(4), action taking (5), evaluation (6) and specifying

learning (7). Finally, the conclusion section (8) sum-

marizes the key ﬁndings.

2 RELATED WORK

Given the importance of risk management, the inte-

gration of these practices into agile methods has at-

tracted the attention of researchers and practitioners

(Garcia et al., 2022). In this section we present works

that propose different strategies for this integration of

risk practices.

Brand

ao (Chiste Brand

ao, 2013), based on em-

pirical research, reported that a signiﬁcant number of

software companies do not regard risk management as

an established practice. Additionally, certain software

methods do not ofﬁcially incorporate risk assessment,

and instead rely on informal practices to mitigate risks

(Chiste Brand

ao, 2013). While in (Elkhatib et al.,

2022), agile or hybrid projects have been found to

carry less risk compared to waterfall projects. Also,

the research suggests several recommendations to re-

duce project risk, such as incorporating customer vi-

sion, selecting the appropriate project approach, hav-

ing an experienced project manager and team, involv-

ing customers in the project, obtaining organizational

policy support for an agile approach, promoting a sup-

portive culture, and utilizing analytical tools.

In (Afshari and Gandomani, 2022), authors have

proposed a risk management model in a hybrid ap-

proach, combining Scrum and XP. The model vali-

dated in a case study showed positive results by re-

ducing the number of duplications, change requests,

identiﬁed risks, and occurred risks. It also increased

team efﬁciency and eliminated risks.

In (Ylimannela, 2012), authors propose modiﬁca-

tions to the risks identifying approach by incorporat-

ing a risk matrix onto a risk board, utilizing red sticky

notes for potential risks and yellow ones for poten-

tial solutions. A checklist is used to identify high-risk

areas, giving weight to the security of the case, and

the checklist is added to the sprint backlog for mon-

itoring. Once a response has been implemented, the

note and risk are transferred to a new location on the

board. Any accepted risks are saved in digital format

for future re-evaluation. However, as helpful as it was,

authors found difﬁcult to decide on a high-risk feature

and there were issues with unnecessary bureaucracy.

Tomanek (Tomanek and Juricek, 2015) suggested

a risk management technique for the Scrum frame-

work utilizing the PRINCE2 approach. The authors

justiﬁed this combination by highlighting the process

oriented nature of both methods. The research found

that the most critical aspect of risk is its seriousness,

and the best moment to address risks is during Sprint

planning. The outcome of the study demonstrated

that incorporating Scrum with PRINCE2 approach re-

sults in enhanced product delivery time. However, the

study also highlighted that while incorporating risk

management and holding risk identiﬁcation meetings

in Scrum can be beneﬁcial, it may not be suitable for

all project types.

In summary, related works propose different ap-

proaches to risk management and provide insights

on the beneﬁts and challenges of implementing risk

management in agile software development projects.

However, none of the related works presented a

generic approach capable of effectively managing

risks across different agile contexts, as they, sepa-

rately, lacked support for various risk management

processes and domains. Thus, different from related

works, this paper presents an action research study

covering different risks practices based on literature,

systematically selecting, applying and evaluating the

most suitable ones and, based on the results of the

Towards an Approach for Integrating Risk Management Practices into Agile Contexts

study, pursuing a possible general approach for the

incorporation of explicit risk management practices

into software organizations that use agile methods.

3 RESEARCH METHOD

This study adopts action research as its methodolog-

ical approach. Action research is an iterative process

that involves researchers and practitioners collaborat-

ing on a particular set of activities, such as problem

diagnosis, action intervention, and reﬂective learning

(Avison et al., 1999).

Based on the organization’s context we deﬁned

our main research question as ”How to integrate ex-

plicit risk management practices in a software devel-

opment organization that uses agile methods?”.

Initially, in order to understand how primary stud-

ies have been reported the integration of explicit risk

management practices into agile methods, we carried

out a Systematic Mapping Study (SLM) (Garcia et al.,

2022). Thus, based on the needs identiﬁed in the or-

ganization and the experiences gained from the SLM,

we carried out this action research study following the

steps proposed by (Petersen et al., 2014) and (Avison

et al., 1999). Figure 1 presents the steps followed in

this study.

Figure 1: Study methodological approach.

As shown in Figure 1, after performing the SLM,

we started the ﬁrst step with a diagnosis to understand

and describe the problem to be solved. In the next

steps, we carried out the planning, diagnosis and de-

sign of the action, where we studied alternatives for

solving the problem and choose the most appropriated

ones. The next step was to carry out the intervention

action itself, where the plan is executed. Next, we

carried out the evaluation of the effects of the action

through the application of structured interviews with

the participants. Finally, the learning gained through

the evaluation of the intervention was documented

as an organization’s draft approach to integrate risks

management practices. These methodological steps

are presented in detail in the next sections.

This study was approved by the Human Research

Ethics Committee of the Federal University of Santa

Catarina number 55851622.8.0000.0121.

4 ACTION RESEARCH

PLANNING

Following the adopted methodological approach, the

ﬁrst step is to plan the action research. The planning

of this study consists of deﬁning the objective of the

action, deﬁning a schedule and allocating the organi-

zational resources needed to carry out the study.

The main objective of the study is derived from the

research question (Section 3): ”To introduce explicit

risk management practices in the context of the target

organization that uses agile methods”.

This study was planned to be carried during three

months involving two of the organization’s develop-

ment teams. The selection criteria for these teams

were: (i) the relevance of the software products de-

veloped by the teams, which are the most important

for the organization; (ii) the proximity to one of the

authors of this study who works in one of the teams;

(iii) the participants’ availability.

The teams involved in this study and their char-

acteristics are presented in Table 1. The two chosen

teams are treated from now on as Team A and Team B.

A more detailed analysis of this context is presented

in the next section.

Table 1: Teams’ characteristics.

Team A Team B

Agile method Scrum Scrum

Team size 6 10

Application domain Healthcare Education

Product type Web Mobile

4.1 Diagnosis

This step consists of understanding and explaining the

problem. Thus, we conducted an initial diagnosis of

the organization’s context and, based in this context,

we formalized the problem deﬁnition.

4.1.1 Context

As already mentioned, the target organization of this

study is a software development organization that de-

velops web and mobile systems for the healthcare and

education domains, reaching thousands of users in

Brazil. With currently 180 employees, the organiza-

tion has 15 development teams working on 3 different

software products.

ICSOFT 2024 - 19th International Conference on Software Technologies

All software development teams in the organiza-

tion are using the agile Scrum method as the main

reference for their software process. However, each

team is free to adapt the method according to their

needs. Below, for a better understanding of the con-

text, the software process of the teams involved in this

study is brieﬂy presented.

Teams work in a hybrid work regime, with some

members working asynchronously most of the time.

They use the Slack tool

to facilitate asynchronous

communication and the Google Meet tool

for syn-

chronous interactions. Teams use GitHub

for source

code management. In addition, they use Google

Drive, Docs and Sheets

to record goals, activities

and other types of shared materials.

The teams use iterations of two weeks-long

Sprints that are initiated with a Sprint planning cere-

mony, which takes place on the ﬁrst day of the Sprint.

Both teams maintain a history of the past Sprints so

that the total effort on selected tasks does not ex-

ceed the team’s development capacity. The roles of

Product Owner (PO) and Scrum Master (SM) are

maintained in both teams (Schwaber and Sutherland,

2020).

After the planning ceremony, the Sprint execution

begins, carrying out the selected tasks. During the

Sprint execution, Daily Meetings are performed, aim-

ing to follow the task’s progress as well as to remove

impediments.

During the Sprints execution, the integration

among the organization’s teams is carried out through

weekly Scrum of Scrums (Sutherland, 2021) cere-

monies, meetings where only the SMs of each team

participate to follow the progress of the Sprints.

At the end of each Sprint, the Sprint Retrospective

and the Sprint Review ceremonies are held, following

Scrum practices (Schwaber and Sutherland, 2020).

The organization also uses the Agile Wheel tech-

nique (Soares, 2017) as a continuous improvement

approach, periodically holding meetings to assess the

progress of the teams.

4.1.2 Problem Deﬁnition

As it can be seen in the previous section, currently

no explicit risk management practices are carried out

by the participant teams. Thus, given the software

products domains (education and especially health-

care) which leads to possible serious impacts of the

materialization of risks, the need to make risk man-

https://slack.com/

https://meet.google.com/

https://github.com/

https://docs.google.com/

agement explicit was perceived by the two authors of

this study who work at the organization.

The main problem, however, emerges from the

need to introduce risk management practices with-

out losing the values of the agile methods adopted by

the organization, such as adaptation to change, short

delivery cycles, open and frequent communication,

which are very important for the organization’s cul-

ture.

4.2 Selection of Solution Alternatives

The analysis of problem solution alternatives begins

with the identiﬁcation of risk management practices

that have been adopted by organizations that use ag-

ile methods. The most used risk management prac-

tices were identiﬁed from the initial Systematic Map-

ping Study (Garcia et al., 2022). The complete list of

identiﬁed agile risk management practices is available

at: http://bit.ly/3pecgSF. As many risk management

practices have been identiﬁed in previous research,

and that we do not wanted to create new practices, in

order to choose alternatives for the problem solution,

we previously selected four practices using as criteria:

• Practices applied in similar contexts (organization

size, agile method or product domain);

• Practices with reported successful results of appli-

cation;

• Practices with a minimal description sufﬁcient to

be applied;

• Practices that can cover risk identiﬁcation, regis-

ter, analysis or monitoring, when possible;

• Practices that are aligned with the values of the

agile manifesto.

Based on these criteria, the pre-selected practices

are deﬁned below, accompanied by a brief description

as present in the primary study that proposed it.

• Risk Brainstorming: This practice is used to

awaken ideas and creative thoughts. For the real-

ity of risk management, this practice helps mainly

in identifying risks, and can be performed in a

structured way, in the form of an interview, in a

coordinated session or in a free form, allowing

everyone to discuss the risks of the project (Ham-

mad and Inayat, 2018).

• Risk checklist: This practice allows a starting

point to identify risks, using checklists developed

from the experiences of other projects, listing the

main risk categories and aspects (Ghobadi and

Mathiassen, 2017). The list of possible risks to

populate this checklist was extracted from those

Towards an Approach for Integrating Risk Management Practices into Agile Contexts

risks identiﬁed in the SLM (Garcia et al., 2022) as

most common in agile methods. The complete list

of identiﬁed risks is available at: bit.ly/3pecgSF.

• Risk matrix: This is a practice to aid in risk anal-

ysis considering the criticality of a risk, in which

the x-axis is cost/impact and the y-axis is the prob-

ability of occurrence. We can evaluate each risk

with the matrix, where a classiﬁcation is deﬁned

for each threat and axis, the greater the severity.

In terms of colors, red, yellow and green indicate

high, medium and low priority risks, respectively.

With the union of the numerical value and the

color of a risk, criticality is determined (Cuong

et al., 2019).

• Risk Register: When a project is already contrac-

tually agreed between the interested parties, a risk

to the threats and opportunities identiﬁed, ana-

lyzes and response strategies, recorded in a table

(Alharbi and Qureshi, 2014).

4.3 Data Collection Planning

The Goal Question Metric (GQM) (Koziolek, 2008)

approach was used to plan the data collection for this

action research. The GQM approach deﬁnes mea-

surement objectives and derives questions and met-

rics that should be collected to reach those objectives.

The measurement objectives (MO) of this study are

derived from the research question and the main ob-

jective, as presented. Thus, planning the study data

collection, measurement objectives and related ques-

tions are presented bellow.

MO1: Analyze the team’s effort during the appli-

cation of risk management practices, from the point of

view of the software development teams in the context

of the target organization.

• Q1.1: What is the total effort spent by the team

members to apply risk management practices?

MO2: Analyze the applicability of risk manage-

ment practices in agile teams at the organization, from

the point of view of the software development teams

in the context of the target organization.

• Q2.1: Are risk management practices easy to un-

derstand and apply?

• Q2.2: Was the experience of applying the prac-

tices beneﬁcial to the participant teams?

• Q2.3: What were the main difﬁculties in applying

risk management practices?

• Q2.4: Was there an impact on the team’s velocity

when applying risk management practices?

MO3: Analyze the acceptance of risk manage-

ment practices, from the point of view of the software

development teams in the context of the target organi-

zation.

• Q3.1: Does the team intend to continue using the

presented practices? If yes, why?

• Q3.2: What were the main results in the applica-

tion of risk management practices?

For each question, following the GQM approach,

metrics were also deﬁned. For the objective OM1,

the effort is collected through a spreadsheet already

used by teams to register their activities and using an

online stopwatch tool during the meetings. For data

collection of OM2 and OM3 metrics, we developed a

questionnaire containing all the derived questions to

be applied at the end of the intervention. The ques-

tionnaire form is available at https://bit.ly/4aj9PjV.

5 ACTION TAKING

This section describes the execution of this action re-

search as deﬁned in the planning section.

Initially we started by motivating team members

to participate in the study, clearly stating how the

study would be conducted and what problem it was

intended to solve. This motivation was carried out

through a meeting with each of the teams.

Then, based on the four practices pre-selected

from the literature, both Team A and Team B were

involved in selecting the practices that seemed most

appropriate for the reality of the teams. This step is

presented in Section 5.1.

After that, tools were developed to facilitate the

recording and monitoring of identiﬁed risks. This step

is presented in Section 5.2.

Finally, the selected practices were applied by

both Team A and Team B during the planned period,

covering six and four Sprints performed by each team,

respectively. Details of this application are presented

in section 5.3.

During the execution of this study, the interven-

tion was monitored by the authors in order to verify

the impact and collect data regarding the metrics de-

scribed in Section 4.3.

5.1 Selection of Risk Management

Practices

The initial step is to choose the risk management prac-

tices that best meet the needs of the organization and

that are better aligned with the organizational context.

ICSOFT 2024 - 19th International Conference on Software Technologies

As already presented, a list of the most used risk

management practices in agile contexts was extracted

from a SLM carried out prior to this study.

In order to involve the teams in the selection of the

risk management practices to be used, a meeting was

held with two members in leadership positions from

each team. In Team A the PO and Tech Leader par-

ticipated, in Team B the members involved were PO

and SM. In the ﬁrst moment of the meeting, we pre-

sented the selected risk management practices with a

brief explanation of each one. Afterwards, an open

discussion took place among the team members to

clarify any doubts and make informed decisions about

which practices would best suit their speciﬁc context.

This decision-making process was based on the team

members’ experiences and considerations of the cur-

rent ceremonies that accompany the development cy-

cle, ensuring that the chosen practices could be seam-

lessly integrated into their existing workﬂow. In ad-

dition, the team members took into consideration the

project constraints while choosing the practices. They

evaluated how the selected practices aligned with the

project’s speciﬁc limitations, such as time, resources,

and budget. This consideration ensured that the cho-

sen practices were not only suitable for their context

but also feasible and practical within the given project

constraints.

From that meeting, the two teams selected the

Risk Checklist and Risk Matrix practices. Brain-

storming was discarded as it required an additional

synchronous ceremony for the risk identiﬁcation,

which would cause signiﬁcant time loss. Also, the

Risk Register was not selected because it required the

creation and implementation of detailed action plans,

which was considered by the participants as possi-

bly compromising the team’s agile values, by requir-

ing additional documentation and possibly impacting

their productivity.

After the meeting, the selected practices were pre-

sented to the teams by the leaders in an informal meet-

ing, so that they could be validated. Then, both teams

agreed with the implementation of the chosen prac-

tices.

5.2 Development of Tools for Risk

Management

After deﬁning the adopted practices, we created a

facilitating tool for the application of the practices.

Both teams wanted at least part of the implementa-

tion of the practices to be asynchronously so that the

team’s productivity was not compromised.

Thus, we created a simple tool on Google Forms

for each team, which is mapped to a table in Google

Sheets. This form implements the Risk Checklist

previously populated with the list of most common

risks reported in the literature (Garcia et al., 2022)

and provides an open ﬁeld for members to ﬁll in other

identiﬁed risks. In this way, it was possible for the

teams to identify risks asynchronously without the

need to use or create a ceremony for this purpose. Fig-

ure 2 shows the deﬁned form.

Figure 2: Risk Checklist form.

Once identiﬁed in the form, the risks are auto-

matically allocated in a line on the spreadsheet and

can then be analyzed later according to the practice

selected by the team. This spreadsheet supports the

Risk Matrix practice, as in addition to the risk de-

scription, the worksheet also allows team members

to rate the risk in Impact (“Insigniﬁcant”, “Moder-

ate” and “Catastrophic”) and in Probability (“Low”,

“Medium” and “High”). Figure 3 shows the spread-

sheet model used by the teams. The Risk Matrix be-

low was also made available so that teams can deﬁne

risk priority.

Figure 3: Qualitative matrix.

In order to facilitate the use of the tools, we

recorded a video with a brief explanation on how to

use the tools. In the video, we presented the selected

risk management practices, the form to be completed

by the team members, and the spreadsheet to which

the risks were mapped, analyzed and monitored.

5.3 Application of the Risk

Management Practices by the

Teams

After the beginning of the applications of the prac-

tices in both teams, during the next three Sprints of

Towards an Approach for Integrating Risk Management Practices into Agile Contexts

two weeks each, the participants were free to apply

the chosen risk management practices in the way they

considered most appropriate for the context of the

team.

Both teams opted to perform risk analysis during

the last Daily Scrum ceremony at the end of each

Sprint. The identiﬁcation of risks occurred recur-

rently during the Sprint duration. One of the authors

of this study accompanied the teams during the cere-

monies clearing everyone’s doubts.

Next, some speciﬁcities of the application of prac-

tices in each of the teams are presented.

5.3.1 Application in Team A

In Team A, the explanatory video was made available

on the Slack tool team channel. Together with the

video, the tools for identifying and analyzing risks

were also made available.

Team A form received 6 responses during the ﬁrst

Sprint of applying the Risk Checklist identiﬁcation

practice and, according to the participants, each mem-

ber responded to the form only once.

The risks identiﬁed by the team were mostly those

already present in the support list. With greater ap-

pearance of risks ”Complex project” and ”Depen-

dency on legacy technologies”.

After identifying the risks, in the following Sprint,

the participants carried out the risk analysis and mon-

itoring through the application of the Risk Matrix

practice. This process was carried out synchronously

during a speciﬁc synchronous online meeting after the

Daily Scrum ceremony.

The discussion lasted for 53 minutes. During this

process, the team evaluated each of the identiﬁed risks

and attributed a value to Impact and another to Prob-

ability, in order to calculate the Priority based on the

Risk Matrix. The prioritization result is compiled on

bit.ly/3NG8uvD.

Team A scored 6 high priority, 7 medium, and 3

low priority risks. In addition, a risk that was ﬁlled

in using the “Other” ﬁeld on the form was not under-

stood by the team and, therefore, was not assessed.

In addition, some identiﬁed risks were later taken

to the Agile Wheel (Soares, 2017) ceremony to be dis-

cussed in greater depth. This culminated in the spon-

taneous creation of an action plan by the team to im-

prove asynchronous communication between mem-

bers, the plan was derived from the risk “Physical dis-

tance between team members”.

5.3.2 Application in Team B

Team B also had the explanatory videos and tools re-

leased, as well as in Team A. However, at ﬁrst, the

team had difﬁculties in identifying risks. The ﬁrst an-

swers were obtained only during the second Sprint

of the application and only one risk was identiﬁed:

“Bureaucratic and centralized organizations”. In all

other responses, members reported not having identi-

ﬁed risks.

However, during the third Sprint, new answers to

the form were obtained, which included both risks

present in the checklist and risks identiﬁed individu-

ally by the participants. The most prominent risk was

”unpredictability on the part of the customer”.

Some risks obtained needed to be adjusted for the

spreadsheet, as they contained more than one risk de-

scribed in the free text ﬁeld. After this adjustment, the

team prioritized risks using the Risk Matrix practice.

Prioritization was also carried out asynchronously, in

the case of this team the PO individually collected

opinions about each of the risks through Slack and

then ﬁlled in the table with the results. Introducing

them later to the team. The result of the application is

available on bit.ly/3NG8uvD.

6 EVALUATION

In this step, the effects of the action are captured and

analyzed. First, we collected data during all Sprints

in both teams. Then we analyze each of the Measure-

ment Objectives based on the collected data.

6.1 Data Collection

Data collection was carried out throughout the dura-

tion of this study, which lasted three months. For the

collection of quantitative data, the tools already used

by the teams were used, including follow-up work-

sheets to collect the effort applied during the Sprints.

Finally, as the practices were applied during team cer-

emonies, it was possible to record the time spent dur-

ing the study using a digital stopwatch to measure the

precise effort spent.

A questionnaire with discursive questions was

also designed to collect qualitative data during imple-

mentation, as mentioned in section 4.3. Furthermore,

the artifacts generated by the two teams during the

application were also collected and analyzed.

The questionnaire was sent at the end of the study

to all team members. A total of 11 responses were re-

ceived, 6 from Team A members and 5 from Team B

members. Some participants also made verbal com-

ments that were registered.

ICSOFT 2024 - 19th International Conference on Software Technologies

6.2 Data Analysis

At the end of data collection, we analyzed it to verify

whether the deﬁned objectives were met. The anal-

ysis is presented individually for each Measurement

Objective, grouping the answers to the questions ac-

cording to the collected data.

Q1.1 - What is the total effort spent by the team

members to apply risk management practices?

As risk identiﬁcation was performed asyn-

chronously, most of the effort on managing risks was

applied to risk analysis. As the risk registration form

was left open to collect the team members’ sugges-

tions throughout the study period, being analyzed at

the beginning of each Sprint, the team members were

registering possible risks asynchronously. Therefore,

this effort was not formally registered, as it was not

considered relevant by the teams.

Team A performed the risk analysis syn-

chronously right after a Daily Scrum meeting. The

risk analysis meeting duration was recorded and

lasted 53 minutes, involving 6 participants, resulting

in a total effort of 5 hours and 18 minutes.

Team B also performed risk analysis asyn-

chronously. We observed that the average time to

ﬁll the analysis risk questionnaire was of 5 minutes.

As the questionnaire was ﬁlled asynchronously by 6

Team B members, the effort on risk analysis was 30

minutes.

With that, the total effort of the teams exclusively

applied in risk management of 5 hours and 48 min-

utes was recorded.

Q2.1 - Are risk management practices easy to

understand and apply?

To obtain these data, a question of the question-

naire was used. For 36.4%(4) of the participants who

answered the questionnaire, the application was con-

sidered very easy. Already 54.5%(6) considered the

application easy and, ﬁnally, 9.1%(1) considered the

application neither easy nor difﬁcult. (Figure 4).

Some respondents also commented the reason

why they rated the ease of application in this way. 3

team members commented that ”the application form

was simpliﬁed due to the tool offered”, which was

”intuitive” and ”quick to use”. Another participant

highlighted that the checklist already has common

and easy-to-understand risks listed, made ”risk iden-

tiﬁcation quite simple”.

Q2.2 - Was the experience of applying the prac-

tices beneﬁcial to the participant teams?

For 63.6%(7) of the participants, the application

of risk management practices was beneﬁcial for the

team. Already 27.3%(3) stated that the application

was very beneﬁcial. While 9.1%(1) considered the

Figure 4: Q2.1 Ease of risk management practices.

Figure 5: Q2.2 Beneﬁt of practices.

application neither beneﬁcial nor harmful (Figure 5).

Two of the participants responded that the ap-

plication was ”beneﬁcial to the team” and that ”the

risk analysis process involved the team in discus-

sions about important issues”. One of the participants

also highlighted that ”the practices allowed for some

reﬂections that brought greater clarity among team

members regarding the team process”. Other respon-

dent also commented that ”it is possible that continu-

ing with the applications will make the process more

reﬁned and bring more beneﬁts”.

Already 2 participants raised the point that al-

though discussions were encouraged, ”there was no

time during the study for the development and ex-

ecution of action plans for risk mitigation”, which

”harmed the results of the application”. That’s why,

one of the members responded that the study brought

neither beneﬁt nor harm to the team.

Q2.3 - What were the main difﬁculties in apply-

ing risk management practices?

This question was collected through an open ques-

tion in the questionnaire. For this question, 4 partic-

ipants from Team A highlighted that a difﬁculty was

”the understanding of one of the risks raised in the

“Others” ﬁeld”. According to the respondents, the de-

scription of the risk was vague and the person respon-

sible for identifying such a risk chose not to comment

in order to clarify the issue. Thus, these 4 participants

stated that the fact that ”risk identiﬁcation through the

tool is anonymous and can cause such conﬂicts”.

One participant commented that a possible difﬁ-

culty encountered was ”the number of risks identi-

ﬁed”, which, according to him, ”was higher than ex-

Towards an Approach for Integrating Risk Management Practices into Agile Contexts

pected”, resulting in ”considerable time devoted to

risk analysis”. However, this participant stated that,

despite this, ”the analysis time was not long enough

to be considered a problem”. Three Team B mem-

bers responded that an obstacle they encountered was

”generating engagement among team members to par-

ticipate in the application”. The justiﬁcation was that

the ”team is working on a relatively new project”.

One Team B member also commented that ”the

lack of familiarity with project risks caused an ini-

tial awkwardness among team members and difﬁculty

in identifying risks even with the help of the check-

list”. This strangeness was also identiﬁed in another

answer, which mentioned that ”it was difﬁcult to as-

sess the priority of risks”. Finally, one response also

mentioned that ”no difﬁculties were encountered”.

Q2.4 - Was there an impact on the team’s ve-

locity when applying risk management practices?

In order to answer this question, we compared the

velocity of the Teams prior to this study and during

this study.

During the four Sprints prior to applying the risk

management practices, Team A had an average ve-

locity of 19.9 story points. During the three Sprints

in which the study was applied, the team’s velocity

dropped to an average of 17.41 story points. Caus-

ing a difference of approximately -12.5%.

Team B, started to use story points only from 2

Sprints prior to the application of the study. So, the

average velocity of these two sprints was 11 story

points. During the sprints in which the practices were

applied, the average velocity was 13.5 points, re-

sulting in an increase of 22.7%. Figure 6 shows the

evolution of Teams A and B velocity by Sprints.

Figure 6: Q2.4 Teams’ velocity.

The participants did not observe any unforeseen

risks during the projects. Additionally, no low

or medium risks were identiﬁed that could poten-

tially waste the team’s time. The project proceeded

smoothly, and the team’s velocity remained unaf-

fected by any unexpected risks.

Q3.1 - Does the team intend to continue using

the presented practices? If yes, why?

All the participants (100% - 11) answered that

they would continue using the risk management prac-

tices applied. The main reason, highlighted by 3

members, was that the checklist contributed to the

identiﬁcation of risks that existed in the project but

that had not been previously noticed.

In addition, 2 participants commented that the dis-

cussions that took place during the risk analysis pro-

cess were taken to other ceremonies, such as the Ag-

ile Wheel meeting, resulting in action plans based on

these analysis.

Finally, 1 member commented that ”identifying

risks adds to the team’s future performance by allow-

ing problems to be recorded as points of attention or

dealt with before they cause greater damage”.

Q3.2 - What were the main results in the appli-

cation of risk management practices?

In general, the biggest positive result identiﬁed by

the respondents was the beginning of the use of a sys-

tematic method for identifying and managing project

risks. 4 members commented on the importance of

such formality and 5 highlighted that risk manage-

ment was not present in the context of the organiza-

tion until then.

In addition, 3 members commented that the dis-

cussions and reﬂections that took place during the risk

analysis process helped the team members to better

understand each other’s point of view, improving the

team’s internal communication. And, 6 praised the

discussions that were raised from the risks identiﬁed.

Two of the participants responded that the prac-

tices made it clearer which risks the team should fo-

cus on initially and 1 of these mentioned that some of

these risks were not visible before applying the risk

identiﬁcation practice.

Finally, 1 participant highlighted that ”the pres-

ence of the checklist helped to identify risks for mem-

bers who had more difﬁculty ﬁnding possible prob-

lems in the project”. Another member stated that ”ap-

plying the practices made the team feel more prepared

to identify and contain new risks that could appear in

the future”.

6.3 Discussion

The results collected and the observation of the par-

ticipating teams during and after the application of

this study raise initial indications that it is possible

to integrate explicit risk management practices in an

organization that uses agile methods.

The total effort applied by 5,48 person/hours rep-

resents 0,21% of the total effort applied by the entire

teams (2.560 person/hours) during the period of the

study. This percentage was considered acceptable by

ICSOFT 2024 - 19th International Conference on Software Technologies

the participants and the team leaders involved. Re-

garding the negative and positive variation in velocity,

it is not possible to state that they were due to the in-

tervention carried out. For the Team A, for instance,

the decrease in the team’s velocity could be explained

by the entry of a new development team member in

lieu of an experienced team member during the appli-

cation of the study. Thus, this variation, both positive

and negative, was not considered relevant by the lead-

ers of the participating teams due to other inﬂuencing

factors.

Therefore, similarly to what was observed in

related works (Afshari and Gandomani, 2022)

(Tomanek and Juricek, 2015), this study reinforces

the tendency that the application of explicit risk man-

agement practices does not signiﬁcantly interfere with

the productivity of the teams.

The risk management practices were considered

easy to understand and apply, as they were selected

based on the literature and applied with the support of

simple tools and short explanatory videos.

In addition, no increase in bureaucracy or loss of

agile values and practices already adopted by the or-

ganization were observed, that is, they complemented

and seamlessly integrated with the existing agile pro-

cesses. The risk management activities did not intro-

duce excessive documentation, unnecessary formal-

ities, or additional overhead that could impede the

agility of the development process. These results con-

trast with some related studies that identiﬁed some

difﬁculties such as the increase in bureaucracy when

introducing risk management practices, such as in

(Ylimannela, 2012).

Also, all team participants intend to continue car-

rying out the practices after the study. However, al-

though the identiﬁcation and analysis steps were suc-

cessful, the participants felt that there was a lack

of support for effectively implementing the risk re-

sponses.

By adopting these practices, the projects gained

several potential beneﬁts. Firstly, the teams would

have improved visibility and awareness of the poten-

tial risks associated with their projects. This increased

awareness allows for early detection and timely re-

sponse to risks, minimizing the likelihood and impact

of potential negative events.

Finally, it is possible to note that the application

of the practices was useful to the teams, in particu-

lar by encouraging discussions and reﬂections on the

project and processes developed. And that, in general,

the teams beneﬁted from both the Risk Checklist and

the Risk Matrix. However, since the teams had never

used traditional risk management processes, we can-

not compare these practices to traditional methods.

We also see the results as positive as the risk man-

agement practices are still being applied on the teams

months after the action research and are been adopted

by the other teams as an organizational process.

6.4 Threats to Validity

There are several factors that may inﬂuence the results

obtained by this study. Such factors may thus threaten

the validity of these results.

In relation to internal validity, given the small

sample size and the restrictions of a study carried out

in a single organization, it is not possible to estab-

lish correlations nor causal relationships between the

results obtained and the intervention (Wohlin et al.,

2012). To minimize this limitation, we carried out a

systematic evaluation of the study, deﬁning data col-

lection using the GQM approach. We also involved

two different teams that develop different software

products for two different domains.

Another factor that may inﬂuence internal validity

is that two of the authors of this study are members

of the target organization. Although they do not work

directly in the participating teams, this proximity may

have inﬂuenced the participants’ responses. In an at-

tempt to minimize this threat, we also collected quan-

titative data that does not depend on the participants’

opinions, such as teams’ effort and velocity, to answer

research questions.

With regard to external validity, this study incor-

porates the typical limitations of this type of research,

as the results of single-organization studies are usu-

ally difﬁcult to generalize and interpret (Zelkowitz,

2009) (Wohlin et al., 2012). The attention to detail of

this type of study can lead to non-observance of rele-

vant aspects of a possible general framework of more

cases. Thus, the effects of a typical situation can be

observed appropriately, but in general it is not possi-

ble to establish a reliably generalization (Zelkowitz,

2009) (Wohlin et al., 2012).

7 SPECIFYING LEARNING

In this action research step, general learning is speci-

ﬁed based on the practical experience and can be used

in future improvements (Lings and Lundell, 2005).

During the study, we realized that the steps

adopted in the intervention could perhaps be gener-

alized to enable the adoption of explicit risk manage-

ment practices in contexts in which this is perceived

as necessary. Thus, by abstracting the steps taken

during the intervention, a possible general approach

emerged. We separate the actions we performed dur-

Towards an Approach for Integrating Risk Management Practices into Agile Contexts

ing this study into two phases: setup, where we

prepare for the introduction of explicit risk manage-

ment practices, and execution, where risk manage-

ment practices are carried out. Figure 7 presents the

general approach in BPMN (OMG, 2010).

Figure 7: Proposed Agile Risk Management approach.

The Setup phase begins with the subprocess Con-

text analysis, which consists of identifying the orga-

nizational context and the need for risk management.

Some of the aspects that may be considered include

the application domain, the team size, and the agile

method used. These main context characteristics were

identiﬁed in our previous SLM (Garcia et al., 2022).

The results of this activity will guide the next steps.

Having the context characteristics in mind, the

best-suited risk management practices are selected in

the subprocess Mapping of practices x context based

on the Catalog of practices collected from the litera-

ture (Garcia et al., 2022). Through this mapping, it is

possible to verify which practices are most appropri-

ate for a given context.

After choosing the practices, the activity Alloca-

tion of practices to ceremonies deﬁnes in which cer-

emonies the chosen risk management practices will

be incorporated, avoiding unnecessary bureaucracy

of additional meetings. To achieve this, the teams

need to analyze each practice and identify which cere-

monies are best suited for its implementation, consid-

ering the frequency of the ceremonies and the level of

involvement of the team members.

The Execution phase begins with the Application

of selected practices subprocess and the Data col-

lect activity running in parallel. During the applica-

tion, the selected risk management practices are im-

plemented in the project. The practices should be im-

plemented according to the plan deﬁned in the Allo-

cation of practices to ceremonies activity. In the Data

collect, data about the implementation of the selected

risk management practices are collected.

The Analysis of the application of practices ac-

tivity is the ﬁnal subprocess of the Execution phase.

This activity aims to evaluate the effectiveness of the

implemented practices. The data collected in the pre-

vious activity are analyzed to evaluate the application

of the practices. The metrics previously deﬁned are

used to measure the effectiveness of the practices in

mitigating risks, providing insights for future applica-

tions.

8 CONCLUSION

This paper presents an action research study on in-

tegrating explicit risk management practices into an

organization that uses agile methods.

The study begins with the diagnosis of the prob-

lem, which involves the analysis of the organizational

context. Based on this analysis, the intervention,

which consists of selecting agile risk management

practices, is planned and alternative solutions are in-

vestigated. The intervention is then executed, starting

with the selection of risk management practices, de-

velopment of the necessary tools and application of

the practices in two different teams.

The observed results raise initial evidence that the

risk management practices were effective and without

harming the productivity of the teams. Implementing

explicit risk management practices had positive im-

pacts on project performance, increased team collab-

oration, and reduced project risks. As another result

of the knowledge acquired with this study, a draft of

a generic approach for the integration of explicit risk

management practices in agile contexts emerges.

As future work, we are currently consolidating the

detailed deﬁnition of the approach and planning to

carry out case studies to evaluate its effectiveness in

different agile contexts.

REFERENCES

Abdelrafe, E., Hussin, B., and Salleh, N. (2016). Top ﬁfty

software risk factors and the best thirty risk manage-

ment techniques in software development lifecycle for

successful software projects. International Journal of

Hybrid Information Technology, 9(6):11–32.

Afshari, M. and Gandomani, T. J. (2022). A novel risk man-

agement model in the scrum and extreme program-

ming hybrid methodology. International Journal of

Electrical and Computer Engineering, 12(3):2911.

Alharbi, E. and Qureshi, M. R. (2014). Implementation of

risk management with scrum to achieve cmmi require-

ICSOFT 2024 - 19th International Conference on Software Technologies

ments. International Journal of Computer Network

and Information Security (IJCNIS), 6:20–25.

Avison, D. E., Lau, F., Myers, M. D., and Nielsen,

P. A. (1999). Action research. Commun. ACM,

42(1):94–97.

Chiste Brand

ao, A. B. (2013). Risk management in soft-

ware projects using scrum framework.

Concha, M., Visconti, M., and Astudillo, H. (2007). Agile

commitments: Enhancing business risk management

in agile development projects. In Agile Processes in

Software Engineering and Extreme Programming: 8th

International Conference, XP 2007, Como, Italy, June

18-22, 2007. Proceedings 8, pages 149–152. Springer.

Cuong, L. G., Hung, P. D., Bach, N. L., and Tung, T. D.

(2019). Risk management for agile projects in off-

shore vietnam. In Proceedings of the Tenth Interna-

tional Symposium on Information and Communica-

tion Technology, SoICT 2019, page 377–384.

Dhir, S., Kumar, D., and Singh, V. (2019). Success and fail-

ure factors that impact on project implementation us-

ing agile software development methodology. In Soft-

ware Engineering: Proceedings of CSI 2015, pages

647–654. Springer.

Elkhatib, M., Hosani, A. A., Hosani, I. A., and Albuﬂasa,

K. (2022). Agile project management and project

risks improvements: Pros and cons. Modern Econ-

omy, 13(09):1157–1176.

Garcia, F., Hauck, J., and Narloch Rizzo Hahn, F. (2022).

Managing risks in agile methods: a systematic lit-

erature mapping. In Proceedings of the 34th In-

ternational Conference on Software Engineering and

Knowledge Engineering, pages 394–399.

Ghobadi, S. and Mathiassen, L. (2017). Risks to effective

knowledge sharing in agile software teams: A model

for assessing and mitigating risks. Information Sys-

tems Journal, 27(6):699–731.

Godse, M. and Rajiv, B. (2021). Improvement of project

management knowledge areas using scrum technique.

In Optimization Methods in Engineering: Select Pro-

ceedings of CPIE 2019, pages 133–149. Springer.

Hammad, M. and Inayat, I. (2018). Integrating risk manage-

ment in scrum framework. In 2018 International Con-

ference on Frontiers of Information Technology (FIT),

pages 158–163.

Hauck, J. C. R. and Vieira, M. (2021). Towards a guide

for risk management integration in agile software

projects. In Systems, Software and Services Process

Improvement: 28th European Conference, EuroSPI

2021, Krems, Austria, September 1–3, 2021, Proceed-

ings, pages 73–87. Springer.

International Organization for Standardization (2009). ISO

31000:2009 risk management – principles and guide-

lines. International Standard. Accessed: April 10,

2023.

International Organization for Standardization (2010). ISO

80001-1:2010 application of risk management for

it-networks incorporating medical devices – part 1:

Roles, responsibilities and activities. International

Standard. Accessed: April 10, 2023.

Islam, S., Mouratidis, H., and Weippl, E. R. (2014). An

empirical study on the implementation and evaluation

of a goal-driven software development risk manage-

ment model. Information and Software Technology,

56(2):117–133.

Koziolek, H. (2008). Goal, Question, Metric, pages 39–42.

Springer Berlin Heidelberg, Berlin, Heidelberg.

Lings, B. and Lundell, B. (2005). On the adaptation of

grounded theory procedures: Insights from the evo-

lution of the 2g method. It & People, 18:196–211.

Nelson, C. R., Taran, G., and de Lascurain Hinojosa, L.

(2008). Explicit risk management in agile processes.

In Agile Processes in Software Engineering and Ex-

treme Programming: 9th International Conference,

XP 2008, Limerick, Ireland, June 10-14, 2008. Pro-

ceedings 9, pages 190–201. Springer.

Nyfjord, J. and Kajko-Mattsson, M. (2007). Commonalities

in risk management and agile process models. In In-

ternational Conference on Software Engineering Ad-

vances (ICSEA 2007), pages 18–18. IEEE.

OMG (2010). Business Process Model and Notation

(BPMN), Version 2.0.

Petersen, K., Gencel, C., Asghari, N., baca, d., and Betz,

S. (2014). Action research as a model for industry-

academia collaboration in the software engineering

context.

PMI (2021). A guide to the project management body

of knowledge (pmbok® guide)-and the standard for

project management. Project Management Institute.

Schwaber, K. and Sutherland, J. (2020). Scrum Guide - The

Deﬁnitive Guide to Scrum: The Rules of the Game.

Shrivastava, S. V. and Rathod, U. (2015). Categorization of

risk factors for distributed agile projects. Information

and Software Technology, 58:373–387.

Soares, A. G. (2017). Roda

Agil: Descubra a maturidade

do seu time.

Sutherland, J. (2021). Scrum at scale guide. Published

is a registered trademark of Scrum Inc. Released un-

der Creative Commons 4.0 Attribution-Sharealike Li-

cense.

Tam, C., da Costa Moura, E. J., Oliveira, T., and Varaj

ao,

J. (2020). The factors inﬂuencing the success of on-

going agile software development projects. Inter-

national Journal of Project Management, 38(3):165–

176.

Tanner, M., von Willingh, U., et al. (2014). Factors lead-

ing to the success and failure of agile projects imple-

mented in traditionally waterfall environments. Hu-

man Capital without Borders: Knowledge and Learn-

ing for the Quality of Life. Portoroz, Slovenia: Make

Learn, pages 693–701.

Tomanek, M. and Juricek, J. (2015). Project risk manage-

ment model based on prince2 and scrum frameworks.

Wohlin, C., Runeson, P., H

ost, M., Ohlsson, M. C., Reg-

nell, B., and Wessl

en, A. (2012). Experimentation in

software engineering. Springer Science & Business

Media.

Towards an Approach for Integrating Risk Management Practices into Agile Contexts

Ylimannela, V. (2012). A model for risk management in ag-

ile software development. Communications of Cloud

Software, 3:1–10.

Zelkowitz, M. V. (2009). An update to experimental models

for validating computer technology. Journal of Sys-

tems and Software, 82(3):373–376.

ICSOFT 2024 - 19th International Conference on Software Technologies