Assessment of Operational Risks Associated to New Product
Development
An Application using a Bayesian Networks Method
Tito Armando Rossi Filho, Guilherme Luis Roehe Vaccaro and José Antônio Valle Antunes Junior
PPGEPS, UNISINOS, Av. Unisinos, 950, São Leopoldo-RS, Brazil
Keywords: Operational Risks, Bayesian Networks, New Product Development, Decision Support System.
Abstract: This work presents an application of a method for assessing the operational risks associated to the
development of new products, aiming to address the uncertainties of technical and managerial risks. The
method has been developed using the Design Research approach, which enabled the development of a set of
artifacts linked through six steps, where the main artifact is a Bayesian Network model. The performance
evaluation of the application was carried out by a tentative application in a new product being developed at
a design center of a global company, where the main focus was to assess the risks associated with reliability
failures. This research, besides contributing with a proposal of an application to support the new product
risk management, indicates potential enhancements to the decision making process, and to the knowledge
management in project environments.
1 INTRODUCTION
During the design of new products risks can be
classified in technical and managerial (Grubisic,
2009). The technical risks include the risks
associated with particular activities during the
development process, but also consider other
activities of product development, such as the setting
of validation tests and quality evaluation of
suppliers. The managerial risks are derived from
some more specific elements of the project, such as
cost, time, communication and scope, i.e. the
variables related to project management.
In discussing strategies for dealing with the risks
associated with new product development, Chen, et
al. (2007) highlight the need for these to be
thoroughly and systematically analyzed. Considering
also the fact that in the business environment the
risks, in general, have increased the product
developments, managerial risks have become a very
important element of project management.
According to Carbone and Tippett (2004), failures in
implementing the risk management process can
cause many undesirable effects therefore this topic
has been increasingly important for success in
business management.
Considering the context of the growing level of
risks it is necessary to use methods and tools to
adequately deal with these uncertainties. However in
several cases only the use of classic methods such as
the tools for risk analysis in project management has
been proven to be not enough to deal with the
complexity of the research object. Furthermore,
although it is usually accepted the idea that many
subjective factors, e.g. experience of the project
team contributes to the product risks, the classical
models for risk assessment do not consider the
subjective information (Neil, Fenton and Nielsen,
2000).
2 LITERATURE REVIEW
The definition of risk is one of the most
controversial issues in the projects risk management
community (Chapman, 2011). Modarres (1993)
explains that it can be seen both qualitatively and
quantitatively when constructing the definition of
risk. Qualitatively when it is exposed to danger and
so there is a possibility of loss or damage. Zio
(2007), on the other hand, explains the first intuitive
observation comes from the fact there is a risk in
case there is a potential source of harm or danger.
The author also emphasize the presence of a danger
alone is not sufficient to set the risk condition - there
is an uncertainty factor in the transformation of a
182
Filho T., Vaccaro G. and Junior J..
Assessment of Operational Risks Associated to New Product Development - An Application using a Bayesian Networks Method.
DOI: 10.5220/0004286003300333
In Proceedings of the 2nd International Conference on Operations Research and Enterprise Systems (ICORES-2013), pages 330-333
ISBN: 978-989-8565-40-2
Copyright
c
 2013 SCITEPRESS (Science and Technology Publications, Lda.)

potential risk in using a real risk of danger.
Although the well-known risk analysis
techniques, e.g. FMEA (Failure Mode and Effect
Analysis), PHA (Preliminary Hazard Analysis) and
BDA (Block Diagram Analysis) are more detailed
than the simple identification of hazards, they
require a greater effort of the organization to
implement them. Nevertheless all are also
considered to be qualitative tools even indicating
levels of risk and none allows systemic and a more
dynamic risk analysis. Certain tools have been
developed to help close these mentioned gaps such
as the Markov chain modeling and dynamic event
trees.
Considering the Risk Management as an element
of Project Management, Project Management
Institute (PMI, 2004) recommends some activities
such as the step in preparation for the identification
of hazards, called Risk Breakdown Structure (RBS).
The goal is to guide the definition of risk categories
with the team.
A Bayesian network (BN) N = (, G, P) consists
of an acyclic graph G = (V, E) with nodes V = {1,
... n} and E directed links, a set of discrete random
variables, , represented by the nodes of G and a set
of conditional probability distributions, P, having a
distribution P (X

| X
PA ()
) for each random variable
X
v
and  (Jensen and Nielsen, 2007). A BN is a
probability distribution over a set of random
variables,  of the problem domain (Kjaerulff and
Madsen, 2008). The set of probability distributions,
P, specifies a multiplication factor of  probability
distribution, as can be seen in equation (1).



Vv
vpav
XXPP )|()(
)(

(1)
Hamada et al. (2008) present an important
consequence of the subjective probability utilized by
Bayesian inference, is that it allows incorporating
the use of information beyond what is classically
obtained from experimental data. The authors also
emphasize the use of relevant information
constitutes a useful and powerful component of the
Bayesian approach. Several sources of information
or knowledge can be incorporated as computational
analysis, general industry data, results of previous
tests of the product, data from similar products and
expert opinion.
Khodakarami (2009) describes the construction
of a BN occurs in three stages. The first phase is the
identification of important variables and their
possible states. The identification of the variables of
a problem is not always an easy task, and require
some practice (Kjaerulff and Madsen, 2008). The
second phase is called the elicitation of the model
structure. It comprehends the identification of
relationships between the variables and the
representation of these relationships through a graph.
The third phase for the construction of BN is called
elicitation of numerical parameters of the model.
The use of Bayesian networks for risk analysis
has the potential to have several positive aspects,
such as the representation of subjective knowledge,
the dynamic update of the model and the
representation of causal variables. However, while
there are some challenges in the construction and
analysis of models, some of them have been at least
partly solved by various researchers and software
developers in the last decade. Some studies have
explicit evidence of a significative ROI (Return on
Investment) in commercial applications of Bayesian
networks, as mentioned by Fenton and Neil (2007)
as well as Pourret, Naim and Marcot (2008),
3 METHODOLOGY
By looking at different research methodologies to
develop a method to be used by product
development professionals the paradigm of Design
Research (DR), also known as Design Science, was
considered appropriate to the research question, the
general purpose and nature of this work. Design
Research defines as a process of using knowledge to
design useful devices and then use various rigorous
methods to analyze whether a particular artifact is
effective or not (Manson, 2006).
The design cycles consist of the following sub
processes (Takeda, et al., 1990): awareness of the
problem (to identify a problem by comparing the
object under examination with the specifications),
conceptualization (to suggest key concepts needed to
solve the problem), development (to construct
candidate solutions from key concepts using various
types of knowledge for Design), Evaluation
(evaluating candidate solutions in various ways such
as computing structured), simulation or evaluation of
the costs and Conclusion (deciding which solution to
adopt, modify the descriptions of the object).
4 RISK ASSESSMENT METHOD
The initial step in designing the application was to
develop a list of general requirements, such as: it
should enable analysis of the risks associated with
the operation with qualitative and quantitative
AssessmentofOperationalRisksAssociatedtoNewProductDevelopment-AnApplicationusingaBayesianNetworks
Method
183

perspectives (COSO, 2004), it should provide tools
for easy computational implementation (Grubisic,
2009), it should provide a systematic way of
transmitting knowledge facilitating the learning
process of the method users (Grubisic, 2009). From
the list of requirements, the theoretical framework
and research methodology, the application was
developed starting with the creation of the model in
Bayesian Networks. A sequence of steps has been
developed and four artifacts required for
instantiation of the model and use the results in the
context of risk management.
At the first method step (Planning), the method
objectives need to be aligned to company’s senior
management. An explanation of the key concepts
related to the method, emphasizing the potential of
the model built on Bayesian networks as well as the
constraints need to be presented and well
understood. Once the objectives are aligned with the
organization’s senior management, a team must be
built to support the method implementation.
The second method step is the BN Model
Construction. The BN model and its implementation
in a software is a mandatory step since the
intelligence related to the calculation of risk levels is
derived from the implemented model together with
the software algorithms and the model constructs
elicitation.
The elicitation of numerical parameters is the
third method step in which greater effort and time is
required, since knowledge of the company experts
must be transferred for each construct to the CPTs
(Conditional Probability Tables). Besides extracting
the knowledge of experts, this step provides the
collection of data from previous projects. It is very
important that all experts are adequately prepared for
elicitation, especially by training on the meaning of
each construct.
During the last step of the method is carried out
the risk assessment itself. At this time, experts must
make inferences based on the artifact BNS and then
must lay the evidence collected from the artifact in
each phase of the project. Once launched, the
experts should receive information from the most
important risk levels to support the decision making
process. The method should allow the simulation of
different scenarios to be assessed how the risk can
be modified depending on the actions taken.
Once each phase of the project is completed, the
final risk must be recorded for subsequent
monitoring, which will allow to be evaluated to what
degree has varied over the projects. The last activity
by the method used should be performed once the
project is completed.
5 CASE STUDY
The application was evaluated in a company of the
electronics industry. In the implementation team
there were three experts with experience in
executing several projects in the company. The
implementation took two weeks for the Planning
step, five to the elicitation, and two weeks for
prioritization and the Risk Assessment itself.
The model was constructed as described in the
previous section. In the final version 21 variables
and 4 main areas (Supply Management, Product
Design, Project Management and Reliability
Engineering) were considered. The final model can
be seen at Figure 1.
Figure 1: Developed Model.
Once the RB was elicited in the software, the
next activity was the adaptation from the collection
of evidence of recent projects completed by the
organization, as the experts considered the number
of cases collected as significant.
Started from the adaptation of cases, evidences
were collected from eight projects previously run by
the organization, representing about 70% of recent
projects carried out over a period of five years. To
implement the process of adaptation by lot, it was
created a list of evidence in a compatible format.
After collecting some few evidences at the first
phase of the development, e.g. project complexity
level, the Risk Index was below the 59.7 level, being
slightly higher than the value without evidence
(58.9). However, the fact that the value of risk have
been above the acceptable threshold for the launch
of the product (52.0) indicated the organization
should take action along the project that reduce this
level of overall risk compared with the usual level
the other variables.
A design risk assessment indicated the need for
caution due to some components that were being
considered for the BOM (bill of material) - due to
strong pressure for lowering the overall product cost,
it was defined the need to use components of lower
ICORES2013-InternationalConferenceonOperationsResearchandEnterpriseSystems
184

price. Given the concerns related to components,
raised at Phase 2 of the project, the construct DF was
assessed at low level ("L"), although there were not
very objective evidences the performance of the
selected suppliers was low.
Once selected all levels of the variables for
which there were evidences, the artifact SOFTRB
executed the algorithm and presented the variable
results. The Risk Index stood at 64.2, i.e. within the
red band and above the risk threshold for phase 2. It
indicated the need to take action to modify some risk
levels before going on with the project, i.e. before
starting the third phase.
6 CONCLUSIONS
This work evaluated a method for assessing
operational risks associated with the development of
new products. Through the Design Research
method, five artifacts have been developed, in
addition to the steps of the risk assessment method
itself. The method seemed to contribute to the
improvement of the process of Risk Management
held during the introduction of a new product.
The main evaluated artifact was a model
constructed using the Bayesian Networks approach.
The model considers the judgment of company
experts with some qualitative data in order to
evaluate technical and managerial risks, which were
linked through probability tables, generating by
means of a prototype system the overall risk
associated to the product launch. For the model to be
used in practice and in any consumer electronics
design center it was proposed a logical sequence of
steps involving the use of all artifacts.
The general objective was the evaluation of the
method in a first real application. Of course, the
conclusion about the method feasibility in all
companies require a more extensive research,
involving several companies from different
segments.
The limitations of this study are related to the
implementation of the empirical part of it. There was
only one attempt to apply the method and in a single
project. This fact restricts the generalization of
findings regarding the applicability of the method in
other projects.
As suggestions for future work, first is the use of
alternative approaches in the process of elicitation of
the probability tables in order to minimize the effort
required. Other possible enhancement is the
inclusion of methods to enable the calculation of
economic loss distributions.
REFERENCES
Carbone, T.A. and Tippett, D.D., 2004. Project risk
management using the Project risk FMEA.
Engineering Management Journal, 16(4), p.28.
Chapman, R. J., 201. Simple Tools and Techniques for
Enterprise Risk Management. 2nd ed. London: J.
Wiley & Sons.
Chen, M., Chen, Y., Chen, B. and Wang, Q., 2007.
Analysis on Product Technical Risk With Bayesian
Belief Networks. Knowledge Enterprise: Intelligent
Strategies in Product Design, Manufacturing and
Management. Boston: Springer, p. 990-995.
Comitee of Sponsoring Oganizations of the Treadway
Comission (COSO), 2004. Enterprise Risk
Management- Integrated Framework. New York:
AICPA.
Fenton, N., Neil, M., 2007. Managing Risk in the Modern
World: Applications of Bayesian Networks. London:
Mathematical Society.
Grubisic, V.V.F., 2009. Metodologia de Gerenciamento
Integrado de Riscos Técnicos e Gerenciais para o
Projeto de Produtos. Ph.D. UFSC.
Hamada, M.S., Wilson, A., Reese, C.S., Martz, H., 2008.
Bayesian Reliability. New York: Springer Science.
Jensen, F.V. and Nielsen, T.D., 2007. Bayesian Networks
and Decision Graphs. 2nd ed. New York: Springer
Science.
Khodakarami, V., 2009. Applying Bayesian Networks to
model Uncertainty in Project Scheduling. Ph. D.
Queen Mary, University of London.
Kjaerulff, U.B. and Madsen, A.L., 2008. Bayesian
Networks and Influence Diagrams: a Guide to
Construction and Analysis. New York: Springer
Science.
Manson, N.J., 2006. Is operations research really research?
Operations Research Society of South Africa, 22(2).
Modarres, M., 1993. What Every Engineer Should Know
About Reliability and Risk Analysis. New York:
Marcel Dekker.
Neil, M., Fenton, N. and Nielsen, L.M., 2000. Building
Large-Scale Bayesian Networks. The Knowledge
Engineering Review, 15(3): p. 257-284.
PMI (Project Management Institute), 2004. ANSI/PMI99-
001-2004 A guide to the project management body of
knowledge. Milton Square: Project Management
Institute.
Pourret, O., Naim, P. and Marcot, B., 2008. Bayesian
Networks: a Practical Guide to Applications. New
York: John Wiley & Sons, 429 p.
Takeda H., Veerkamp, P., Tomiyama, T. and Yoshikawa,
H. 1990. Modeling design processes. Artificial
Intelligence Magazine, 11 (4).
Zio, E., 2007. An Introduction to the Basics of Reliability
and Risk Analysis. Danver: World Scientific
Publishing.
AssessmentofOperationalRisksAssociatedtoNewProductDevelopment-AnApplicationusingaBayesianNetworks
Method
185