Which Tools Are Needed to Assist Audit Managers in Project
Portfolio Selection When Divergent Views Emerge?
Vivian Vivas
1,2
and Mónica Duarte Oliveira
1
1
Centre for Management Studies of Instituto Superior Técnico (CEG-IST), Universidade de Lisboa, Lisbon, Portugal
2
Ministry of Transparency and Comptroller General of the Union, Brasilia, Brazil
Keywords: Conflict Analysis, Multicriteria Decision Analysis, Portfolio Decision Analysis, Auditing.
Abstract: In Brazil, government programs differ in terms of territorial coverage, of sectorial action, and vary according
to regional contexts. Similarly to other public auditing organizations, the Ministry of Transparency and
Comptroller General of the Union (CGU) is responsible for monitoring the application of funds in these
government programs as a means to promote transparency and accountability in public spending. In order to
fulfil this mission, it is necessary to select the audit projects to be executed by the CGU teams. Several
multidisciplinary teams and stakeholders participate in this process of selecting auditing projects across
distinct management and public policies themes, with different opinions and views arising. The most
important challenges for CGU are to compare the audit projects submitted by these teams on a common and
transparent basis (under the presence of scarce resources) and to consider in the selection of projects the
divergent and often conflicting perspectives of those involved. In this study, we explore which tools from the
multicriteria portfolio decision analysis and negotiation literature can be used to inform a transparent and
negotiated selection of audit projects, as well as discuss which type of multicriteria portfolio decision analysis
models and features for negotiation should be combined in future research.
1 INTRODUCTION
The Ministry of Transparency and Comptroller
General of the Union (CGU) is responsible for
checking the application of public funds in
government programs as a means of promoting
transparency and accountability in public spending.
Decisions regarding which projects the CGU should
audit can be considered multicriteria resource
allocation problems, since they involve multiple
strategic objectives, the benefits are usually multi-
dimensional (related with the delivery of effective
and quality public policies, losses recovery,
transparency in public management, social
responsibility), and there limitations on the resources
available to dedicate to auditing. At a time when
public resources are particularly scarce, CGU needs
to decide on this allocation of resources in audit
projects. This allocation of resources is not always
straightforward as it is typical to have divergent and
conflicting views among those participating in the
auditing selection process, and a compromise solution
needs to be negotiated.
Thus, it is central for the CGU to decide: how to
select the audit projects to compose the Operational
Plan while considering the different views and
perspectives from CGU stakeholders? Selecting sub-
set of projects is a portfolio decision, and up to our
knowledge we have not identified literature
integrating negotiation with portfolio analysis, nor
negotiation literature associated with auditing
decision-making. This study aims to contribute to
these fields by digesting concepts and models from
the Multicriteria Decision Analysis (MCDA),
Portfolio Decision Analysis (PDA) and negotiation
literature, and reflecting upon how such concepts and
models can support the selection of projects in
auditing contexts.
In the next section we detail the results from a
review of MCDA, PDA and negotiation studies that
provides useful concepts and models to support the
selection of audit projects under the presence of
divergent views among those involved in the
selection process and under limited resources.
Departing from the results of the review, section 3
systematizes tools that can inform the selection of
audit projects, as well as identifies which type of
338
Vivas, V. and Oliveira, M.
Which Tools Are Needed to Assist Audit Managers in Project Portfolio Selection When Divergent Views Emerge?.
DOI: 10.5220/0007374003380345
In Proceedings of the 8th International Conference on Operations Research and Enterprise Systems (ICORES 2019), pages 338-345
ISBN: 978-989-758-352-0
Copyright
c
2019 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
models may be developed. Final conclusions and
remarks are drawn in section 4.
2 REVIEW OF STUDIES
Taking a constructivist view in MCDA (Lee, 2012;
Peterson, 2012) that involves research is performed
with the collaboration of participants/evaluators and
accepts that “a decision situation is, in general, an ill-
defined entity, unclear even to the actors involved in
the decision process” (Bana e Costa and Pirlot,
1997), our review of studies started by searching
studies that report multicriteria evaluation models and
tools to evaluate projects and that use some type of
negotiation instrument to deal with divergent
opinions. Our search protocol focused on
combinations of several keywords negotiation,
group decisions, conflict analysis, disagreement,
MCDA, PDA, integrative negotiation, auditing in
the data sources B-on; Web of Science;
ScienceDirect; SCITEPRESS Digital Library.
Surprisingly, a small number of studies filled these
search criteria. These studies typically make use of
multicriteria resource allocation and of multicriteria
portfolio decision analysis models and explicitly
make use of some type of negotiation or consensus
building mechanism, which are key features for
models to inform the selection of projects when there
are scarce resources and participatory processes are
endorsed. Before introducing these studies, we
present key concepts.
2.1 MCDA and PDA
Here, we begin by highlighting the elementary
principles of building decision support models that
are key for multicriteria modelling. From the
perspective of MCDA, one can say that evaluation
models for decision support require three interactive
phases: (i) the structuring of the decision-making
context; (ii) the construction of evaluation model; and
(iii) impact assessment and analysis (Bana e Costa
and Beinat, 2005). The structuring phase requires
understanding the problem and the decision context.
To achieve this, a representation in the form of a
hierarchical value structure commonly named as a
value tree accepted and negotiated by all
stakeholders is constructed (Keeney, 1992). This
tree should represent, in an organized way, the
dimensions of values and key-concerns that are
relevant to the evaluation process and according to
which the options/projects/actions will be evaluated.
In the evaluation phase, a mathematical evaluation
model (most commonly an additive value model),
through which options are evaluated, is constructed.
The impact assessment phase is sought to provide
those developing the model with the analysis of the
consequences of implementing each one of the
options considered, with model adjustment and
validation procedures being also used (Bana e Costa
and Beinat, 2005).
In turn, in case of limited resources we are dealing
with portfolio problems and Portfolio Decision
Analysis (PDA) is applicable. According to (Salo et
al., 2011) PDA means a body of theory, methods,
and practice which seeks to help decision makers
make informed multiple selections from a discrete set
of alternatives through mathematical modelling that
accounts for relevant constraints, preferences, and
uncertainties. When one comes across a real
situation of audit project selection to compose a
portfolio, we need mechanisms to evaluate these
projects in multiple dimensions and this has been
done by MCDA literature more specifically by
multicriteria portfolio decision analysis literature
applied to real situations, whose models have a
potential to help building multicriteria models (Bana
e Costa, 2001; Bana e Costa et al., 2001; Mateus et
al., 2017; Oliveira et al., 2012) in the auditing context.
Since these models are already consolidated in the
literature we suggest making use of built-on-purpose
additive value models, similar to those presented in
these references, to evaluate the audit projects. It is
worth mentioning that under the presence of
divergent views, the models should reflect the
different opinions of the decision-makers' groups (or
stakeholders, or those involved in evaluation) and
make use of well-designed participatory processes
(Phillips and Bana e Costa, 2007); and that when one
identifies non-additive cases, models should be
restructured so that additivity conditions are
respected (having additive models the advantage of
being transparent and better understood by evaluators
and stakeholders).
Seeking for applications of MCDA in conflict
analysis contexts, various approaches and techniques
for dealing with divergent views have been proposed.
Bana e Costa (2001) explored the use of MCDA to
support the search for less conflicting policy options.
The author highlighted that public resource allocation
often requires the management of conflicting
objectives of multiple policy actors at different spatial
levels. The mix of limited financial resources,
multiple and conflicting concerns, spatial variability
of policy impacts and several types of uncertainty in
the data available for policy evaluation, make this
process problematic. In this case, conflict analysis
Which Tools Are Needed to Assist Audit Managers in Project Portfolio Selection When Divergent Views Emerge?
339
was based on the spatial analysis of the results of the
value model. Bana e Costa et al. (2001) presented a
case study of conflict dissolution in the public sector
through identification of the fundamental points of
view characterising the different value systems of the
stakeholders. The authors brought a pre-negotiation
approach aiming to dissolve the conflict in an
enlarged frame. Impact assessment revealed the
conflicting nature of the alternatives. The authors
then engaged the planners in a decision-analysis
process oriented towards the generation of win-win
solutions to dissolve the intrinsic value conflict.
Losa and Belton (2006) brought an exploratory
application of an integrated approach, combining
MCDA and conflict analysis. They have integrated
Drama Theory and MCDA to model the situation
using confrontation analysis with the following
elements: characters, actions, positions and fallbacks,
and preferences. The resolution of the conflict
consisted of detailed analysis of the characters
positions, threats and dilemmas, through a
multicriteria evaluation of the different futures. Also
worth mentioning is the use of the MACBETH
(Measuring Attractiveness through a Category Based
Evaluation TecHnique) approach, that only requires
qualitative judgments about differences in
attractiveness to build a multicriteria value model
(Bana e Costa et al., 2012), and has been used for
consensus generation. Specifically, two studies
reported the use of a “MACBETH Voting” decision
support system to promote compromise in model
building. “MACBETH Voting” is characterised by
using the MACBETH (intuitive) qualitative
questioning protocol together with voting procedures
that potentiate convergence of views in a decision
conferencing environment in which key stakeholders
physically meet (Bana e Costa et al., 2014; Mateus et
al., 2017).
Mateus et al. (2017) describe a real-world
application of MCDA and related Decision Support
Systems (M-MACBETH, MACBETH Voting, and
Web-MACBETH) to support the engagement and
participation of a group of key players. Two
alternative multicriteria aggregation schemes were
applied in order to assist the group in evaluating the
added value and doability of the proposed actions.
New measures and methods to analyse the dominance
relationships between the actions were proposed,
further assisting the group in the priority selection of
the most effective and doable actions.
Fasth et al. (2016) presented a method based on
disagreement constrained action selection in
participatory Portfolio Decision Analysis. They
investigated the stakeholders' disagreement with
regard to each action, and how portfolios can be
generated that elucidate how conflicting preferences
affect the portfolio composition. Their method for
participatory PDA consisted on: eliciting stakeholder
preferences; measuring stakeholder disagreement;
disagreement constrained portfolio generation; and
sensitivity analysis.
Salo (1995) developed an interactive approach for
the aggregation of group members' preference
judgements and presented joint preference
representation in the form of value trees that conveys
areas of conflict and disagreement. Vilkkumaa et al.
(2014) described a multicriteria portfolio modelling
for the development of shared action agendas in
group decision and negotiation. When seeking for
consensual compromise solutions, non-dominated
portfolios with a high acceptability index are viable
candidates because they contain topics that are in the
core or borderline for many group members.
2.2 Negotiation
Moving towards studies focusing mainly on the
negotiation process, as emphasized by Vetschera
(2013), negotiations are one means of resolving
conflicts. Negotiation depends exclusively on the
parties involved and on their attempt to reach an
agreement that is acceptable to all parties. It can be
seen from a prescriptive-descriptive perspective,
where rationality of the negotiator are supported by
prescriptive theories such as game theory and, on the
other hand, actual human behavior is considered
(Raiffa, 1982). Therefore, a negotiation can be seen
as a process at the group level, in which those
involved influence each other and try to converge
toward some point of agreement. The author points
out that negotiation processes can be based on
concessions, in which each party begins from a
desirable position and over time reduces its demand
until a point considered satisfactory for both parties
and an agreement is found. Or the parties can start
from a solution which is not attractive to either party,
and jointly look for improvements, as in single
negotiation text (SNT) type of negotiation (Raiffa,
1982).
Keeney (1992) also suggested procedures for
empathetic negotiation within a value focused
thinking frame: view the situation from the
perspective of other stakeholders; structure his
values as much as possible; begin by identifying the
negative impacts of your desired alternative relative
to the status quo in terms of his values; follow their
implications through a mean-end objectives network
to the fundamental objectives of the stakeholder;
ICORES 2019 - 8th International Conference on Operations Research and Enterprise Systems
340
create modified alternatives that can at least improve
matters in terms of these objectives while maintaining
the key consequences desired by you. In the end, the
goal is to create an alternative that both parties win.
In an integrative negotiation process, as explained
by Sarabando et al. (2013), successful strategies
include cooperation, information sharing and joint
resolution of problems. Mediation and arbitration are
particularly useful in integrative negotiation, since
they can help negotiators to identify potential areas of
improvement for both sides. A value-based
evaluation model allows each party to evaluate their
potential own proposals, proposals made by the other
party, and their BATNA (best alternative to a
negotiated agreement).
Filzmoser and Vetschera (2008) highlighted the
bargaining process that can be seen as a sequence of
offers and often, formal models of negotiation
processes based on theories such as game theory or
decision analysis focus on the exchange of offers.
Greenhalgh and Chapman (1998) showed that
information sharing could facilitate joint gain because
negotiators disclose and learn about the interests of
each party, providing integrative bargaining.
Górecka et al. (2016) presented an approach in the
verbal and holistic evaluation of the negotiation
template to evaluating negotiation offers when the
negotiator’s preferences are expressed verbally.
Present Measuring Attractiveness near Reference
Situations (MARS) approach, these authors
combined the algorithms of two multiple criteria
decision making methods: ZAPROS and
MACBETH. They also suggested a pre-negotiation
preparation, with a negotiation template, designed
and evaluated by means of the negotiation offer
scoring system. The problem of evaluating the
negotiation template from an individual negotiator’s
viewpoint is similar to a decision-making problem
with multiple criteria involved and negotiation offer
scoring system was modelling as a simple additive
weighting (SAW) method.
2.3 Key Aspects from Reviewed Studies
Table 1 summarizes key aspects from the reviewed
studies that may be specifically useful to the design of
inform and tools to assist the selection of auditing
projects in the presence of divergent opinions and of
resource constraints.
From the review of studies, we observe that no study
has focused on the negotiation process for conflict
resolution in auditing portfolio management context,
and that there is space for developing new models and
tools in the area of multicriteria PDA.
Table 1: Key aspects from reviewed studies.
Reference
Field of
Knowledge
Area of study
application
Study features with special relevance for a negotiated
selection of auditing projects
Keeney (1992)
MCDA
Negotiation
Conceptual
examples
Concepts for an empathetic negotiation. Structuring values in
mean-end objectives network.
Salo (1995)
MCDA
Group decisions
Marketing and
production
Joint preference representation and dominance concepts. Value
tree
Greenhalgh and
Chapman (1998)
Integrative bargaining
Negotiation tactic
laboratory study
Cohesive relationships encourage information-sharing and
discourage use of coercive tactics. Integrative bargaining.
Bana e Costa
(2001)
MCDA
Resource Allocation
Public sector
(road-links)
Conflict analysis based on the spatial analysis of the results of the
value model. Structuring multicriteria resource allocation model.
Bana e Costa et al.
(2001)
MCDA
Transport
planning
Value systems of the stakeholders. Pre-negotiation Conflict
dissolution through ‘win-win’ compromise solution.
Losa and Belton
(2006)
MCDA
Group decisions
Analysis of
conflicts in a
social service
Conflict analysis.
Integrated use of Drama Theory and MCDA.
Filzmoser and
Vetschera (2008)
Bargaining process
Offers
Electronic
negotiations
Develop a typology of bargaining steps for multi-issue
negotiations
Vetschera (2013)
Negotiation
Offers
Conceptual
examples
Survey of process models of negotiations. Concession-based
negotiation. Improvement-based negotiation
Sarabando et al.
(2013)
Integrative
negotiation
Conceptual
examples
Integrative negotiation. Value-based evaluation model.
BATNA (best alternative to a negotiated agreement)
Vilkkumaa et al.
(2014)
PDA
Group decisions
Agenda building
(wood products)
Interactive decision process. Group members’ preferences
synthesized into shared priorities for action topics.
Fasth et al. (2016)
PDA
Urban planning
Disagreement constrained portfolio generation.
Górecka et al.
(2016)
Negotiation offer
scoring system
Conceptual
examples
Pre-negotiation: negotiation template, designed and evaluated by
means of the negotiation offer scoring system.
Mateus et al. (2017)
MCDA
Group decisions
Brownfield
Evaluating the added value and doability of the actions. Application
of MCDA and related DSSs to support the engagement and
participation of a group of key players.
Which Tools Are Needed to Assist Audit Managers in Project Portfolio Selection When Divergent Views Emerge?
341
3 WHICH TYPE OF CONCEPTS
AND TOOLS ARE USEFUL FOR
SELECTING AUDITING
PROJECTS?
Departing from the auditing context, our proposal for
systematizing useful models and tools is organized in
the structure shown in Figure 1: we starting by
describing the building of multicriteria portfolio
decision analysis models through a socio-technical
approach; we then describe the tools to display
information concerning auditing projects or
portfolios; we then focus on participants’ interaction
for negotiation; and we finally focus upon on relevant
negotiation approaches that one can follow.
Figure 1: Components for systematizing models and tools
to be used in the selection of auditing projects.
3.1 Multicriteria Portfolio Modelling
To build a multicriteria portfolio model (by following
a socio-technical approach (Phillips and Bana e
Costa, 2007)), one can start by building the value
trees, which structure the fundamental
dimensions/objectives of the problem and the criteria
to assess benefits of audit projects, according to the
view of each group of stakeholders (Bana e Costa,
2001; Bana e Costa et al., 2004). At this point, the
MACBETH approach (Bana e Costa et al., 2012),
supported by the DSS M-MACBETH, is an available
approach to assist the construction of a model to
evaluate auditing projects (entailing an intuitive
protocols and being compatible with the use of voting
and negotiation procedures).
The selection of audit projects can inherently
make use of multicriteria PDA models, as shown by
Oliveira et al. (2012). Adapting for the auditing
context, stakeholders involved in the evaluation
process must evaluate each audit project j to be
included in the portfolio, according to the group
model. The performance

of each audit project on
each benefit criterion can be measured by a level in
the respective descriptor with partial value


, in
which we have 

and


. Under an additive
structure, which requires the respect for mutual
independence conditions, the value of the overall
benefit
of the project , can be determined as:






  

(1)
Moreover, in the case of the information on the
costs of audit projects are not available, one can make
use of the doability concept (Bana e Costa et al.,
2014) as proxy for audit cost. Doability can represent
the extent to which each audit project can effectively
be implemented, considering the limited resources
and financial, legal, social, and other constraints that
are beyond CGU managers’ control (Mateus et al.,
2017). The analysis of the divergences will be pointed
out from the results of the evaluations of the audit
projects in terms of benefits and doability according
to each model.
In view of maximizing the value of Equation (1),
the following optimization problem must be solved,
considering each project has
and cost
and
is the total of available resources (as
, if the
project is included in the portfolio and 
otherwise):


(2)



  
(3)
Additional constraints can be considered.
On the social component of portfolio modelling,
to involve and model the preferences of stakeholders
and/or decision-makers, depending on the context,
several formats can be adopted in model building. For
instance, semi-structured interviews, web
questionnaires, Delphi processes (Bowling, 2009), as
well as decision conferences that involve a face-to-
face meeting with key players (stakeholders and
ICORES 2019 - 8th International Conference on Operations Research and Enterprise Systems
342
experts) and an impartial facilitator, typically with the
support of MCDA tools (Phillips and Bana e Costa,
2007) can be adopted. Participants need to participate
in several modelling tasks that include the structuring
of a value model and the building of partial value
functions and of weighting scales.
3.2 Information Basis
From the reviewed studies, there are a set of tools that
can help auditing stakeholders to analyse projects or
portfolios, and to portray the consequences of
differences in opinion for an informed negotiation.
These tools include:
Info 1: Evaluation of Benefits and Doability,
separately: analysis/view of the benefits and
doability of each project according to a group or
groups of stakeholders (Bana e Costa et al., 2014).
Info 2: Benefit x Doability Graph: trading
benefit off against doability should drive the selection
of the best actions (Mateus et al., 2017). This graph
allows stakeholders to perceive the expected impact
and cost of each audit project, in the view of each
party involved in the negotiation. We also suggested
make use of Strategic Matrix, with 4 distinct
quadrants, include the pearls in the portfolio, and
negotiate the oyster and bread and butter(Bana e
Costa et al., 2014).
Info 3: Benefit/Effort Ratio: prioritise the audit
projects by their value-for-effort, defined as the ratio
between the benefit and the effort scores (Bana e
Costa et al., 2014). This information is useful to rank
the list of audit projects, allowing to perceive, in the
view of each party, the order of projects with the best
benefit / effort ratio.
Info 4: Portfolios According to the Views of
Each Group: from the definition of the available
budget, it is possible to define which audit projects
will compose the portfolios according to the models
of each stakeholders group, which means different
portfolios of projects may be obtained. For instance,
analysis of the differences and implications according
to each auditing stakeholder group will allow the
identification of divergences should focus in
preparation for discussion and negotiation between
groups.
Info 5: Dominance: dominance analysis can be
useful in negotiation as the dominance criterion is a
natural starting point in selecting proposals
(Sarabando et al., 2013). When seeking for
consensual compromise solutions, the examination of
core index values makes it possible to analyse the
non-dominated portfolios and provide relevant inputs
for group deliberations about viable candidates (Salo,
1995; Vilkkumaa et al., 2014).
3.3 Interaction between Participants
for Negotiation
Assuming that we have two groups of stakeholders
with divergent views, we must model the different
views of each group and how the groups may interact.
Figure 2 summarizes key aspects from a project
selection process that can help defining the audit
projects, for instance to compose the Operational Plan
at CGU. Each stakeholder group has representatives
who will define the projects of their teams.
Considering a group A, with visions according to
model G1, and a group B, with vision according to
model G2. Individually, each leader of these groups
must evaluate their projects to enter the Operational
Plan. From the results, differences are raised and the
space for negotiation is defined. The groups will need
to negotiate to reach the compromise solution. We
would thus have general models, based on a common
and transparent basis, and at the same time, space to
include the knowledge of each stakeholder, for
instance, an expert in the field, in the final decision
project selection.
Figure 2: Illustration of the negotiation space for the audit
project selection process.
3.4 Negotiation Strategies
As emphasized by (Górecka et al., 2016), a pre-
negotiation phase should be defined to establish a
detailed vision of the negotiation problem, the
parties involved and the context and, after analysing
them, define a negotiation strategy that would allow
the party to obtain the goals. Thus, the initial
negotiation template would be composed of the
projects that compose the sets of divergence. The
Which Tools Are Needed to Assist Audit Managers in Project Portfolio Selection When Divergent Views Emerge?
343
actual negotiation can start by an initial stage, in
which two offers are submitted at the negotiation
table, one by each of the parties. At each negotiation
stage one can observe the scale of differences that
needs to be eliminated to achieve a compromise
between the parties and what their endeavours are in
achieving the current negotiation status. Moments of
reverse concessions should be identified and, by
analysing the structure of the offers being presented,
competing issues can be identified and addressed
within negotiation.
In case of an integrative negotiation is adopted, it
is essential to identify potential areas of improvement
for both sides, on the exchange of offers, as also their
BATNA (best alternative to a negotiated agreement).
Thus, parties have to share information to facilitate
joint gain.
At the end, by following structured, transparent
and informed decision processes, it is expected that a
compromise solution will be reached and a final
portfolio defined.
4 CONCLUSIONS
The CGU in Brazil faces the challenge of, under the
presence of scarce resources, executing auditing
projects across distinct management and public
policies themes, involving multidisciplinary teams
and stakeholders, with different opinions and views.
We have shown in this study that portfolio resource
allocation models can be built to reflect the opinions
of distinct decision-maker and stakeholder groups,
and simultaneously auditing projects can be analysed
on a common and transparent basis. Negotiation tools
can be used to systematize the confrontation of
situations of divergence and conflict, and to search
converges toward some point of agreement. To
implement that socio-technical processes (Phillips
and Bana e Costa, 2007) must be considered, in which
methods, techniques and tools for model building are
intrinsically combined with participatory processes
involving stakeholders, decision-makers and/or
experts. By combining multicriteria methodology
with negotiation tools and techniques, it is possible to
build multicriteria resource allocation tools that
support the negotiation process in a shape of an
informed negotiation framework.
Thus, in this study we show multicriteria PDA
concepts and tools couples with negotiation strategies
that can be used to inform a transparent and
negotiated selection of audit projects; and that there
is relevance and scope for developing and testing
such type of models to assist the evaluation and
negotiation of auditing projects to integrate the
Operational Plan at CGU. A real case study is
currently being developed through a methodology
inspired and combining many of the concepts
introduced in this paper.
REFERENCES
Bana e Costa, C.A., 2001. The use of multi-criteria decision
analysis to support the search for less conflicting policy
options in a multi-actor context: case study. J. Multi-
Criteria Decis. Anal. 10, 111125.
Bana e Costa, C.A., Antão da Silva, P., Nunes Correia, F.,
2004. Multicriteria Evaluation of Flood Control
Measures: The Case of Ribeira do Livramento. Water
Resour. Manag. 18, 263283.
Bana e Costa, C.A., Beinat, E., 2005. Model-structuring in
public decision-aiding, LSEOR 05.79. Operational
Research working papers, London.
Bana e Costa, C.A., De Corte, J.-M., Vansnick, J.-C., 2012.
MACBETH. Int. J. Inf. Technol. Decis. Mak. 11, N° 2,
359387.
Bana e Costa, C.A., Lourenço, J.C., Oliveira, M.D., Bana e
Costa, J.C., 2014. A Socio-technical Approach for
Group Decision Support in Public Strategic Planning:
The Pernambuco PPA Case. Gr. Decis. Negot. 23, 5
29.
Bana e Costa, C.A., Nunes Da Silva, F., Vansnick, J.-C.,
2001. Conflict dissolution in the public sector: A case-
study. Eur. J. Oper. Res. 130, 388401.
Bana e Costa, C.A., Pirlot, M., 1997. Thoughts on the
Future of the Multicriteria Field: Basic Convictions
and Outline for a General Methodology. In: Clímaco J.
(Eds) Multicriteria Analysis. Springer, Berlin,
Heidelberg.
Bowling, A., 2009. Research Methods in Health:
Investigating health and health services, Third Edit. ed,
McGraw Hill. Open University Press, Berkshire.
Fasth, T., Larsson, A., Kalinina, M., 2016. Disagreement
Constrained Action Selection in Participatory Portfolio
Decision Analysis. Int. J. Innov. Manag. Technol. 7, 1
7.
Filzmoser, M., Vetschera, R., 2008. A classification of
bargaining steps and their impact on negotiation
outcomes. Gr. Decis. Negot. 17, 421443.
Górecka, D., Roszkowska, E., Wachowicz, T., 2016. The
MARS Approach in the Verbal and Holistic Evaluation
of the Negotiation Template. Gr. Decis. Negot. 25,
10971136.
Greenhalgh, L., Chapman, D.I., 1998. Negotiator
Relationships: Construct Measurement, and
Demonstration of Their Impact on the Process and
Outcomes of Negotiation. Gr. Decis. Negot. 7, 465
489.
Keeney, R.L., 1992. Value-focused thinking: A Path to
Creative Decisionmaking. Harvard University Press,
Cambridge.
ICORES 2019 - 8th International Conference on Operations Research and Enterprise Systems
344
Lee, C.J.G., 2012. Reconsidering Constructivism in
Qualitative Research. Educ. Philos. Theory 44, 403
412.
Losa, F.B., Belton, V., 2006. Combining MCDA and
conflict analysis: An exploratory application of an
integrated approach. J. Oper. Res. Soc. 57, 510525.
Mateus, R.J.G., Bana e Costa, J.C., Matos, P.V., 2017.
Supporting Multicriteria Group Decisions with
MACBETH Tools : Selection of Sustainable
Brownfield Redevelopment Actions. Gr. Decis. Negot.
26, 495521.
Oliveira, M.D., Rodrigues, T.C., Bana e Costa, C.A., Brito
de Sá, A., 2012. Prioritizing health care interventions:
A multicriteria resource allocation model to inform the
choice of community care programmes. Advanced
Decision Making Methods Applied to Health Care,
International Series in Operations Research &
Management Science. Springer.
Peterson, T.E., 2012. Constructivist pedagogy and
symbolism: Vico, cassirer, piaget, bateson. Educ.
Philos. Theory 44, 878891.
Phillips, L.D., Bana e Costa, C.A., 2007. Transparent
prioritisation, budgeting and resource allocation with
multi-criteria decision analysis and decision
conferencing. Ann. Oper. Res. Springer Sci. 154, 51
68.
Raiffa, H., 1982. The Art and Science of Negotiation.
Harvard University Press, Cambridge.
Salo, A., 1995. Interactive decision aiding for group
decision support. Eur. J. Oper. Res. 84, 134149.
Salo, A., Keisler, J., Morton, A., 2011. Portfolio Decision
Analysis: Improved Methods for Resource Allocation,
International Series in Operations Research &
Management Science. Springer, New York.
Sarabando, P., Dias, L.C., Vetschera, R., 2013. Mediation
with Incomplete Information: Approaches to Suggest
Potential Agreements. Gr. Decis. Negot. 22, 561597.
Vetschera, R., 2013. Negotiation processes: an integrated
perspective. EURO J. Decis. Process. 1, 135164.
Vilkkumaa, E., Salo, A., Liesiö, J., 2014. Multicriteria
Portfolio Modeling for the Development of Shared
Action Agendas. Gr. Decis. Negot. 23, 4970.
Which Tools Are Needed to Assist Audit Managers in Project Portfolio Selection When Divergent Views Emerge?
345