Multiple search phrases should be tried, as different
words may be used to describe equivalent
functionalities. The search should be continued until
no more matching solutions can be found.
3.2 Stage 2: Evaluate found Solutions
Phase 2.1: Specifying evaluation Criteria. The
evaluation criteria are specified by assigning
measurable fulfilment levels to the requirements.
Every criterion should have at least one defined
fulfilment level: acceptable; that is, solutions
evaluated below this level must not be chosen.
However, there could be more levels defined, so that
solutions that pass the acceptable level could be
compared between each other. A satisfactory level
can be defined, such that no solution can be
evaluated as better with regards to a given criterion
if they both attain this level.
The evaluation criteria can be grouped,
depending on their measurability, into: (1) objective
and easily measurable, (2) objective and not easily
measurable, (3) subjective.
The weights of criteria may be established using
a simple ranking done by end users, or using more
sophisticated approaches, like pair-wise comparisons
(see, e.g., the AHP method – Saaty, 1980).
Phase 2.2: Preliminary evaluation. The goal of
the preliminary evaluation is to shorten the candidate
list by removing software that does not attain
acceptable level for criteria based on the core
requirements. In this phase only objective and easily
measurable criteria should be considered.
Phase 2.3: Main evaluation. This phase starts
with eliminating software that does not attain
acceptable level for objective criteria (now also not
easily measurable) based on the core requirements.
Both the criteria fulfilment levels and criteria
weights are scaled and normalized, and then used to
construct an aggregate measure for each candidate
solution and group of criteria (core criteria,
combined core and additional criteria, and all
criteria). One or two highest-evaluated solutions are
chosen for each group of criteria.
Phase 2.4: In-depth evaluation. The goal of this
phase is to prepare full information that will be
considered for making the choice, including
subjective criteria. Evaluation is done by several
people, working as a team or independently – the
results are averaged in the second case. In contrast to
the two previous phases, the end users should be
involved in the evaluation.
3.3 Stage 3: Choose the Most
Appropriate Solution
Phase 3.1: Discussion of evaluation Results. A
meeting of the stakeholders should be organised that
starts with presentation of the evaluation results after
which every person should express their opinion and
provide additional information that could impact the
choice, e.g.:
the end users should point to drawbacks or
special advantages of respective solutions,
the invited experts should clarify, if the
mentioned drawbacks are specific to solutions,
or they are merely results of improper usage,
the members of the development team should
declare if they are capable of fixing the
drawbacks, and what the cost would be ,
the sponsors of the project should declare if they
will contribute necessary resources.
Phase 3.2: Making the Choice. The decision on
choosing the solution for adoption is made by the
sponsors of the project. It should be based on the
results of the evaluation process, but if two or more
solutions were evaluated closely, the decision
maker(s) should pick one of them using their own
opinion rather than the aggregated measure value.
3.4 Stage 4: Adapt the Solution
Adapting the solution consists of acquiring,
installing, and configuring the core solution, then the
required add-ons, and, finally, applying
modifications required for the solution to meet the
requirements.
What makes the modifications applied at this
stage distinctive from those of the next stage is that
they are aimed at the requirements of the specific
end user group, and as such they will rarely be
useful for other users, and that they are not usually
defined as a separate entity (module or even
function), being often a list of file/line updates.
Every modification made to the original solution
must be explained in the technical documentation of
the final product, in terms of its purpose, relation to
other modifications, assumed conditions and
possible risk factors. Regression testing should
assure that the modification does not hurt stability,
security or degrade performance of the system.
3.5 Stage 5: Develop New Modules
Sometimes, the modifications related to a certain
group of requirements can be implemented as a new
module for the chosen solution.
AnOutlineofDevelopmentProcessFrameworkforSoftwarebasedonOpen-sourceComponents
185