is a restriction considering our definition of the func-
tional fulfillment analysis. A broader trade-off anal-
ysis considering several goals simultaneously, would
have been even more realistic. Another important fact
is that the case study mainly aimed at testing feasibil-
ity of the approach. The models developed have not
been verified; their main role was 1) to provide an
example which demonstrates application of the ap-
proach and 2) to facilitate further improvement and
evaluation of the approach. Therefore, the models
should not be considered as correct, nor should the
case study results be regarded as a security analysis
of SensApp.
One missing part we experienced (which a risk
analysis would have included) is the notion of the ac-
ceptance level regarding the degree of fulfillment. We
could namely not tell whether our best decision alter-
native (or combination of thereof) was good enough,
since it did not fully reach the objective. Another
missing feature of the method was explicit optimiza-
tion with respect to degree of fulfillment and overlap.
Yet another assumption we have made when propos-
ing the degree of overlap, is that several decision alter-
natives can be combined. Decision alternatives may
however not always be compatible, in which case it
will not make sense to consider degree of overlap.
The approach has been proposed and evaluated in
the context of security. Rather than applying the exist-
ing security threat and risk oriented approaches (e.g.,
attack trees (Schneier, 1999), CRAMM (Barber and
Davey, 1992), OCTAVE (Alberts and Davey, 2004),
and CORAS (Lund et al., 2011)) we aimed at ex-
plicitly modeling how security features and measures
contribute to the overall security goal. As such, our
approach is more oriented towards early design of a
system, rather than protection of an existing one. A
challenge of the early design is however lack of em-
pirical data for modeling, particularly estimation of
the weights.
Our success criteria are concerned about correct-
ness, expressiveness, and comprehensibility. As ar-
gued above, correctness needs further evaluation and
explicit uncertainty handling. We were able to model
the objective and all decision alternatives, express all
subgoals, functions and mechanisms in the models,
as well as to analyze both degree of fulfillment and
the degree of overlap. The scale proposed seemed to
provide sufficient intuition to assign estimates to the
objective. Hence, there are indications of expressibil-
ity of the models. Moreover, the active participation
of the domain experts and the fact that they were able
to agree upon the final models, indicates comprehen-
sibility of the approach. Main challenge in develop-
ing of the approach was the balancing of the success
criteria. Practical usefulness requires that the models
are sufficiently informative and correct, at the same
time as being easy to understand for a non-expertuser.
Therefore, we have for the sake of simplicity put some
restrictions on the granularity of the models, and the
amount of the information being visualized in the last
step of the approach. Although our results indicate
practical feasibility of the approach, further evalua-
tion is needed in order to assess validity and reliability
of the approach.
7 CONCLUSIONS AND FUTURE
WORK
We have put forward an approach to functional fulfill-
ment analysis. By functional fulfillment analysis we
mean the analysis of 1) degree of fulfillment of a goal,
and 2) degree of overlap between the decision alter-
natives with respect to the goal. The degree of fulfill-
ment expresses the coverage of the measures support-
ing the goal, while overlap expresses the similarity
between the various decision alternatives with respect
to the measures involved. By considering the degree
of fulfillment and the degree of overlap in a uniform
view, we can select a combination of decision alter-
natives which includes most influential security mea-
sures, at the same time as overlaps are avoided. Over-
lapping measures are particularly relevant to avoid
when repetition imposes additional costs.
The approach has been evaluated in a case study
targeting a system called SensApp. The evaluation in-
dicates feasibility in the sense that the approach could
be applied on a case study and provide useful infor-
mation regarding the performance of the decision al-
ternatives. We were able to model functional capabil-
ities of the goal, and the decision alternatives, as well
as to fully analyze both the degree of fulfillment and
the degree of overlap. Additionally, we were able to
visualize the overall performance of the decision al-
ternatives by employing our approach to visualizing
the decision alternatives.
The comprehensibility and the exspressivness of
the models seemed to be satisfactory in the context of
the case study, while correctness of the models need
further enhancements of the approach. The largest
concern is the lack of confidence in the estimates.
The main threat to our findings is that they are de-
pendent and based on subjective matters. More eval-
uation is furthermore needed in order to address the
threats to validity and reliability, but we believe that
the approach could be useful in the context of deci-
sion making where multiple decision alternatives can
be selected and combined.
TowardsFeature-drivenGoalFulfillmentAnalysis-AFeasibilityStudy
203