on more than one subject, resource, action and en-
vironment.
• It is possible to reduce the number of requests
for both strategies keeping the same test effective-
ness. That means that the introduced stopping cri-
terion is a good upper bound. However, further
criteria for test reduction could be conceived.
• The high variability of the XPT strategy can limit
its performance when policies are very simple and
the stopping criterion of Simple Combinatorial
strategy is assumed. For this, it is needed further
study for achieving a trade-off between the struc-
ture variability and the cardinality of the test suite.
Preliminary results about this have been presented
in (Bertolino et al., 2012).
Note that, the percentage of mutants killed by the
test suite derived by Simple Combinatorial strategy
is the maximum reachable. As it was conceived, it
is not possible to include additional test cases to the
test suite and consequently to get higher value of fault
detection.
Of course such conclusions must be taken in light
of the threats to validity of the performed experiment.
We need to make larger experiments to generalize the
statement as well as consider further mutation opera-
tors than those of (Martin and Xie, 2007b).
From the performed analysis we noticed an im-
pact of the policy specification on the effectiveness
of the derived test suite. Thus, we would like to in-
vestigate other methodologies for requests generation
taking into account this.
In particular, a limitation of Simple Combinato-
rial strategy was that it is not able to detect situa-
tions where the satisfiability of the policy rules de-
pends simultaneously on the values of more than one
entity. We would like to force by construction the re-
quests derived by this strategy to contain all the possi-
ble combinations of more than one subject, resource,
action and environment entity. In this way, the num-
ber of requests increases exponentially and could be
soon comparable to the maximum number of requests
obtained by the XPT testing strategy, i.e MAXREQ
introduced in Section 3.
As a future work, we plan to investigate about the
comparison between XPT approach and this new test
inputs derivation proposal in terms of fault detection
effectiveness.
ACKNOWLEDGEMENTS
This work has been partially funded by the Network
of Excellence on Engineering Secure Future Inter-
net Software Services and Systems (NESSoS) FP7
Project contract n. 256980. We also thank the EC FP7
TAS
3
(Trusted Architecture for Securely Shared Ser-
vices) project for providing us with XACML policies.
REFERENCES
Bertolino, A., Lonetti, F., Daoudagh, S., and Marchetti, E.
(2012). Automatic XACML requests generation for
policy testing. submitted to The Third International
Workshop on Security Testing 2012.
Bertolino, A., Lonetti, F., and Marchetti, E. (2010). Sys-
tematic XACML Request Generation for Testing Pur-
poses. In Proc. of 36th EUROMICRO Conference
on Software Engineering and Advanced Applications
(SEAA), pages 3 –11.
Cohen, D. M., Dalal, S. R., Fredman, M. L., and Patton,
G. C. (1997). The AETG system: An approach to
testing based on combinatiorial design. IEEE Trans.
on Soft. Eng., 23(7):437–444.
DeMillo, R., Lipton, R., and Sayward, F. (1978). Hints on
test data selection: Help for the practicing program-
mer. Computer, 11(4):34–41.
Fisler, K., Krishnamurthi, S., Meyerovich, L., and
Tschantz, M. (2005). Verification and change-impact
analysis of access-control policies. In Proc. of ICSE,
pages 196–205.
Liu, A. X., Chen, F., Hwang, J., and Xie, T. (2011). Design-
ing fast and scalable xacml policy evaluation engines.
IEEE Transactions on Computers, 60(12):1802–1817.
Martin, E. and Xie, T. (2006). Automated test generation
for access control policies. In Supplemental Proc. of
ISSRE.
Martin, E. and Xie, T. (2007a). Automated test generation
for access control policies via change-impact analysis.
In Proc. of Third International Workshop on Software
Engineering for Secure Systems (SESS), pages 5–12.
Martin, E. and Xie, T. (2007b). A fault model and mutation
testing of access control policies. In Proc. of WWW,
pages 667–676.
OASIS (1 Feb 2005). eXtensible Access Control Markup
Language (XACML) Version 2.0. http://docs.oasis-
open.org/xacml/2.0/access control-xacml-2.0-core-
spec-os.pdf.
Ostrand, T. J. and Balcer, M. J. (1988). The category-
partition method for specifying and generating func-
tional tests. Commun. ACM, 31(6):676–686.
Pretschner, A., Mouelhi, T., and Le Traon, Y. (2008).
Model-based tests for access control policies. In Proc.
of ICST, pages 338–347.
Sun Microsystems (2006). Sun’s XACML Implementation.
http://sunxacml.sourceforge.net/.
TAS3 Project (2011). Trusted Architecture for Securely
Shared Services. http://www.tas3.eu/.
Traon, Y., Mouelhi, T., and Baudry, B. (2007). Testing se-
curity policies: going beyond functional testing. In
Proc. of ISSRE, pages 93–102.
WEBIST2012-8thInternationalConferenceonWebInformationSystemsandTechnologies
160