CONSTRAINT REASONING IN FOCALTEST

Matthieu Carlier, Catherine Dubois, Arnaud Gotlieb

2010

Abstract

Property-based testing implies selecting test data satisfying coverage criteria on user-specified properties. However, current automatic test data generation techniques adopt direct generate-and-test approaches for this task. In FocalTest, a testing tool designed to generate test data for programs and properties written in the functional language Focal, test data are generated at random and rejected when they do not satisfy selected coverage criteria. In this paper, we improve FocalTest with a test-and-generate approach, through the usage of constraint reasoning. A particular difficulty is the generation of test data satisfying MC/DC on the precondition of a property, when it contains function calls with pattern matching and higher-order functions. Our experimental results show that a non-naive implementation of constraint reasoning on these constructions outperform traditional generation techniques when used to find test data for testing properties.

References

  1. Ayrault, P., Hardin, T., and Pessaux, F. (2008). Development life cycle of critical software under focal. In Int. Workshop on Harnessing Theories for Tool Support in Software, TTSS.
  2. Carlier, M. (2009). Test. CEDRIC http://cedric.cnam.fr.
  3. Carlier, M. and Dubois, C. (2008). Functional testing in the focal environment. In Test And Proof, TAP.
  4. Christiansen, J. and Fischer, S. (2008). Easycheck - test data for free. In 9th Int. Symp. on Func. and Logic Prog, FLOPS.
  5. Claessen, K. and Hughes, J. (2000). Quickcheck: a lightweight tool for random testing of haskell programs. ACM SIGPLAN Notices, 35(9):268-279.
  6. Dick, J. and Faivre, A. (1993). Automating the generation and sequencing of test cases from model-based specifications. In First Int. Symp. of Formal Methods Europe, FME, pages 268-284.
  7. Dubois, C., Hardin, T., and Viguié Donzeau-Gouge, V. (2006). Building certified components within focal. In Fifth Symp. on Trends in Functional Prog., TFP'04, volume 5, pages 33-48.
  8. Fink, G. and Bishop, M. (1997). Property-based testing: a new approach to testing for assurance. SIGSOFT Softw. Eng. Notes, 22(4):74-80.
  9. Fischer, S. and Kuchen, H. (2007). Systematic generation of glass-box test cases for functional logic programs. In Conf. on Princ. and Practice of Declarative Programming (PPDP'07), pages 63-74.
  10. Fischer, S. and Kuchen, H. (2008). Data-flow testing of declarative programs. In Proc. of ICFP'08), pages 201-212.
  11. Godefroid, P., Klarlund, N., and Sen, K. (2005). Dart: directed automated random testing. In ACM Conf. on Prog. lang. design and impl., PLDI, pages 213-223.
  12. Gotlieb, A. (2009). Euclide: A constraint-based testing platform for critical c programs. In Int. Conf. on Software Testing, Validation and Verification, ICST.
  13. Gotlieb, A., Botella, B., and Rueher, M. (1998). Automatic test data generation using constraint solving techniques. In Int. Symp. on Soft. Testing and Analysis, ISSTA, pages 53-62.
  14. Johnsson, T. (1985). Lambda lifting: Transforming programs to recursive equations. In Conference on Functional Programming Languages and Computer Architecture, pages 190-203. Springer-Verlag.
  15. Koopman, P., Alimarine, A., Tretmans, J., and Plasmeijer, R. (2002). Gast: Generic automated software testing. In Workshop on the Impl. of Func. Lang., IFL02, pages 84-100. Springer.
  16. Legeard, B. and Peureux, F. (2001). Generation of functional test sequences from B formal specifications - presentation and industrial case-study. In Int. Conf. on Automated Soft. Eng., ASE01, pages 377-381.
  17. Marre, B. (1991). Toward Automatic Test Data Set Selection using Algebraic Specifications and Logic Programming. In K. Furukawa, editor, Int. Conf. on Logic Programming, ICLP, pages 202-219.
  18. Pretschner, A. (2001). Classical search strategies for test case generation with constraint logic programming. In Formal Approaches to Testing of Soft., FATES, pages 47-60.
  19. Sen, K., Marinov, D., and Agha, G. (2005). Cute: a concolic unit testing engine for c. In ESEC/FSE-13, pages 263- 272. ACM Press.
  20. Uzuncaova, E. and Khurshid, S. (2008). Constraint prioritization for efficient analysis of declarative models. In 15th Int. Symp. on Formal Methods, FM.
  21. Williams, N., Marre, B., Mouy, P., and Roger, M. (2005). Pathcrawler: Automatic generation of path tests by combining static and dynamic analysis. In Dependable Computing, EDCC, pages 281-292.
Download


Paper Citation


in Harvard Style

Carlier M., Dubois C. and Gotlieb A. (2010). CONSTRAINT REASONING IN FOCALTEST . In Proceedings of the 5th International Conference on Software and Data Technologies - Volume 2: ICSOFT, ISBN 978-989-8425-23-2, pages 82-91. DOI: 10.5220/0003009800820091


in Bibtex Style

@conference{icsoft10,
author={Matthieu Carlier and Catherine Dubois and Arnaud Gotlieb},
title={CONSTRAINT REASONING IN FOCALTEST},
booktitle={Proceedings of the 5th International Conference on Software and Data Technologies - Volume 2: ICSOFT,},
year={2010},
pages={82-91},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003009800820091},
isbn={978-989-8425-23-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Software and Data Technologies - Volume 2: ICSOFT,
TI - CONSTRAINT REASONING IN FOCALTEST
SN - 978-989-8425-23-2
AU - Carlier M.
AU - Dubois C.
AU - Gotlieb A.
PY - 2010
SP - 82
EP - 91
DO - 10.5220/0003009800820091