ON-THE-FLY INTERPRETATION OF TEST CASES IN AN AUTOMATICALLY GENERATED TTCN-3 TEST SUITE

Winfried Dulz

Abstract

The TestUS framework (Statistical Testing based on use case scenarios) offers unique techniques and tools to obtain a TTCN-3 test suite starting from UML 2.0 requirement definitions. Use case diagrams that contain functional and non-functional requirements are transformed to a Markov Chain usage model (MCUM) in a completely automatic approach. The annotation of outgoing MCUM transitions by probabilities in the derived UML2 protocol state machine enables the generation of TTCN-3 test cases according to the expected occurrence frequencies of the specified usage pattern. However, compiling the derived TTCN-3 test suite can take quite a long time for a realistic SUT (System under Test). Consequently, we decided to map the MCUM directly into the executable test suite without generating test cases in advance. Test cases and the evaluation of test verdicts are therefore interpreted on-the-fly inside the executable TTCN-3 test suite. We proved the concept by testing an existing DECT communication system. The compilation time in the order of 20 hours for deriving the test suite was reduced to only 15 minutes and we got a TTCN-3 test suite that interprets as many test cases as one likes for the DECT system on-the-fly.

References

  1. Sayre, K., 1999. Improved Techniques for Software Testing Based on Markov Chain Usage Models. PhD thesis, University of Tennessee, Knoxville.
  2. Whittaker, J. A., Poore, J.H., C. J. Trammel, 1995. Statistical testing of software based on a usage model. Software-practice and experience.
  3. Broy, M., Jonsson, B., Katoen, J.-P. (eds), 2005. ModelBased Testing of Reactive Systems. Springer LNCS 3472.
  4. OMG, 2007. Unified Modeling Language: Superstructure. version 2.1.1.
  5. ETSI, 2005. Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language. ES 201 873-1 V3.1.1.
  6. Tretmans, J., Brinksma, E., 2002. Automated Model Based Testing. University of Twente.
  7. Beyer, M., Dulz, W., 2005. Scenario-Based Statistical Testing of Quality of Service Requirements. Springer LNCS 3466.
  8. Walton, G. H., Poore, J.H., 2000. Generating transition probabilities to support model-based software testing. Software - Practice and Experience, 30, p. 1095-1106.
  9. Musa, J. D., 1993. Operational Profiles in SoftwareReliability Engineering. IEEE Software.
  10. Gutjahr, Walter J., 1997. Importance Sampling of Test Cases in Markovian Software Usage Models. Department of Statistics, Operations Research and Computer Science, University of Vienna.
  11. Beyer, M., Dulz, W., Hielscher, K.-S. J., 2006. Performance Issues in Statistical Testing. Proceedings 3th GI/ITG Conference on Measurement, Modeling, and Evaluation of Computer and Communication Systems (MMB 2006), Nuremberg, Germany.
  12. Biegel, M., 2006. StatisticalTesting of DECT Modules. Proceedings ITG Workshop on Model-Based Testing, Nuremberg, Germany.
Download


Paper Citation


in Harvard Style

Dulz W. (2008). ON-THE-FLY INTERPRETATION OF TEST CASES IN AN AUTOMATICALLY GENERATED TTCN-3 TEST SUITE . In Proceedings of the Third International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE, ISBN 978-989-8111-28-9, pages 72-80. DOI: 10.5220/0001762800720080


in Bibtex Style

@conference{enase08,
author={Winfried Dulz},
title={ON-THE-FLY INTERPRETATION OF TEST CASES IN AN AUTOMATICALLY GENERATED TTCN-3 TEST SUITE},
booktitle={Proceedings of the Third International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE,},
year={2008},
pages={72-80},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001762800720080},
isbn={978-989-8111-28-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Third International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE,
TI - ON-THE-FLY INTERPRETATION OF TEST CASES IN AN AUTOMATICALLY GENERATED TTCN-3 TEST SUITE
SN - 978-989-8111-28-9
AU - Dulz W.
PY - 2008
SP - 72
EP - 80
DO - 10.5220/0001762800720080