Reduction in Mutation Testing of Java Classes

Ilona Bluemke, Karol Kulesza

2014

Abstract

In mutation analysis many simple modification of the original program called “mutants” are created. Test cases which are supposed to identify the introduced program changes are designed. Each mutant must be “killed” by a test case, i.e. the test case should detect the purposely introduced modification. Mutation testing is known to be effective but computationally demanding and time consuming because a large number of mutants has to be tested. Mutation score, which is the fraction of mutants that are killed by a test set, is often used to evaluate the effectiveness of mutation testing. An interesting research question is if the number of mutants can be reduced without significantly decreasing the effectiveness of the test. We were exploring selective reductions of mutants generated for Java programs. The results of several experiments conducted in the Eclipse environment are presented in this paper. These results show that selective reduction in mutants can significantly reduce the cost of testing with acceptable mutation score and code coverage.

References

  1. Acree A.T., 1980. On mutation, Ph.D. thesis, Georgia Institute of Technology, Atlanta.
  2. Agrawal H. et al., 1989. Design of mutant operators for the C programming language. Software Engineering Research Center, Purdue University, West Lafayette in Technical Report SERC-TR-41-P, March 1989.
  3. Ammann P., Delamaro M.E., Offutt J., 2014. Establishing Theoretical Minimal Sets of Mutants. In Proc. of the IEEE Int. Conf on Software Testing, Verification, and Validation. 21-30.
  4. Andrews J. H., Briand L. C., Labiche Y., 2005. Is mutation an appropriate tool for testing experiments? In: Proc. ICSE, pp. 402-411.
  5. Bashir B. M., Nadeem A, 2012. Object Oriented Mutation Testing: A Survey. IEEE: 978-1-4673-4451-7/12.
  6. Bluemke I., Kulesza K., 2011. A Comparison of Dataflow and Mutation Testing of Java Methods. In:. Advances in Intelligent and Soft Computing, vol. 97, pp. 17-30, Springer.
  7. Bluemke I., Kulesza K., 2013. Reduction of computational cost in mutation testing by sampling mutants. In Advances in Intelligent and Soft Computing, vol. 224, Springer, pp. 41-51. DOI:10.1007/978-3-319-00945- 2_4
  8. Budd T.A. , 1980. Mutation analysis of program test data. Ph.D. thesis, Yale Univesity, New Haven, Connecticut.
  9. CoffeeMaker, accessed 2012 - http://agile.csc.ncsu.edu/ SEMaterials/tutorials/coffee_maker
  10. Delamaro M.E., Offutt J., Ammann P., 2014. Designing Deletion Mutation Operators. In Proc. of the IEEE Int. Conf on Software Testing, Verification, and Validation. 11-20.
  11. Derezinska A., Rudnik M., 2012. Quality Evaluation of Object-Oriented and Standard Mutation Operators Applied to C# Programs, TOOLS Europe 2012, LNCS 7304, pp. 42-57, Springer.
  12. Derezinska A., 2013. A Quality Estimation of Mutation Clustering in C# Programs. In: New Results in Dependability and Computer Systems, AISC 224, pp. 183-194, Springer.
  13. Frankl P. G., Weiss S. N., Hu C. , 1997. All-uses versus mutation testing: an experimental comparison of effectiveness. J. Syst. Softw., vol. 38, no. 3: 235-253.
  14. Grun B., Schuler D., Zeller A., 2009. The impact of equivalent mutants. In Proceedings of the 4th International Workshop on Mutation Testing.
  15. Hierons R., Harman M., Danicic S., 1999. Using program slicing to assist in the detection of equivalent mutants. Softw. Test. Verif. Rel., vol. 9, no. 4: 233-262.
  16. javasol - http://sourceforge.net/projects/javasol accessed 2012.
  17. Jia Y, Harman M., 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering, vol. 37 no. 5, 649 - 678.
  18. Jumble - http://jumble.sourceforge.net/index.ht, 2010.
  19. Kaminski G. et al. , 2011. A logic mutation approach to selective mutation for programs and queries. Information and Software Technology, vol. 53, pp. 1137-1152.
  20. Kim S., Clark J. A., McDermi J. A., 2001. Investigating the effectiveness of object-oriented testing strategies using the mutation method. In Proceedings of the 1st Workshop on Mutation Analysis (MUTATION'00), published ii Mutation Testing for the New Century. San Jose, California, 6-7 October 2001, pp. 207-225.
  21. Kulesza K, 2012. Mutation testing computational cost reduction using mutants sampling and selective mutation, M.Sc. thesis, Institute of Computer Science, Warsaw University of Technology.
  22. Madeyski L., et al., 2014. Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation. IEEE Trans. on Soft. Eng.,vol. 40: no 1. 23 - 42.
  23. MapMaker - Godlewski L., 2008. , Institute of Computer Science, Warsaw University of Technology, unpublished.
  24. Mateo P. R., Usaola M. P.., 2013. Parallel mutation testing. Softw. Test. Verif. Reliab. vol.23: 315-350.
  25. Mathur A. P., Wong W.E, 1995. Reducing the cost of mutation testing: an empirical study. J. Syst. Softw., vol. 31, no. 3 185-196.
  26. Mresa E.S., Bottaci L., 1999. Efficiency of mutation operators and selective mutation strategies: An empirical study. Soft. Testing, Ver. and Rel. , vol. 9 (4): 205-232.
  27. Mutation repository (modified VII 2011) http://crestweb .cs.ucl.ac.uk/resources/mutation_testing_repository/,
  28. MuClipse, accessed 2012 http://muclipse.sourceforge.net/ index.php .
  29. NetworkShipsBattle , 2010. Network game, Suchowski J Institute of Computer Science, Warsaw University of Technology, unpublished.
  30. Offutt A. J., Craft W. M., 1994. Using compiler optimization techniques to detect equivalent mutants. Softw. Test. Verif. Rel., vol. 4, no. 3: 131-154.
  31. Offutt J., Rothermel G., Zapf C., 1996. An experimental determination of suffcient mutation operators. ACM Trans. on Soft. Eng. and Methodology, vol. 5 (2): 99- 118.
  32. Offutt A. J., Pan J., 1997. Automatically detecting equivalent mutants and infeasible paths. Softw. Test. Verif. Rel., vol. 7, no. 3: 165-192, Sep. 1997
  33. Offutt J., 2011. A mutation carol: Past, present and future, Information and Software Technology vol. 53, pp. 1098-1107.
  34. Patrick M., Oriol M., Clark J.A., 2012. MESSI: Mutant Evaluation by Static Semantic Interpretation. In: Proc. IEEE Fifth Int. Conf. on Software Testing, Verification and Validation. pp. 711- 719.
  35. Segura S., et al.,2011. Mutation testing on an objectoriented framework: An experience report. Inf. and Soft. Technology vol. 53, pp. 1124-1136
  36. Schuler D., Zeller A., 2010. (Un-)covering equivalent mutants. In: Proc. ICST, pp. 45-54.
  37. Scholive M., Beroulle V., Robach C., 2005. Mutation Sampling Technique for the Generation of Structural Test Data. In: Proc. of the Conf. on Design, Automation and Test in Europe, vol. 2: 1022 - 1023.
Download


Paper Citation


in Harvard Style

Bluemke I. and Kulesza K. (2014). Reduction in Mutation Testing of Java Classes . In Proceedings of the 9th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2014) ISBN 978-989-758-036-9, pages 297-304. DOI: 10.5220/0004992102970304


in Bibtex Style

@conference{icsoft-ea14,
author={Ilona Bluemke and Karol Kulesza},
title={Reduction in Mutation Testing of Java Classes},
booktitle={Proceedings of the 9th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2014)},
year={2014},
pages={297-304},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004992102970304},
isbn={978-989-758-036-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2014)
TI - Reduction in Mutation Testing of Java Classes
SN - 978-989-758-036-9
AU - Bluemke I.
AU - Kulesza K.
PY - 2014
SP - 297
EP - 304
DO - 10.5220/0004992102970304