ELUSIVE BUGS, BOUNDED EXHAUSTIVE TESTING AND INCOMPLETE ORACLES

W. E. Howden

Abstract

Elusive bugs involve combinations of conditions that may not fit into any informal or intuitive testing scheme. One way to attack them is with Bounded Exhaustive Testing, in which all combinations of inputs for a bounded version of an application are tested. Studies of BET effectiveness for known bugs indicate that it is a promising approach. Because of the numbers of tests that are involved, BET normally depends on automated test generation and execution. This in turn requires the use of an automated oracle. In some cases the construction of a complete automated oracle would require the development of a second version of the application. This may be avoidable if incomplete oracles are used. Two classes of incomplete oracles are identified: necessity and sufficiency oracles. Examples are given of experiments using a necessity and a sufficiency oracle.

References

  1. Barzin, R., Fukushima, S., Howden, W., Sharifi, S., SuperFIT Combinational Elusive Bug Detection, Proceedings, COMPSAC 2008, IEEE, 2008.
  2. Boyapati, C., Khurshid, S., Marinov, D., Korat: Automated Testing Based on Java Predicates, Procs. ISSTA, IEEE, 2002.
  3. Cheon, Y., Leavens, G. A Simple and Practical Approach to Unit Testing: The JML and the JUnit Way, In ECOOP 2002 -- Object-Oriented Programming, 16th European Conference, Malaga, Spain, June 2002, Proceedings. Volume 2374 of Lecture Notes in Computer Science, Springer-Verlag, 2002.
  4. Howden, William E., Functional Program Testing, IEEETSE, 6-2, 1980.
  5. Howden, W.E. Introduction to the Theory of Testing, in Software Testing and Validation Techniques, E. Miller and William E. Howden, IEEE, 1978.
  6. Howden, W.E., Rhyne, C., Test Frameworks for Elusive Bug Testing, Proceedings ICSOFT 07, Barcelona, 2007.
  7. Howden, William E., Symbolic Evaluation and the DISSECT Symbolic Evaluation System, IEEE TSE, SE-3, 4, July, 1977.
  8. Memon A., Banerjee I., A. Ngarajan, A., What Test Oracles Should I use for Effective GUI Testing? IEEE TSE, 31-10, Oct 2005.
  9. Miller, B., Forrester J.E., and Miller, B.P., An empirical study of the robustness of Windows NT applications using random testing, Proc. 4th Usenix Windows System Symposium, 2000.
  10. Mugridge, R., Cunningham W., FIT for Developing Software: Framework for Integrated Tests, Prentice Hall, 2005.
  11. Richardson, D.J., TAOS: Testing with analysis and oracle support, ISSTA: Proceedings of the International Symposium on Software Testing and Analysis, ACM, 1994.
  12. Sullivan, K, J., Yang, J., Coppit, D., Khurshid, S., Jackson, D., Software Assurance by Bounded Exhaustive Testing, Proc. ISSTA, 2004.
  13. Weyuker, E.J. On testing non-testable programs, The Computing Journal, 25-4, 1982.
Download


Paper Citation


in Harvard Style

E. Howden W. (2008). ELUSIVE BUGS, BOUNDED EXHAUSTIVE TESTING AND INCOMPLETE ORACLES . In Proceedings of the Third International Conference on Software and Data Technologies - Volume 2: ICSOFT, ISBN 978-989-8111-52-4, pages 115-121. DOI: 10.5220/0001868501150121


in Bibtex Style

@conference{icsoft08,
author={W. E. Howden},
title={ELUSIVE BUGS, BOUNDED EXHAUSTIVE TESTING AND INCOMPLETE ORACLES},
booktitle={Proceedings of the Third International Conference on Software and Data Technologies - Volume 2: ICSOFT,},
year={2008},
pages={115-121},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001868501150121},
isbn={978-989-8111-52-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Third International Conference on Software and Data Technologies - Volume 2: ICSOFT,
TI - ELUSIVE BUGS, BOUNDED EXHAUSTIVE TESTING AND INCOMPLETE ORACLES
SN - 978-989-8111-52-4
AU - E. Howden W.
PY - 2008
SP - 115
EP - 121
DO - 10.5220/0001868501150121