TESTING IN PARALLEL - A Need for Practical Regression Testing

Zhenyu Zhang, Zijian Tong, Xiaopeng Gao

Abstract

When software evolves, its functionalities are evaluated using regression testing. In a regression testing process, a test suite is augmented, reduced, prioritized, and run on a software build version. Regression testing has been used in industry for decades; while in some modern software activities, we find that regression testing is yet not practical to apply. For example, according to our realistic experiences in Sohu.com Inc., running a reduced test suite, even concurrently, may cost two hours or longer. Nevertheless, in an urgent task or a continuous integration environment, the version builds and regression testing requests may come more often. In such a case, it is not strange that a new round of test suite run needs to start before all the previous ones have terminated. As a solution, running test suites on different build versions in parallel may increase the efficiency of regression testing and facilitate evaluating the fitness of software evolutions. On the other hand, hardware and software resources limit the number of paralleled tasks. In this paper, we raise the problem of testing in parallel, give the general problem settings, and use a pipeline presentation for data visualization. Solving this problem is expected to make practical regression testing.

References

  1. H. Do, G. Rothermel, and A. Kinneer (2006). Prioritizing JUnit test cases: an empirical assessment and costbenefits analysis. Empirical Software Engineering.
  2. S. G. Elbaum, A. G. Malishevsky, and G. Rothermel (2000). Prioritizing test cases for regression testing. In ISSTA 2000.
  3. S. G. Elbaum, A. G. Malishevsky, and G. Rothermel (2002). Test case prioritization: a family of empirical studies. TSE.
  4. S. G. Elbaum, G. Rothermel, S. Kanduri, and A. G. Malishevsky (2004). Selecting a cost-effective test case prioritization technique. Software Quality Control.
  5. B. Jiang, Z. Zhang, W. K. Chan and T. H. Tse (2009a). Adaptive random test case prioritization. In ASE 2009.
  6. B. Jiang, Z. Zhang, T. H. Tse, and T. Y. Chen (2009b). How well do test case prioritization techniques support statistical fault localization. In COMPSAC 2009.
  7. Z. Li, M. Harman, and R. M. Hierons (2007). Search algorithms for regression test case prioritization. TSE.
  8. A. K. Onoma, W.-T. Tsai, M. Poonawala, and H. Suganuma (1998). Regression testing in an industrial environment. Communications of the ACM.
  9. M. K. Ramanathan, M. Koyuturk, A. Grama, and S. Jagannathan (2008). PHALANX: a graph-theoretic framework for test case prioritization. In SAC 2008.
  10. G. Rothermel, S. G. Elbaum, A. G. Malishevsky, P. Kallakuri, and X. Qiu (2004). On test suite composition and costeffective regression testing. TOSEM.
  11. G. Rothermel and M. J. Harrold (1997). A safe, efficient regression test selection technique. TOSEM.
  12. G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold (2001). Prioritizing test cases for regression testing. TSE.
  13. A. Srivastava and J. Thiagarajan (2002). Effectively prioritizing tests in development environment. In ISSTA 2002.
  14. K. R. Walcott, M. L. Soffa, G. M. Kapfhammer, and R. S. Roos (2006). Timeaware test suite prioritization. In ISSTA 2006.
  15. X. Wang, S. C. Cheung, W. K. Chan, and Z. Zhang (2009). Taming coincidental correctness: coverage refinement with context patterns to improve fault localization. In ICSE 2009.
  16. W. E. Wong, J. R. Horgan, S. London, and H. Agrawal (1997). A study of effective regression testing in practice. In ISSRE 1997.
Download


Paper Citation


in Harvard Style

Zhang Z., Tong Z. and Gao X. (2010). TESTING IN PARALLEL - A Need for Practical Regression Testing . In Proceedings of the 5th International Conference on Software and Data Technologies - Volume 2: ICSOFT, ISBN 978-989-8425-23-2, pages 344-348. DOI: 10.5220/0003041503440348


in Bibtex Style

@conference{icsoft10,
author={Zhenyu Zhang and Zijian Tong and Xiaopeng Gao},
title={TESTING IN PARALLEL - A Need for Practical Regression Testing},
booktitle={Proceedings of the 5th International Conference on Software and Data Technologies - Volume 2: ICSOFT,},
year={2010},
pages={344-348},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003041503440348},
isbn={978-989-8425-23-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Software and Data Technologies - Volume 2: ICSOFT,
TI - TESTING IN PARALLEL - A Need for Practical Regression Testing
SN - 978-989-8425-23-2
AU - Zhang Z.
AU - Tong Z.
AU - Gao X.
PY - 2010
SP - 344
EP - 348
DO - 10.5220/0003041503440348