Dependability Testing of MapReduce Systems

João Eugenio Marynowski, Andrey Ricardo Pimentel, Taisy Silva Weber, Antonio Junior Mattos


MapReduce systems have been widely used by several applications, from search engines to financial and commercial systems. There is considerable enthusiasm around MapReduce systems due to their simplicity and scalability. However, they lack a testing approach and framework ensuring their dependability. In this work, we propose a complete dependability testing solution for MapReduce systems. This solution is a model-based approach to generate representative fault cases, and a testing framework to automate their execution. Moreover, we introduce a new way to model distributed components using Petri Nets, and we show the promising results of the proposed testing framework, HadoopTest, on identifying faulty systems in real deployment scenarios.


  1. Abouzeid, A., Bajda-Pawlikowski, K., Abadi, D., Silberschatz, A., and Rasin, A. (2009). HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads. In VLDB - International Conference on Very Large Data Bases, pages 922-933. VLDB Endowment.
  2. Ambrosio, A. M., Mattiello-Francisco, F., Vijaykumar, N. L., de Carvalho, S. V., Santiago, V., and Martins, E. (2005). A methodology for designing fault injection experiments as an addition to communication systems conformance testing. In DSN-W - International Conference on Dependable Systems and Networks Workshops, Yokohama, Japan.
  3. Ammann, P. and Offutt, J. (2008). Introduction to Software Testing. Cambridge University Press.
  4. Arlat, J., Crouzet, Y., Karlsson, J., Folkesson, P., Fuchs, E., and Leber, G. (2003). Comparison of Physical and Software-Implemented Fault Injection Techniques. IEEE Transactions on Computers, 52(9):1115-1133.
  5. Avizienis, A., Laprie, J.-C., Randell, B., and Landwehr, C. E. (2004). Basic Concepts and Taxonomy of Dependable and Secure Computing. IEEE Transactions on Dependable and Secure Computing, 1(1):11-33.
  6. Benso, A., Bosio, A., Carlo, S. D., and Mariani, R. (2007). A Functional Verification based Fault Injection Environment. In DFT - International Symposium on Defect and Fault-Tolerance in VLSI Systems, pages 114-122. IEEE.
  7. Bernardi, S., Merseguer, J., and Petriu, D. C. (2012). Dependability Modeling and Assessment in UML-Based Software Development. The Scientific World Journal, 2012:1-11.
  8. Boudnik, K., Rajagopalan, B., and Murthy, A. C. (2010). Herriot. HADOOP-6332.
  9. Callou, G., Maciel, P., Tutsch, D., and Araújo, J. (2012). A Petri Net-Based Approach to the Quantification of Data Center Dependability. In Pawlewski, P., editor, Petri Nets - Manufacturing and Computer Science, page 492. InTech.
  10. Chandra, T. D., Griesemer, R., and Redstone, J. (2007). Paxos Made Live: An Engineering Perspective. In PODC - Symposium on Principles of Distributed Computing, pages 398-407, New York, New York, USA. ACM Press.
  11. Csallner, C., Fegaras, L., and Li, C. (2011). New Ideas Track: Testing MapReduce-Style Programs. In ESEC/FSE'11, Szeged, Hungary.
  12. de Almeida, E. C., Marynowski, J. E., Sunyé, G., and Valduriez, P. (2010a). PeerUnit: a framework for testing peer-to-peer systems. In ASE - International Conference on Automated Software Engineering, pages 169- 170, New York, USA. ACM.
  13. de Almeida, E. C., Sunyé, G., Traon, Y. L., and Valduriez, P. (2010b). Testing peer-to-peer systems. ESE - Empirical Software Engineering, 15(4):346-379.
  14. Dean, J. and Ghemawat, S. (2004). MapReduce: Simplified Data Processing on Large Clusters. In OSDI - USENIX Symposium on Operating Systems Design and Implementation, pages 137-149, San Francisco, California. ACM Press.
  15. Dragan, F., Butnaru, B., Manolescu, I., Gardarin, G., Preda, N., Nguyen, B., Pop, R., and Yeh, L. (2006). P2PTester: a tool for measuring P2P platform performance. In BDA conference.
  16. Echtle, K. and Leu, M. (1994). Test of fault tolerant distributed systems by fault injection. In FTPDS - Workshop on Fault-Tolerant Parallel and Distributed Systems, pages 244-251. IEEE.
  17. Fu, C., Ryder, B. G., Milanova, A., and Wonnacott, D. (2004). Testing of java web services for robustness. In ISSTA - International Symposium on Software Testing and Analysis, pages 23-33.
  18. Huang, S., Huang, J., Dai, J., Xie, T., and Huang, B. (2010). The HiBench benchmark suite: Characterization of the MapReduce-based data analysis. In ICDEW - International Conference on Data Engineering Workshops, pages 41-51. IEEE.
  19. Jacques-Silva, G., Drebes, R., Gerchman, J., F. Trindade, J., Weber, T., and Jansch-Porto, I. (2006). A NetworkLevel Distributed Fault Injector for Experimental Validation of Dependable Distributed Systems. In COMPSAC - International Computer Software and Applications Conference, pages 421-428. IEEE.
  20. Joshi, P., Gunawi, H. S., and Kou (2011). PREFAIL: A Programmable Tool for Multiple-Failure Injection. In OOPSLA - Conference on Object-Oriented Programming, Portland, Oregon, USA.
  21. Lefever, R., Joshi, K., Cukier, M., and Sanders, W. (2004). A global-state-triggered fault injector for distributed system evaluation. IEEE Transactions on Parallel and Distributed Systems, 15(7):593-605.
  22. Marinescu, P. D., Banabic, R., and Candea, G. (2010). An extensible technique for high-precision testing of recovery code. In USENIXATC - Conference on USENIX Annual Technical Conference, page 23. USENIX.
  23. Natella, R., Cotroneo, D., Duraes, J. A., and Madeira, H. S. (2012). On Fault Representativeness of Software Fault Injection. TSE - IEEE Transactions on Software Engineering.
  24. Offutt, A. J. (1994). A Practical System for Mutation Testing: Help for the Common Programmer. In ITC - International Test Conference, pages 824-830. IEEE.
  25. Pan, X., Tan, J., Kalvulya, S., Gandhi, R., and Narasimhan, P. (2009). Blind Men and the Elephant: Piecing Together Hadoop for Diagnosis. In ISSRE - International Symposium on Software Reliability Engineering.
  26. Pan, X., Tan, J., Kavulya, S., Gandhi, R., and Narasimhan, P. (2010). Ganesha: blackBox diagnosis of MapReduce systems. In SIGMETRICS Performance Evaluation Review, page 8. ACM Press.
  27. Pham, C., Chen, D., Kalbarczyk, Z., and Iyer, R. K. (2011). CloudVal: A framework for validation of virtualization environment in cloud infrastructure. In DSN - International Conference on Dependable Systems and Networks, pages 189-196. IEEE.
  28. Stott, D., Floering, B., Burke, D., Kalbarczpk, Z., and Iyer, R. (2000). NFTAPE: a framework for assessing dependability in distributed systems with lightweight fault injectors. In IPDS - International Computer Performance and Dependability Symposium, pages 91- 100. IEEE Comput. Soc.
  29. Tan, J., Pan, X., Kavulya, S., Gandhi, R., and Narasimhan, P. (2008). SALSA: analyzing logs as state machines. In WASL - Conference on Analysis of System Logs, page 6, CA, USA. USENIX.
  30. Tan, J., Pan, X., Kavulya, S., Gandhi, R., and Narasimhan, P. (2009). Mochi: visual log-analysis based tools for debugging hadoop. In Proceedings of the 2009 conference on Hot topics in cloud computing, pages 18-18. USENIX Association.
  31. Teradata Coorporation (2012). MapReduce, SQLMapReduce Resources and Hadoop Integration - Aster Data. mapreduce.php, 09/02/12.
  32. Zhou, Z., Wang, H., Zhou, J., Tang, L., and Li., K. (2006). Pigeon: A Framework for Testing Peer-to-Peer Massively Multiplayer Online Games over Heterogeneous Network. In CCNC - Consumer Communications and Networking Conference. IEEE.

Paper Citation

in Harvard Style

Marynowski J., Pimentel A., Weber T. and Mattos A. (2013). Dependability Testing of MapReduce Systems . In Proceedings of the 15th International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-8565-60-0, pages 165-172. DOI: 10.5220/0004436101650172

in Bibtex Style

author={João Eugenio Marynowski and Andrey Ricardo Pimentel and Taisy Silva Weber and Antonio Junior Mattos},
title={Dependability Testing of MapReduce Systems},
booktitle={Proceedings of the 15th International Conference on Enterprise Information Systems - Volume 2: ICEIS,},

in EndNote Style

JO - Proceedings of the 15th International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - Dependability Testing of MapReduce Systems
SN - 978-989-8565-60-0
AU - Marynowski J.
AU - Pimentel A.
AU - Weber T.
AU - Mattos A.
PY - 2013
SP - 165
EP - 172
DO - 10.5220/0004436101650172