Distributed Parallel Algorithm for Numerical Solving of 3D Problem of Fluid Dynamics in Anisotropic Elastic Porous Medium Using MapReduce and MPI Technologies

Madina Mansurova, Darkhan Akhmed-Zaki, Matkerim Bazargul, Bolatzhan Kumalakov

2014

Abstract

Paper presents an advanced iterative MapReduce solution that employs Hadoop and MPI technologies. First, we present an overview of working implementations that make use of the same technologies. Then we define an academic example of numeric problem with an emphasis on its computational features. The named definition is used to justify the proposed solution design.

References

  1. Becker, J. C. and Dagum, L. (1992). Particle simulation on heterogeneous distributed supercomputers. In HPDC, pages 133-140.
  2. Biardzki, C. and Ludwig, T. (2009). Analyzing metadata performance in distributed file systems. In Malyshkin, V., editor, Parallel Computing Technologies (10th PaCT'09), volume 5698 of Lecture Notes in Computer Science (LNCS), pages 8-18. Springer-Verlag (New York), Novosibirsk, Russia.
  3. Bu, Y., Howe, B., Balazinska, M., and Ernst, M. D. (2012). The haloop approach to large-scale iterative data analysis. VLDB J, 21(2):169-190.
  4. Cappello, F., Djilali, S., Fedak, G., Hérault, T., Magniette, F., Néri, V., and Lodygensky, O. (2005). Computing on large-scale distributed systems: Xtremweb architecture, programming models, security, tests and convergence with grid. Future Generation Comp. Syst, 21(3):417-437.
  5. Chen, S. S., Dongarra, J. J., and Hsiung, C. C. (1984). Multiprocessing linear algebra algorithms on the Cray XMP-2: Experiences with small granularity. Journal of Parallel and Distributed Computing, 1(1):22-31.
  6. Cohen, J. (2009). Graph twiddling in a mapreduce world. Computing in Science and Engineering, 11(4):29-41.
  7. Dean, J. and Ghemawat, S. (2008). MapReduce: simplified data processing on large clusters. CACM, 51(1):107- 113.
  8. Díaz, J., Mun˜oz-Caro, C., and Nin˜o, A. (2012). A survey of parallel programming models and tools in the multi and many-core era. IEEE Trans. Parallel Distrib. Syst, 23(8):1369-1386.
  9. Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.- H., Qiu, J., and Fox, G. (2010). Twister: a runtime for iterative mapreduce. In Hariri, S. and Keahey, K., editors, HPDC, pages 810-818. ACM.
  10. Fagg, G. E. and Dongarra, J. (2000). FT-MPI: Fault tolerant MPI, supporting dynamic applications in a dynamic world. In Dongarra, J., Kacsuk, P., and Podhorszki, N., editors, PVM/MPI, volume 1908 of Lecture Notes in Computer Science, pages 346-353. Springer.
  11. Fougère, D., Gorodnichev, M., Malyshkin, N., Malyshkin, V. E., Merkulov, A. I., and Roux, B. (2005). Numgrid middleware: MPI support for computational grids. In Malyshkin, V. E., editor, PaCT, volume 3606 of Lecture Notes in Computer Science, pages 313-320. Springer.
  12. Gropp, W., Lusk, E. L., Doss, N. E., and Skjellum, A. (1996). A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing, 22(6):789-828.
  13. Jin, H. and Sun, X.-H. (2013). Performance comparison under failures of MPI and mapreduce: An analytical approach. Future Generation Comp. Syst, 29(7):1808- 1815.
  14. Liu, H. and Orban, D. (2008). Gridbatch: Cloud computing for large-scale data-intensive batch applications. In CCGRID, pages 295-305. IEEE Computer Society.
  15. Lu, X., Wang, B., Zha, L., and Xu, Z. (2011). Can MPI benefit hadoop and mapreduce applications? In Sheu, J.- P. and Wang, C.-L., editors, ICCP Workshops, pages 371-379. IEEE.
  16. Malyshkin, V. (2010). Assembling of Parallel Programs for Large Scale Numerical Modelling. IGI Global, Chicago, USA.
  17. Matsunaga, A. M., Tsugawa, M. O., and Fortes, J. A. B. (2008). CloudBLAST: Combining mapreduce and virtualization on distributed resources for bioinformatics applications. In eScience, pages 222-229. IEEE Computer Society.
  18. Mohamed, H. and Marchand-Maillet, S. (2012). Enhancing mapreduce using MPI and an optimized data exchange policy. In ICPP Workshops, pages 11-18. IEEE Computer Society.
  19. Ng, R. T. and Han, J. (1994). Efficient and effective clustering methods for spatial data mining. Technical Report TR-94-13, Department of Computer Science, University of British Columbia. Tue, 22 Jul 1997 22:21:44 GMT.
  20. Pandey, S. and Buyya, R. (2012). Scheduling workflow applications based on multi-source parallel data retrieval in distributed computing networks. Comput. J, 55(11):1288-1308.
  21. Slawinski, J. and Sunderam, V. S. (2012). Adapting MPI to mapreduce paaS clouds: An experiment in crossparadigm execution. In UCC, pages 199-203. IEEE.
  22. Srirama, S. N., Batrashev, O., Jakovits, P., and Vainikko, E. (2011). Scalability of parallel scientific applications on the cloud. Scientific Programming, 19(2-3):91- 105.
  23. Sunderam, V. S., Geist, G., and Dongarra, J. (1994). The PVM concurrent computing system: evolution, experiences, and trends. Parallel Computing, 20(4):531- 545.
  24. Valilai, O. and Houshmand, M. (2013). A collaborative and integrated platform to support distributed manufacturing system using a service-oriented approach based on cloud computing paradigm. Robotics and computerintegrated manufacturing, 1(29):110-127.
  25. Wang, J. and Liu, Z. (2008). Parallel data mining optimal algorithm of virtual cluster. In Ma, J., Yin, Y., Yu, J., and Zhou, S., editors, FSKD (5), pages 358-362. IEEE Computer Society.
Download


Paper Citation


in Harvard Style

Mansurova M., Akhmed-Zaki D., Bazargul M. and Kumalakov B. (2014). Distributed Parallel Algorithm for Numerical Solving of 3D Problem of Fluid Dynamics in Anisotropic Elastic Porous Medium Using MapReduce and MPI Technologies . In Proceedings of the 9th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2014) ISBN 978-989-758-036-9, pages 524-528. DOI: 10.5220/0005110605240528


in Bibtex Style

@conference{icsoft-ea14,
author={Madina Mansurova and Darkhan Akhmed-Zaki and Matkerim Bazargul and Bolatzhan Kumalakov},
title={Distributed Parallel Algorithm for Numerical Solving of 3D Problem of Fluid Dynamics in Anisotropic Elastic Porous Medium Using MapReduce and MPI Technologies},
booktitle={Proceedings of the 9th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2014)},
year={2014},
pages={524-528},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005110605240528},
isbn={978-989-758-036-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2014)
TI - Distributed Parallel Algorithm for Numerical Solving of 3D Problem of Fluid Dynamics in Anisotropic Elastic Porous Medium Using MapReduce and MPI Technologies
SN - 978-989-758-036-9
AU - Mansurova M.
AU - Akhmed-Zaki D.
AU - Bazargul M.
AU - Kumalakov B.
PY - 2014
SP - 524
EP - 528
DO - 10.5220/0005110605240528