Iterative Mapreduce MPI Oil Reservoir Simulator

Madina Mansurova, Darkhan Akhmed-Zaki, Adai Shomanov, Bazargul Matkerim, Ermek Alimzhanov

2015

Abstract

Paper presents an advanced Iterative MapReduce MPI oil reservoir simulator. First we present an overview of working implementations that make use of the same technologies. Then we define an academic example of numeric problem with an emphasis on its computational features. We present a distributed parallel algorithm of hybrid solution of the problem using MapReduce Hadoop and MPI technologies and describe an improved variant of the algorithm using memory-mapped files.

References

  1. Biardzki, C., Ludwig, T. 2009. Analyzing Metadata Performance in Distributed File Systems. In: Malyshkin, V. (ed.): PACT 2009. LNCS. Vol. 5698: 8-18.
  2. Bu, Y., Howe, B., Balazinska, M., Ernst, M.D., 2012. The HaLoop approach to large-scale iterative data analysis. VLDB Journal 21(2): 169-190.
  3. Chen, S. S., Dongarra, J. J., Hsiung, C. C., 1984. Multiprocessing linear algebra algorithms on the CRAY X-MP-2 - experiences with small granularity. Journal of Parallel and Distributed Computing, 1(1): 22-31.
  4. Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.- H., Qiu, J., and Fox, G., 2010. Twister: a runtime for iterative MapReduce. In: HPDC, pages 810-818. ACM.
  5. Gropp, W., Lusk, E., and Doss, N., 1996. A highperformance, portable implementation of the MPI message passing interface standard. Parallel Computing 22(6): 789-828.
  6. Hoefler, T., Lumsdaine, A., and Dongarra, J., 2009. Towards efficient MapReduce using MPI. In: Ropo, M., Westerholm, J., Dongarra, J. PVM/MPI, Lecture notes in computer science, Springer, 5759: 240-249.
  7. Imankulov, T. S., Mukhambetzhanov, S. T., and AhmedZaki, D.Zh. 2013. Simulation of generalized plane fluid filtration in a deformable environment. Bulletin of the D.Serikbaev East Kazakhstan State Technical University. Computational technologies. Part 1, ? 3, pages 183-191.
  8. Lu, X., Wang, B., Zha, L., and Xu, Z., 2011. Can MPI Benefit Hadoop and MapReduce Applications? In Proceeding ICPPW 7811 Proceedings of the 2011 40th International Conference on Parallel Processing Workshops, pages 371-379.
  9. Mansurova, M., Akhmed-Zaki, D., Kumalakov, B., and Matkerim, B., 2014. Distributed parallel algorithm for numerical solving of 3D problem of fluid dynamics in anisotropic elastic porous medium using MapReduce and MPI technologies. In Proceedings of 9th International Joint Conference on Software Technologies, Vienna, Austria, pages 525-528.
  10. Matkerim, B., Akhmed-Zaki, D., and Barata, M. 2013. Development High Performance Scientific Computing Application Using Model-Driven Architecture. Applied Mathematical Sciences. Vol. 7, ?. 100, pages 4961-4974.
  11. Mohamed, H., and Marchand-Maillet S., 2012. Enhancing MapReduce Using MPI and an Optimized Data Exchange Policy In Proceeding ICPPW 7812 Proceedings of the 2012 41st International Conference on Parallel Processing Workshops, pages 11-18.
  12. Matsunaga, A., Tsugawa, M., Fortes, J. 2008. CloudBLAST: Combining MapReduce and virtualization on distributed resources for bioinformatics applications in eScience'08. In Proceeding IEEE 4th Int. Conference. IEEE.
  13. Mohamed, H., and Marchand-Maillet S., 2012. Distributed media indexing based on MPI and MapReduce. Springer, Science+Business Media.
  14. Slawinski, J., and Sunderam, V., 2012.Adapting MPI to MapReduce PaaS Clouds: An Experiment in CrossParadigm Execution. In Proceedings of the 2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing, pages 199-203.
  15. Srirama, S. N., Batrashev, O., Jakovits, P., et al. 2011. Scalability of parallel scientific applications on the cloud. Scientific programming, 19(2-3): 91-105.
  16. Sunderam, V. S., Geist, G. A., and Dongarra, J., 1994. The PVM concurrent computing system - evolution, experiences, and trends. Parallel Computing 20(4): 531-545.
  17. Tokarev, M. Y., Tyurin, E. A., and Sinitsyn, M. N. 2012. Supercomputer technologies in science, education and industry. Moscow State University.
Download


Paper Citation


in Harvard Style

Mansurova M., Akhmed-Zaki D., Shomanov A., Matkerim B. and Alimzhanov E. (2015). Iterative Mapreduce MPI Oil Reservoir Simulator . In Proceedings of the 10th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2015) ISBN 978-989-758-114-4, pages 456-461. DOI: 10.5220/0005559204560461


in Bibtex Style

@conference{icsoft-ea15,
author={Madina Mansurova and Darkhan Akhmed-Zaki and Adai Shomanov and Bazargul Matkerim and Ermek Alimzhanov},
title={Iterative Mapreduce MPI Oil Reservoir Simulator},
booktitle={Proceedings of the 10th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2015)},
year={2015},
pages={456-461},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005559204560461},
isbn={978-989-758-114-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2015)
TI - Iterative Mapreduce MPI Oil Reservoir Simulator
SN - 978-989-758-114-4
AU - Mansurova M.
AU - Akhmed-Zaki D.
AU - Shomanov A.
AU - Matkerim B.
AU - Alimzhanov E.
PY - 2015
SP - 456
EP - 461
DO - 10.5220/0005559204560461