Unified Cloud Orchestration Framework for Elastic High Performance Computing in the Cloud

Lukasz Miroslaw, Michael Pantic, Henrik Nordborg

2016

Abstract

The demand for computational power and storage in industry and academia is continuously increasing. One of the key drivers of this demand is the increased use of numerical simulations, such as Computational Fluid Dynamics for product development. This type of simulations generates huge amounts of data and demands massively parallel computing power. Traditionally, this computational power is provided by clusters, which require large investments in hardware and maintenance. Cloud computing offers more flexibility at significantly lower costs but the deployment of numerical applications is time-consuming, error-prone and requires a high level of expertise. The purpose of this paper is to demonstrate the SimplyHPC framework that automatizes the deployment of the cluster in the cloud, deploys and executes large scale and parallel numerical simulations, and finally downloads the results and shuts down the cluster. Using this tool, we have been able to successfully run the widely accepted solvers, namely PETSc, HPCG and ANSYS CFX, in a performant and scalable manner on Microsoft Azure. It has been shown that the cloud computing performance is comparable to on-premises clusters in terms of efficiency and scalability and should be considered as an economically viable alternative.

References

  1. Ben Belgacem, M. and Chopard, B. (2015). A hybrid HPC/cloud distributed infrastructure: Coupling EC2 cloud resources with HPC clusters to run large tightly coupled multiscale applications. Future Generation Computer Systems, 42:11-21.
  2. Che, Y., Xu, C., Fang, J., Wang, Y., and Wang, Z. (2015). Realistic performance characterization of cfd applications on intel many integrated core architecture. The Computer Journal.
  3. Gentzsch, W. (2012). How cost efficient is hpc in the cloud? Technical report, Ubercloud.
  4. Kaminsky, A. (2015). Solving the World's Toughest Computational Problems with Parallel Computing. Creative Commons Attribution.
  5. Marozzo Fabrizio, Talia Dominico, T. P. (2013). A cloud framework for big data analytics Workflows on Azure . Advances in Parallel Computing. IOS Press.
  6. Miroslaw, L., Baros, V., Pantic, M., and Nordborg, H. (2015). Unified cloud orchestration framework for elastic high performance computing on microsoft azure. In Summary of Proceedings, NAFEMS World Congress, page 216. NAFEMS Ltd.
  7. Pham, L. M., Tchana, A., Donsez, D., de Palma, N., Zurczak, V., and Gibello, P.-Y. (2015). Roboconf: A hybrid cloud orchestrator to deploy complex applications. In Cloud Computing (CLOUD), 2015 IEEE 8th International Conference on, pages 365-372.
  8. Popinet, S. (2003). Gerris: a tree-based adaptive solver for the incompressible Euler equations in complex geometries. Journal of Computational Physics, 190(2):572-600.
  9. Tomczak, T., Zadarnowska, K., Koza, Z., Matyka, M., and Miroslaw, L. (2013). Acceleration of iterative navierstokes solvers on graphics processing units. International Journal of Computational Fluid Dynamics, 27(4-5):201-209.
  10. Wong, A. K. and Goscinski, A. M. (2013). A unified framework for the deployment, exposure and access of HPC applications as services in clouds. Future Generation Computer Systems, 29(6):1333-1344.
Download


Paper Citation


in Harvard Style

Miroslaw L., Pantic M. and Nordborg H. (2016). Unified Cloud Orchestration Framework for Elastic High Performance Computing in the Cloud . In Proceedings of the International Conference on Internet of Things and Big Data - Volume 1: IoTBD, ISBN 978-989-758-183-0, pages 291-298. DOI: 10.5220/0005842402910298


in Bibtex Style

@conference{iotbd16,
author={Lukasz Miroslaw and Michael Pantic and Henrik Nordborg},
title={Unified Cloud Orchestration Framework for Elastic High Performance Computing in the Cloud},
booktitle={Proceedings of the International Conference on Internet of Things and Big Data - Volume 1: IoTBD,},
year={2016},
pages={291-298},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005842402910298},
isbn={978-989-758-183-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Internet of Things and Big Data - Volume 1: IoTBD,
TI - Unified Cloud Orchestration Framework for Elastic High Performance Computing in the Cloud
SN - 978-989-758-183-0
AU - Miroslaw L.
AU - Pantic M.
AU - Nordborg H.
PY - 2016
SP - 291
EP - 298
DO - 10.5220/0005842402910298