Container-based Virtualization for HPC

Holger Gantikow, Sebastian Klingberg, Christoph Reich

2015

Abstract

Experts argue that the resource demands of High Performance Computing (HPC) clusters request bare-metal installations. The performance loss of container virtualization is minimal and close to bare-metal, but in comparison has many advantages, like ease of provisioning. This paper presents the use of the newly adopted container technology and its multiple conceptional advantages for HPC, compared to traditional bare-metal installations or the use of VMs. The setup based on Docker (Docker, 2015) shows a possible use in private HPC sites or public clouds as well. The paper ends with a performance comparison of a FEA job run both bare-metal and using Docker and a detailed risk analysis of Docker installations in a multi-tenant environment, as HPC sites usually are.

References

  1. Abaqus FEA, S. (2015). www.simulia.com.
  2. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., and Warfield, A. (2003). Xen and the art of virtualization. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP 7803, pages 164-177, New York, NY, USA. ACM.
  3. Biederman, E. W. (2006). Multiple instances of the global linux namespaces. In Proceedings of the 2006 Ottawa Linux Symposium, Ottawa Linux Symposium, pages 101-112.
  4. Bui, T. (2015). Analysis of docker security. abs/1501.02967.
  5. Chef (2015). Chef: Automation for Web-Scale IT. https:// www.chef.io/.
  6. Clark, C., Fraser, K., Hand, S., Hansen, J. G., Jul, E., Limpach, C., Pratt, I., and Warfield, A. (2005). Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2, NSDI'05, pages 273-286, Berkeley, CA, USA. USENIX Association.
  7. CRIU-Project (2015). Checkpoint/Restore In Userspace (CRIU). http://www.criu.org/.
  8. Docker (2015). Docker. https://www.docker.com/.
  9. Felter, W., Ferreira, A., Rajamony, R., and Rubio, J. (2014). An updated performance comparison of virtual machines and linux containers. technology, page 28:32.
  10. IBM (2015). LSF. http://www-03.ibm.com/systems/ platformcomputing/products/lsf/.
  11. Jackson, I. (2015). Surviving the zombie apocalypse - security in the cloud containers, kvm and xen. http:// xenbits.xen.org/people/iwj/2015/fosdem-security/.
  12. Jay, T. (2014). Before you initiate a docker pull. https:// securityblog.redhat.com/2014/12/18/before-youinitiate-a-docker-pull/.
  13. Jéroˆme Petazzoni (2013). Containers & Docker: How Secure Are They? https://blog.docker.com/2013/08/ containers-docker-how-secure-are-they/.
  14. Kivity, A., Kamay, Y., Laor, D., Lublin, U., and Liguori, A. (2007). kvm: the linux virtual machine monitor. In Proceedings of the Linux Symposium, volume 1, pages 225-230, Ottawa, Ontario, Canada.
  15. Matthews, J. N., Hu, W., Hapuarachchi, M., Deshane, T., Dimatos, D., Hamilton, G., McCabe, M., and Owens, J. (2007). Quantifying the performance isolation properties of virtualization systems. In Proceedings of the 2007 Workshop on Experimental Computer Science, ExpCS 7807, New York, NY, USA. ACM.
  16. McDougall, R. and Anderson, J. (2010). Virtualization performance: Perspectives and challenges ahead. SIGOPS Oper. Syst. Rev., 44(4):40-56.
  17. Miller, F., Vandome, A., and John, M. (2010). FreeBSD Jail. VDM Publishing.
  18. MPI (2015). Message Passing Interface (MPI) standard. http://www.mcs.anl.gov/research/projects/mpi/.
  19. Oracle (2015). Grid Engine. http://www.oracle.com/us/ products/tools/oracle-grid-engine-075549.html.
  20. Padala, P., Zhu, X., Wang, Z., Singhal, S., Shin, K. G., Padala, P., Zhu, X., Wang, Z., Singhal, S., and Shin, K. G. (2007). Performance evaluation of virtualization technologies for server consolidation. Technical report.
  21. Pék, G., Buttyán, L., and Bencsáth, B. (2013). A survey of security issues in hardware virtualization. ACM Comput. Surv., 45(3):40:1-40:34.
  22. Price, D. and Tucker, A. (2004). Solaris zones: Operating system support for consolidating commercial workloads. In Proceedings of the 18th Conference on Systems Administration (LISA 2004), Atlanta, USA, November 14-19, 2004, pages 241-254.
  23. Puppet (2015). puppet: Automate IT. http:// puppetlabs.com/.
  24. Quintero, D., Brandon, S., Buehler, B., Fauck, T., Felix, G., Gibson, C., Maher, B., Mithaiwala, M., Moha, K., Mueller, M., et al. (2011). Exploiting IBM AIX Workload Partitions. IBM redbooks. IBM Redbooks.
  25. Reshetova, E., Karhunen, J., Nyman, T., and Asokan, N. (2014). Security of os-level virtualization technologies: Technical report. CoRR, abs/1407.4245.
  26. Rudenberg, J. (2014). Docker image insecurity. https:// titanous.com/posts/docker-insecurity.
  27. Russell, R. (2008). Virtio: Towards a de-facto standard for virtual i/o devices. SIGOPS Oper. Syst. Rev., 42(5):95-103.
  28. Stanfield, J. and Dandapanthula, N. (2014). HPC in an OpenStack Environment.
  29. Unionfs (2015). Unionfs: A Stackable Unification File System. http://unionfs.filesystems.org.
  30. Xavier, M., Neves, M., Rossi, F., Ferreto, T., Lange, T., and De Rose, C. (2013). Performance evaluation of container-based virtualization for high performance computing environments. In Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference on, pages 233-240.
Download


Paper Citation


in Harvard Style

Gantikow H., Klingberg S. and Reich C. (2015). Container-based Virtualization for HPC . In Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-104-5, pages 543-550. DOI: 10.5220/0005485605430550


in Bibtex Style

@conference{closer15,
author={Holger Gantikow and Sebastian Klingberg and Christoph Reich},
title={Container-based Virtualization for HPC},
booktitle={Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2015},
pages={543-550},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005485605430550},
isbn={978-989-758-104-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Container-based Virtualization for HPC
SN - 978-989-758-104-5
AU - Gantikow H.
AU - Klingberg S.
AU - Reich C.
PY - 2015
SP - 543
EP - 550
DO - 10.5220/0005485605430550