Managing and Unifying Heterogeneous Resources in Cloud Environments

Dapeng Dong, Paul Stack, Huanhuan Xiong, John P. Morrison

2017

Abstract

A mechanism for accessing heterogeneous resources through the integration of various cloud management platforms is presented. In this scheme, hardware resources are offered using virtualization, containerization and as bare metal. Traditional management frameworks for managing these offerings are employed and invoked using a novel resource coordinator. This coordinator also provides an interface for cloud consumers to deploy applications on the underlying heterogeneous resources. The realization of this scheme in the context of the CloudLightning project is presented and a demonstrative use case is given to illustrate the applicability of the proposed solution.

References

  1. Benthin, C., Wald, I., Woop, S., Ernst, M., and Mark, W. R. (2012). Combining single and packet-ray tracing for arbitrary ray distributions on the intel mic architecture. IEEE Transactions on Visualization and Computer Graphics, 18(9):1438-1448.
  2. Boutin, E., Ekanayake, J., Lin, W., Shi, B., Zhou, J., Qian, Z., Wu, M., and Zhou, L. (2014). Apollo: Scalable and coordinated scheduling for cloud-scale computing. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, OSDI'14, pages 285-300, Berkeley, CA, USA. USENIX Association.
  3. Burns, B., Grant, B., Oppenheimer, D., Brewer, E., and Wilkes, J. (2016). Borg, omega, and kubernetes. Commun. ACM, 59(5):50-57.
  4. Docker Swarm (2016). https://github.com/docker/ swarm. [Accessed on 15-June-2016].
  5. Embree, I. (2016). https://embree.github.io. [Accessed on 05-December-2016].
  6. Flannel (2016). https://github.com/coreos/flannel# flannel. [Accessed on 16-June-2016].
  7. Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A. D., Katz, R., Shenker, S., and Stoica, I. (2011). Mesos: A platform for fine-grained resource sharing in the data center. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, NSDI'11, pages 295-308, Berkeley, CA, USA. USENIX Association.
  8. Ironic, O. (2016). http://docs.openstack.org/ developer/ironic/deploy/user-guide.html. [Accessed on 14-June-2016].
  9. Krishnan, S. P. T., Krishnan, S. P. T., Veeravalli, B., Krishna, V. H., and Sheng, W. C. (2014). Performance characterisation and evaluation of wrf model on cloud and hpc architectures. In High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS), 2014 IEEE Intl Conf on, pages 1280-1287.
  10. Kubernetes (2016). http://kubernetes.io/. [Accessed on 14-June-2016].
  11. (2016). http://docs.openstack.org/ developer/kuryr/. [Accessed on 14-June-2016].
  12. Libnetwork (2016). https://github.com/docker/ libnetwork. [Accessed on 16-June-2016].
  13. Magnum, O. (2016). https://github.com/openstack/ magnum. [Accessed on 13-June-2016].
  14. Nova, O. (2016). http://docs.openstack.org/ developer/nova/. [Accessed on 14-June-2016].
  15. OpenStack Neutron. https://github.com/openstack/ neutron. [Accessed on 14-June-2016].
  16. Schwarzkopf, M., Konwinski, A., Abd-El-Malek, M., and Wilkes, J. (2013). Omega: Flexible, scalable schedulers for large compute clusters. In Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 7813, pages 351-364, New York, NY, USA. ACM.
  17. Serrano, E., Bermejo, G., Blas, J. G., and Carretero, J. (2014). Evaluation of the feasibility of making largescale x-ray tomography reconstructions on clouds. In Cluster, Cloud and Grid Computing (CCGrid), 2014 14th IEEE/ACM International Symposium on, pages 748-754.
  18. Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., and Wilkes, J. (2015). Large-scale cluster management at google with borg. In Proceedings of the Tenth European Conference on Computer Systems, EuroSys 7815, pages 18:1-18:17, New York, NY, USA. ACM.
  19. Wald, I. (2012). Fast construction of sah bvhs on the intel many integrated core (mic) architecture. IEEE Transactions on Visualization and Computer Graphics, 18(1):47-57.
  20. WeaveNet, W. (2016). https://www.weave.works/ docs/net/latest/introducing-weave/. [Accessed on 16-June-2016].
  21. Zaspel, P. and Griebel, M. (2011). Massively parallel fluid simulations on amazon's hpc cloud. In Network Cloud Computing and Applications (NCCA), 2011 First International Symposium on, pages 73-78.
  22. Zhang, Z., Li, C., Tao, Y., Yang, R., Tang, H., and Xu, J. (2014). Fuxi: A fault-tolerant resource management and job scheduling system at internet scale. Proc. VLDB Endow., 7(13):1393-1404.
Download


Paper Citation


in Harvard Style

Dong D., Stack P., Xiong H. and P. Morrison J. (2017). Managing and Unifying Heterogeneous Resources in Cloud Environments . In Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-243-1, pages 143-150. DOI: 10.5220/0006300901430150


in Bibtex Style

@conference{closer17,
author={Dapeng Dong and Paul Stack and Huanhuan Xiong and John P. Morrison},
title={Managing and Unifying Heterogeneous Resources in Cloud Environments},
booktitle={Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2017},
pages={143-150},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006300901430150},
isbn={978-989-758-243-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Managing and Unifying Heterogeneous Resources in Cloud Environments
SN - 978-989-758-243-1
AU - Dong D.
AU - Stack P.
AU - Xiong H.
AU - P. Morrison J.
PY - 2017
SP - 143
EP - 150
DO - 10.5220/0006300901430150