CLOUDLIGHTNING: A Framework for a Self-organising and Self-managing Heterogeneous Cloud

Theo Lynn, Huanhuan Xiong, Dapeng Dong, Bilal Momani, George Gravvanis, Christos Filelis-Papadopoulos, Anne Elster, Malik Muhammad Zaki Murtaza Khan, Dimitrios Tzovaras, Konstantinos Giannoutakis, Dana Petcu, Marian Neagul, Ioan Dragan, Perumal Kuppudayar, Suryanarayanan Natarajan, Michael McGrath, Georgi Gaydadjiev, Tobias Becker, Anna Gourinovitch, David Kenny, John Morrison

2016

Abstract

As clouds increase in size and as machines of different types are added to the infrastructure in order to maximize performance and power efficiency, heterogeneous clouds are being created. However, exploiting different architectures poses significant challenges. To efficiently access heterogeneous resources and, at the same time, to exploit these resources to reduce application development effort, to make optimisations easier and to simplify service deployment, requires a re-evaluation of our approach to service delivery. We propose a novel cloud management and delivery architecture based on the principles of self-organisation and self-management that shifts the deployment and optimisation effort from the consumer to the software stack running on the cloud infrastructure. Our goal is to address inefficient use of resources and consequently to deliver savings to the cloud provider and consumer in terms of reduced power consumption and improved service delivery, with hyperscale systems particularly in mind. The framework is general but also endeavours to enable cloud services for high performance computing. Infrastructure-as-a-Service provision is the primary use case, however, we posit that genomics, oil and gas exploration, and ray tracing are three downstream use cases that will benefit from the proposed architecture.

References

  1. Alzamil, I., Djemame, K., Armstrong, D., and Kavanagh, R. 2015. Energy-Aware Profiling for Cloud Computing Environments. Electronic Notes in Theoretical Computer Science, 318, 91-108.
  2. Awada, U., Li, K., and Shen, Y. 2014. Energy Consumption in Cloud Computing Data Centers. International Journal of Cloud Computing and Services Science (IJ-CLOSER), 3(3), 145-162.
  3. Barroso, L. A., and Hölzle, U. 2007. The case for energyproportional computing. Computer, (12), 33-37.
  4. Beloglazov, A., Abawajy, J., and Buyya, R. 2012. Energyaware resource allocation heuristics for efficient management of data centers for cloud computing. Future generation computer systems, 28(5), 755-768.
  5. Brandic, I. 2009. “Towards self-manageable cloud services” in 33rd Annual IEEE Conference on Computer Software and Applications Conference 2009 (COMSAC 7809), vol. 2, IEEE, pp. 128-133.
  6. Crago, S. P., & Walters, J. P. 2015. Heterogeneous Cloud Computing: The Way Forward. Computer, 48(1), 59- 61.
  7. Esmaeilzadeh, H., Blem, E., Amant, R. S., Sankaralingam, K., & Burger, D. 2011. Dark silicon and the end of multicore scaling. In Computer Architecture (ISCA), 2011 38th Annual International Symposium on (pp. 365-376). IEEE.
  8. Gell-Mann, M. 1988. Simplicity and complexity in the description of nature. Engineering and Science, 57(3), 2-9.
  9. Herrmann, K., Mühl, G., and Geihs, K. 2005. Self management: the solution to complexity or just another problem? Distributed Systems Online, IEEE, 6(1).
  10. Heylighen, F. 2001. The science of self-organisation and adaptivity. The encyclopedia of life support systems, 5(3), 253-280.
  11. Heylighen, F., & Gershenson, C. 2003. The meaning of self-organisation in computing. IEEE Intelligent Systems, 18(4).
  12. Intersect360 Research, 2015. Top Six Predictions for HPC in 2015. Special Report. February 2015. California, USA.
  13. IDC, 2014a. Worldwide Broader HPC 2014-2018 Forecast: Servers, Storage, Software, Middleware, and Services. IDC. Massachusetts, USA.
  14. IDC, 2014b. Market Analysis Perspective: Worldwide HPC, 2014 - Directions, Trends, and Customer Requirements. IDC. Massachusetts, USA.
  15. Kephart, J., Kephart, J., Chess, D., Boutilier, C., Das, R., Kephart, J. O., and Walsh, W. E. 2007. An architectural blueprint for autonomic computing. IEEE internet computing, 18(21).
  16. Kliazovich, D., Bouvry, P., & Khan, S. U. 2013. DENS: data center energy-efficient network-aware scheduling. Cluster computing, 16(1), 65-75.
  17. Kim. W. 2009. Cloud computing: Today and Tomorrow. Journal of Object Technology. 8(1), 65-72.
  18. Kramer, J., and Magee, J. 2007. Self-managed systems: an architectural challenge. In Future of Software Engineering, 2007. FOSE'07 (pp. 259-268). IEEE.
  19. Lee, Y. C., & Zomaya, A. Y. 2012. Energy efficient utilization of resources in cloud computing systems. The Journal of Supercomputing, 60(2), 268-280.
  20. Marinescu, D. C., Paya, A., Morrison, J. P., and Healy, P. 2013. An auction-driven self-organising cloud delivery model. arXiv preprint arXiv:1312.2998.
  21. Parashar, M., and Hariri, S. 2005. Autonomic computing: An overview. In Unconventional Programming Paradigms (pp. 257-269). Springer Berlin Heidelberg.
  22. Puviani, M. and Frei. R. 2013. Self-Management for cloud computing, in Science and Information Conference 2013, London, UK.
  23. Schuster, P. 2007. Nonlinear dynamics from physics to biology. Complexity, 12(4), 9-11.
  24. Scogland, T. R., Steffen, C. P., Wilde, T., Parent, F., Coghlan, S., Bates, N., Feng, W.C. & Strohmaier, E. 2014. A power-measurement methodology for largescale, high-performance computing. In Proceedings of the 5th ACM/SPEC international conference on Performance engineering (pp. 149-159). ACM.
  25. Turing, A. M. 1952. The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 237(641), 37-72.
  26. Zhang, Q., Cheng, L., and Boutaba, R. 2010. Cloud computing: state-of-the-art and research challenges. Journal of internet services and applications, 1(1), 7- 18.
Download


Paper Citation


in Harvard Style

Lynn T., Xiong H., Dong D., Momani B., Gravvanis G., Filelis-Papadopoulos C., Elster A., Khan M., Tzovaras D., Giannoutakis K., Petcu D., Neagul M., Dragan I., Kuppudayar P., Natarajan S., McGrath M., Gaydadjiev G., Becker T., Gourinovitch A., Kenny D. and Morrison J. (2016). CLOUDLIGHTNING: A Framework for a Self-organising and Self-managing Heterogeneous Cloud . In Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-182-3, pages 333-338. DOI: 10.5220/0005921503330338


in Bibtex Style

@conference{closer16,
author={Theo Lynn and Huanhuan Xiong and Dapeng Dong and Bilal Momani and George Gravvanis and Christos Filelis-Papadopoulos and Anne Elster and Malik Muhammad Zaki Murtaza Khan and Dimitrios Tzovaras and Konstantinos Giannoutakis and Dana Petcu and Marian Neagul and Ioan Dragan and Perumal Kuppudayar and Suryanarayanan Natarajan and Michael McGrath and Georgi Gaydadjiev and Tobias Becker and Anna Gourinovitch and David Kenny and John Morrison},
title={CLOUDLIGHTNING: A Framework for a Self-organising and Self-managing Heterogeneous Cloud},
booktitle={Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2016},
pages={333-338},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005921503330338},
isbn={978-989-758-182-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - CLOUDLIGHTNING: A Framework for a Self-organising and Self-managing Heterogeneous Cloud
SN - 978-989-758-182-3
AU - Lynn T.
AU - Xiong H.
AU - Dong D.
AU - Momani B.
AU - Gravvanis G.
AU - Filelis-Papadopoulos C.
AU - Elster A.
AU - Khan M.
AU - Tzovaras D.
AU - Giannoutakis K.
AU - Petcu D.
AU - Neagul M.
AU - Dragan I.
AU - Kuppudayar P.
AU - Natarajan S.
AU - McGrath M.
AU - Gaydadjiev G.
AU - Becker T.
AU - Gourinovitch A.
AU - Kenny D.
AU - Morrison J.
PY - 2016
SP - 333
EP - 338
DO - 10.5220/0005921503330338