Fast Analysis and Prediction in Large Scale Virtual Machines Resource Utilisation

Abdullahi Abubakar, Sakil Barbhuiya, Peter Kilpatrick, Ngo Vien, Dimitrios Nikolopoulos

2020

Abstract

Most Cloud providers running Virtual Machines (VMs) have a constant goal of preventing downtime, increasing performance and power management among others. The most effective way to achieve these goals is to be proactive by predicting the behaviours of the VMs. Analysing VMs is important, as it can help cloud providers gain insights to understand the needs of their customers, predict their demands, and optimise the use of resources. To manage the resources in the cloud efficiently, and to ensure the performance of cloud services, it is crucial to predict the behaviour of VMs accurately. This will also help the cloud provider improve VM placement, scheduling, consolidation, power management, etc. In this paper, we propose a framework for fast analysis and prediction in large scale VM CPU utilisation. We use a novel approach both in terms of the algorithms employed for prediction and in terms of the tools used to run these algorithms with a large dataset to deliver a solid VM CPU utilisation predictor. We processed over two million VMs from Microsoft Azure VM traces and filter out the VMs with complete one month of data which amount to 28,858VMs. The filtered VMs were subsequently used for prediction. Our Statistical analysis reveals that 94% of these VMs are predictable. Furthermore, we investigate the patterns and behaviours of those VMs and realised that most VMs have one or several spikes of which the majority are not seasonal. For all the 28,858VMs analysed and forecasted, we accurately predicted 17,523 (61%) VMs based on their CPU. We use Apache Spark for parallel and distributed processing to achieve fast processing. In terms of fast processing (execution time), on average, each VM is analysed and predicted within three seconds.

Paper Citation

in Harvard Style

Abubakar A., Barbhuiya S., Kilpatrick P., Vien N. and Nikolopoulos D. (2020). Fast Analysis and Prediction in Large Scale Virtual Machines Resource Utilisation.In Proceedings of the 10th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-424-4, pages 115-126. DOI: 10.5220/0009408701150126

in Bibtex Style

@conference{closer20,
author={Abdullahi Abubakar and Sakil Barbhuiya and Peter Kilpatrick and Ngo Vien and Dimitrios Nikolopoulos},
title={Fast Analysis and Prediction in Large Scale Virtual Machines Resource Utilisation},
booktitle={Proceedings of the 10th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2020},
pages={115-126},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009408701150126},
isbn={978-989-758-424-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Fast Analysis and Prediction in Large Scale Virtual Machines Resource Utilisation
SN - 978-989-758-424-4
AU - Abubakar A.
AU - Barbhuiya S.
AU - Kilpatrick P.
AU - Vien N.
AU - Nikolopoulos D.
PY - 2020
SP - 115
EP - 126
DO - 10.5220/0009408701150126