Tail-Latency Aware and Resource-Efficient Bin Pack Autoscaling for Distributed Event Queues

Mazen Ezzeddine, Mazen Ezzeddine, Françoise Baude, Fabrice Huet

2024

Abstract

Distributed event queues are currently the backbone for many large-scale real-time cloud applications including smart grids, intelligent transportation, and health care monitoring. Applications (event consumers) that process events from distributed event queue are latency-sensitive. They require that a high percentile of events be served in less than a desired latency. Meeting such desired latency must be accomplished at low cost in terms of resources used. In this research, we first express the problem of targeting resource-efficient and latency-aware event consuming from distributed event queues as a bin pack problem. This bin pack depends on the arrival rate of events, the number of events in the queue backlog, and the maximum consumption rate of event consumers. We show that the proposed bin pack solution outperforms a linear autoscaling solution by 3.5% up to 10% in terms of latency SLA. Furthermore, we discuss how dynamic event consumers provisioning in distributed event queues necessitates a blocking synchronization protocol. We show that this blocking synchronization protocol is at conflict with meeting a desired latency for high percentile of events. Hence, we propose an extension to the bin pack autoscaler logic in order to reduce the tail latency caused by the events accumulated during the blocking synchronisation protocol.

Paper Citation

in Harvard Style

Ezzeddine M., Baude F. and Huet F. (2024). Tail-Latency Aware and Resource-Efficient Bin Pack Autoscaling for Distributed Event Queues. In Proceedings of the 14th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER; ISBN 978-989-758-701-6, SciTePress, pages 50-64. DOI: 10.5220/0012630100003711

in Bibtex Style

@conference{closer24,
author={Mazen Ezzeddine and Françoise Baude and Fabrice Huet},
title={Tail-Latency Aware and Resource-Efficient Bin Pack Autoscaling for Distributed Event Queues},
booktitle={Proceedings of the 14th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER},
year={2024},
pages={50-64},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012630100003711},
isbn={978-989-758-701-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER
TI - Tail-Latency Aware and Resource-Efficient Bin Pack Autoscaling for Distributed Event Queues
SN - 978-989-758-701-6
AU - Ezzeddine M.
AU - Baude F.
AU - Huet F.
PY - 2024
SP - 50
EP - 64
DO - 10.5220/0012630100003711
PB - SciTePress