Optimizing Resource Allocation in Edge-distributed Stream Processing

Aluizio Neto, Thiago Silva, Thais Batista, Frederico Lopes, Flávia Delicato, Paulo Pires

2021

Abstract

Emerging Web applications based on distributed IoT sensor systems and machine intelligence, such as in smart city scenarios, have posed many challenges to network and processing infrastructures. For example, environment monitoring cameras generate massive data streams to event-based applications that require fast processing for immediate actions. Finding a missing person in public spaces is an example of these applications, since his/her location is a piece of perishable information. Recently, the integration of edge computing with machine intelligence has been explored as a promising strategy to interpret such massive data near the sensor and reduce the end-to-end latency of processing events. However, due to the limited capacity and heterogeneity of edge resources, the placement of task processing is not trivial, especially when applications have different quality of service (QoS) requirements. In this paper, we develop an algorithm to solve the optimization problem of allocating a set of nodes with sufficient processing capacity to execute a pipeline of tasks while minimizing the operational cost related to latency and energy and maximizing availability. We compare our algorithm with the resource allocation algorithms (first-fit, best-fit, and worst-fit), achieving a lower cost in scenarios with different nodes’ heterogeneity. We also demonstrate that distributing processing across multiple edge nodes reduces latency and energy consumption and still improves availability compared to processing only in the cloud.

Paper Citation

in Harvard Style

Neto A., Silva T., Batista T., Lopes F., Delicato F. and Pires P. (2021). Optimizing Resource Allocation in Edge-distributed Stream Processing. In Proceedings of the 17th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-989-758-536-4, pages 156-166. DOI: 10.5220/0010714700003058

in Bibtex Style

@conference{webist21,
author={Aluizio Neto and Thiago Silva and Thais Batista and Frederico Lopes and Flávia Delicato and Paulo Pires},
title={Optimizing Resource Allocation in Edge-distributed Stream Processing},
booktitle={Proceedings of the 17th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2021},
pages={156-166},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010714700003058},
isbn={978-989-758-536-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - Optimizing Resource Allocation in Edge-distributed Stream Processing
SN - 978-989-758-536-4
AU - Neto A.
AU - Silva T.
AU - Batista T.
AU - Lopes F.
AU - Delicato F.
AU - Pires P.
PY - 2021
SP - 156
EP - 166
DO - 10.5220/0010714700003058