A Containerized Big Data Streaming Architecture for Edge Cloud Computing on Clustered Single-board Devices

Remo Scolati, Ilenia Fronza, Nabil El Ioini, Areeg Samir, Claus Pahl

2019

Abstract

The constant increase of the amount of data generated by Internet of Things (IoT) devices creates challenges for the supporting cloud infrastructure, which is often used to process and store the data. This work focuses on an alternative approach, based on the edge cloud computing model, i.e., processing and filtering data before transferring it to a backing cloud infrastructure. We describe the implementation of a low-power and low-cost cluster of single board computers (SBC) for this context, applying models and technologies from the Big Data domain with the aim of reducing the amount of data which has to be transferred elsewhere. To implement the system, a cluster of Raspberry Pis was built, relying on Docker to containerize and deploy an Apache Hadoop and Apache Spark cluster, on which a test application is then executed. A monitoring stack based on Prometheus, a popular monitoring and alerting tool in the cloud-native industry, is used to gather system metrics and analyze the performance of the setup. We evaluate the complexity of the system, showing that by means of containerization increased fault tolerance and ease of maintenance can be achieved, which makes the proposed solution suitable for an industrial environment. Furthermore, an analysis of the overall performance, which takes into account the resource usage of the proposed solution with regards to the constraints imposed by the devices, is presented in order to discuss the capabilities and limitations of proposed architecture.

Paper Citation

in Harvard Style

Scolati R., Fronza I., El Ioini N., Samir A. and Pahl C. (2019). A Containerized Big Data Streaming Architecture for Edge Cloud Computing on Clustered Single-board Devices.In Proceedings of the 9th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-365-0, pages 68-80. DOI: 10.5220/0007695000680080

in Bibtex Style

@conference{closer19,
author={Remo Scolati and Ilenia Fronza and Nabil El Ioini and Areeg Samir and Claus Pahl},
title={A Containerized Big Data Streaming Architecture for Edge Cloud Computing on Clustered Single-board Devices},
booktitle={Proceedings of the 9th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2019},
pages={68-80},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007695000680080},
isbn={978-989-758-365-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - A Containerized Big Data Streaming Architecture for Edge Cloud Computing on Clustered Single-board Devices
SN - 978-989-758-365-0
AU - Scolati R.
AU - Fronza I.
AU - El Ioini N.
AU - Samir A.
AU - Pahl C.
PY - 2019
SP - 68
EP - 80
DO - 10.5220/0007695000680080