Reducing IoT Big Data for Efficient Storage and Processing

Eleftheria Katsarou, Stathes Hadjiefthymiades

2023

Abstract

We focus on the very important problem of managing IoT data. We consider the data gathering process that yields big data intended for CDN/cloud storage. We aim to reduce big data into small data to efficiently exploit available storage without compromising their usability and interpretation. This reduction process is to be performed at the edge of the infrastructure (IoT edge devices, CDN edge servers) in a computationally acceptable way. Therefore, we employ reservoir sampling, a method that stochastically samples data and derives synopses that are finally pushed and maintained in the available storage capability. We implemented the discussed architecture using reverse proxy technologies and in particular the Varnish open source server. We provide details of our implementation and discuss critical parameters like the frequency of synopsis generation and CDN/cloud storage.

Paper Citation

in Harvard Style

Katsarou E. and Hadjiefthymiades S. (2023). Reducing IoT Big Data for Efficient Storage and Processing. In Proceedings of the 8th International Conference on Internet of Things, Big Data and Security - Volume 1: IoTBDS, ISBN 978-989-758-643-9, SciTePress, pages 226-230. DOI: 10.5220/0011983900003482

in Bibtex Style

@conference{iotbds23,
author={Eleftheria Katsarou and Stathes Hadjiefthymiades},
title={Reducing IoT Big Data for Efficient Storage and Processing},
booktitle={Proceedings of the 8th International Conference on Internet of Things, Big Data and Security - Volume 1: IoTBDS,},
year={2023},
pages={226-230},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011983900003482},
isbn={978-989-758-643-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 8th International Conference on Internet of Things, Big Data and Security - Volume 1: IoTBDS,
TI - Reducing IoT Big Data for Efficient Storage and Processing
SN - 978-989-758-643-9
AU - Katsarou E.
AU - Hadjiefthymiades S.
PY - 2023
SP - 226
EP - 230
DO - 10.5220/0011983900003482
PB - SciTePress