Transforming Data Lakes to Data Meshes Using Semantic Data Blueprints

Michalis Pingos, Athos Mina, Andreas Andreou

2024

Abstract

In the continuously evolving and growing landscape of Big Data, a key challenge lies in the transformation of a Data Lake into a Data Mesh structure. Unveiling a transformative approach through semantic data blueprints enables organizations to align with changing business needs swiftly and effortlessly. This paper delves into the intricacies of detecting and shaping Data Domains and Data Products within Data Lakes and proposes a standardized methodology that combines the principles of Data Blueprints with Data Meshes. Essentially, this work introduces an innovative standardization framework dedicated to generating Data Products through a mechanism of semantic enrichment of data residing in Data Lakes. This mechanism not only enables the creation readiness and business alignment of Data Domains, but also facilitates the extraction of actionable insights from software products and processes. The proposed approach is qualitatively assessed using a set of functional attributes and is compared against established data structures within storage architectures yielding very promising results.

Paper Citation

in Harvard Style

Pingos M., Mina A. and Andreou A. (2024). Transforming Data Lakes to Data Meshes Using Semantic Data Blueprints. In Proceedings of the 19th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE; ISBN 978-989-758-696-5, SciTePress, pages 344-352. DOI: 10.5220/0012620200003687

in Bibtex Style

@conference{enase24,
author={Michalis Pingos and Athos Mina and Andreas Andreou},
title={Transforming Data Lakes to Data Meshes Using Semantic Data Blueprints},
booktitle={Proceedings of the 19th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE},
year={2024},
pages={344-352},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012620200003687},
isbn={978-989-758-696-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE
TI - Transforming Data Lakes to Data Meshes Using Semantic Data Blueprints
SN - 978-989-758-696-5
AU - Pingos M.
AU - Mina A.
AU - Andreou A.
PY - 2024
SP - 344
EP - 352
DO - 10.5220/0012620200003687
PB - SciTePress