Towards Distributed Model Analytics with Apache Spark

Önder Babur, Loek Cleophas, Mark van den Brand

2018

Abstract

The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.

Paper Citation

in Harvard Style

Babur Ö., Cleophas L. and van den Brand M. (2018). Towards Distributed Model Analytics with Apache Spark.In Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development - Volume 1: MOMA3N, ISBN 978-989-758-283-7, pages 767-772. DOI: 10.5220/0006735407670772

in Bibtex Style

@conference{moma3n18,
author={Önder Babur and Loek Cleophas and Mark van den Brand},
title={Towards Distributed Model Analytics with Apache Spark},
booktitle={Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development - Volume 1: MOMA3N,},
year={2018},
pages={767-772},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006735407670772},
isbn={978-989-758-283-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development - Volume 1: MOMA3N,
TI - Towards Distributed Model Analytics with Apache Spark
SN - 978-989-758-283-7
AU - Babur Ö.
AU - Cleophas L.
AU - van den Brand M.
PY - 2018
SP - 767
EP - 772
DO - 10.5220/0006735407670772