ETL Development using Patterns: A Service-Oriented Approach

Bruno Oliveira, Óscar Oliveira, Vasco Santos, Orlando Belo

2019

Abstract

Extract-Transform-Load (ETL) workflows are commonly developed using frameworks and tools that provide a set of useful pre-configured components to develop complete ETL packages. The pattern concept for ETL development is being studied as a way to simplify and improve the ETL development lifecycle. Patterns are independent composite tasks that can be changed without affecting the ETL structure. The pattern implementation reveals several challenges when used with existing ETL tools, mainly due to the monolith architectural style usually followed. The use of small and loosely-coupled components provided by the microservices architectural style can improve the way ETL patterns are used. In this paper, we present an analysis for the use of microservices for ETL application development using patterns.

Paper Citation

in Harvard Style

Oliveira B., Oliveira Ó., Santos V. and Belo O. (2019). ETL Development using Patterns: A Service-Oriented Approach.In Proceedings of the 21st International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-372-8, pages 216-222. DOI: 10.5220/0007727502160222

in Bibtex Style

@conference{iceis19,
author={Bruno Oliveira and Óscar Oliveira and Vasco Santos and Orlando Belo},
title={ETL Development using Patterns: A Service-Oriented Approach},
booktitle={Proceedings of the 21st International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2019},
pages={216-222},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007727502160222},
isbn={978-989-758-372-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 21st International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - ETL Development using Patterns: A Service-Oriented Approach
SN - 978-989-758-372-8
AU - Oliveira B.
AU - Oliveira Ó.
AU - Santos V.
AU - Belo O.
PY - 2019
SP - 216
EP - 222
DO - 10.5220/0007727502160222