ETL Standard Processes Modelling - A Novel BPMN Approach

Bruno Oliveira, Orlando Belo

2013

Abstract

ETL systems modelling have been a topic quite explored by researchers in Data Warehousing. However, we believe that there isn’t yet a convinced and simply approach that provides the necessary bridges to validate conceptual and logical models and testing them before its real implementation. In this work we explore the use of BPMN for ETL conceptual modelling, presenting an extension to the BPMN 2.0 meta-model and notation to support modelling and visualization of ETL activities. We intend to provide a set of BPMN meta-models especially designed to map standard ETL processes, providing the necessary bridges to translate conceptual models into their correspondent implementation testing correctness and effectiveness of its execution. For this particular work, we specially designed a standard ETL process – Change Data Capture based on log files – to demonstrate the viability and effectiveness of the approach presented.

References

  1. Akkaoui, Z. El et al., 2011. A model-driven framework for ETL process development. In DOLAP 7811 Proceedings of the ACM 14th international workshop on Data Warehousing and OLAP. pp. 45-52.
  2. Akkaoui, Z. El et al., 2012. BPMN-Based Conceptual Modeling of ETL Processes. Data Warehousing and Knowledge Discovery Lecture Notes in Computer Science, 7448, pp.1-14.
  3. Akkaoui, Z. El & Zimanyi, E., 2009. Defining ETL worfklows using BPMN and BPEL. In DOLAP 7809 Proceedings of the ACM twelfth international workshop on Data warehousing and OLAP. pp. 41- 48.
  4. Golfarelli, M. & Rizzi, S., 2009. Data Warehouse Design: Modern Principles and Methodologies, McGraw-Hill. Available at: http://books.google.pt/books?id=R7 qqNwAACAAJ
  5. Inmon, W. H., 2005. Building the data warehouse, Wiley. Available at: http://books.google.pt/books?id=Ee VQAAAAMAAJ.
  6. Kimball, R & Ross, M., 2002. The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling, Wiley. Available at: http://books.google.pt/ books?id=2OCbq8Azdm8C.
  7. Kimball, Ralph & Caserta, J., 2004. The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data,
  8. Kleppe, A. G., Warmer, J.B. & Bast, W., 2003. MDA Explained: The Model Driven Architecture?: Practice and Promise, Addison-Wesley Professional.
  9. Losavio, F., Chirinos, L. & Pérez, M.A., 2001. Quality Models to Design Software Architectures. In TOOLS 7801 Proceedings of the Technology of Object-Oriented Languages and Systems.
  10. Oliveira, B. & Belo, O., 2012. BPMN Patterns for ETL Conceptual Modelling and Validation. In 20th International Symposium on Methodologies for Intelligent Systems.
  11. OMG, 2011. Documents Associated With Business Process Model And Notation (BPMN) Version 2.0. In Documents Associated With Business Process Model And Notation (BPMN) Version 2.0.
  12. Scacchi, W., 2001. Process Models in Software Engineering. In J.J. Marciniak (ed.), Encyclopedia of Software Engineering, 2 nd Edition, John Wiley and Sons, Inc.
  13. Simitsis, A. & Vassiliadis, P., 2003. A Methodology for the Conceptual Modeling of ETL Processes. In The 15th Conference on Advanced Information Systems Engineering (CAiSE 7803). pp. pp. 305-316.
  14. Stroppi, L. J. R., Chiotti, O. & Villarreal, P.D., 2011. Extending BPMN 2.0: Method and Tool Support. Lecture Notes in Business Information Processing, 95(59-73).
  15. Trujillo & Luján-Mora, S., 2003. A UML Based Approach for Modeling ETL Processes in Data Warehouses. Conceptual Modeling - ER 2003 - Lecture Notes in Computer Science, 2813, pp.307-320.
  16. Vassiliadis, P. et al., 2003. A framework for the design of ETL scenarios. In Proceedings of the 15th international conference on Advanced information systems engineering. Berlin, Heidelberg: SpringerVerlag, pp. 520-535. Available at: http://dl.acm.org/ citation.cfm?id=1758398.1758445
  17. Vassiliadis, P., Simitsis, A. & Skiadopoulos, S., 2002a. Conceptual modeling for ETL processes. In DOLAP 7802 Proceedings of the 5th ACM international workshop on Data Warehousing and OLAP. pp. 14-21.
  18. Vassiliadis, P., Simitsis, A. & Skiadopoulos, S., 2002b. On the Logical Modeling of ETL Processes. In In International Conference on Advanced Information Systems Engineering (CAiSE. pp. 782-786.
  19. Weske, M., Aalst, W. M. P. van der & Verbeek, H.M.W., 2004. Advances in business process management. Data & Knowledge Engineering 50, 50(1-8).
  20. Wilkinson, K. et al., 2010. Leveraging Business Process Models for ETL Design. Lecture Notes in Computer Science, 6412/2010(15-30).
Download


Paper Citation


in Harvard Style

Oliveira B. and Belo O. (2013). ETL Standard Processes Modelling - A Novel BPMN Approach . In Proceedings of the 15th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-8565-59-4, pages 120-127. DOI: 10.5220/0004418301200127


in Bibtex Style

@conference{iceis13,
author={Bruno Oliveira and Orlando Belo},
title={ETL Standard Processes Modelling - A Novel BPMN Approach},
booktitle={Proceedings of the 15th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2013},
pages={120-127},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004418301200127},
isbn={978-989-8565-59-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 15th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - ETL Standard Processes Modelling - A Novel BPMN Approach
SN - 978-989-8565-59-4
AU - Oliveira B.
AU - Belo O.
PY - 2013
SP - 120
EP - 127
DO - 10.5220/0004418301200127