loading
Papers

Research.Publish.Connect.

Paper

Authors: Thalita Vergilio and Muthu Ramachandran

Affiliation: School of Computing, Creative Technologies and Engineering, Leeds Beckett University, Leeds and U.K.

ISBN: 978-989-758-320-9

Keyword(s): Big Data, Facebook, Twitter, Netflix, Scalability, Distribution, Fault Tolerance, Processing Guarantees.

Abstract: This research represents a unique contribution to the field of Software Engineering for Big Data in the form of an investigation of the big data architectures of three well-known real-world companies: Facebook, Twitter and Netflix. The purpose of this investigation is to gather significant non-functional requirements for real-world big data systems, with an aim to addressing these requirements in the design of our own unique architecture for big data processing in the cloud: MC-BDP (Multi-Cloud Big Data Processing). MC-BDP represents an evolution of the PaaS-BDP architectural pattern, previously developed by the authors. However, its presentation is not within the scope of this paper. The scope of this comparative study is limited to the examination of academic papers, technical blogs, presentations, source code and documentation officially published by the companies under investigation. Ten non-functional requirements are identified and discussed in the context of these companies’ ar chitectures: batch data, stream data, late and out-of-order data, processing guarantees, integration and extensibility, distribution and scalability, cloud support and elasticity, fault-tolerance, flow control, and flexibility and technology agnosticism. They are followed by the conclusion and considerations for future work. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.227.233.78

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Vergilio, T. and Ramachandran, M. (2018). Non-functional Requirements for Real World Big Data Systems - An Investigation of Big Data Architectures at Facebook, Twitter and Netflix.In Proceedings of the 13th International Conference on Software Technologies - Volume 1: SE-CLOUD, ISBN 978-989-758-320-9, pages 833-840. DOI: 10.5220/0006825408330840

@conference{se-cloud18,
author={Thalita Vergilio. and Muthu Ramachandran.},
title={Non-functional Requirements for Real World Big Data Systems - An Investigation of Big Data Architectures at Facebook, Twitter and Netflix},
booktitle={Proceedings of the 13th International Conference on Software Technologies - Volume 1: SE-CLOUD,},
year={2018},
pages={833-840},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006825408330840},
isbn={978-989-758-320-9},
}

TY - CONF

JO - Proceedings of the 13th International Conference on Software Technologies - Volume 1: SE-CLOUD,
TI - Non-functional Requirements for Real World Big Data Systems - An Investigation of Big Data Architectures at Facebook, Twitter and Netflix
SN - 978-989-758-320-9
AU - Vergilio, T.
AU - Ramachandran, M.
PY - 2018
SP - 833
EP - 840
DO - 10.5220/0006825408330840

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.