aim is to develop a monitoring service to inform cloud
consumers of delays in the processing of windows of
data, thus highlighting the need to increase processing
capacity by scaling the system vertically (i.e. adding
more virtual machines to the pool). Correspondingly,
the monitoring service would gather information on
whether data is waiting too long to be processed, thus
suggesting the need to scale the system horizontally
(i.e. increase the number of containers running the
framework’s workers). This monitoring service from
a unique data flow perspective is also a contribution
to the field, and will be used to gather performance
metrics to further evaluate the architectural pattern
proposed in this paper.
ACKNOWLEDGEMENTS
This work made use of the Open Science Data Cloud
(OSDC) which is an Open Commons Consortium
(OCC)-sponsored project.
Cloud computing resources were provided by a
Microsoft Azure for Research award.
REFERENCES
Apache Beam (2017) Apache Beam [Online]. Available
from: <https://beam.apache.org/> [Accessed 28
February 2017].
Apache Beam Capability Matrix (n.d.) Apache Beam
Capability Matrix [Online]. Available from:
<https://beam.apache.org/documentation/runners/capa
bility-matrix/> [Accessed 9 August 2017].
Apache Storm - Project Information (n.d.) Project
Information [Online]. Available from:
<http://storm.apache.org/about/multi-language.html>
[Accessed 7 August 2017].
Bernstein, D. (2014) Containers and Cloud: From LXC to
Docker to Kubernetes. IEEE Cloud Computing, 1 (3)
September, pp. 81–84.
Celesti, A., Mulfari, D., Fazio, M., Villari, M. & Puliafito,
A. (2016) Exploring Container Virtualization in IoT
Clouds. In: 2016 IEEE International Conference on
Smart Computing (SMARTCOMP), May 2016. pp. 1–6.
Chen, H. M., Kazman, R., Haziyev, S., Kropov, V. &
Chtchourov, D. (2016) Big Data as a Service: A Neo-
Metropolis Model Approach for Innovation. In: 2016
49th Hawaii International Conference on System
Sciences (HICSS), January 2016. pp. 5458–5467.
Guillén, J., Miranda, J., Murillo, J. M. & Canal, C. (2013)
A UML Profile for Modeling Multicloud Applications.
In: Lau, K.-K., Lamersdorf, W. & Pimentel, E. ed.,
Service-Oriented and Cloud Computing, September 11,
2013. Springer Berlin Heidelberg, pp. 180–187.
Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A.,
Joseph, A. D., Katz, R., Shenker, S. & Stoica, I. (2011)
Mesos: A Platform for Fine-Grained Resource Sharing
in the Data Center. In: Proceedings of the 8th USENIX
Conference on Networked Systems Design and
Implementation, 2011. Berkeley, CA, USA: USENIX
Association, pp. 295–308.
Kogias, D. G., Xevgenis, M. G. & Patrikakis, C. Z. (2016)
Cloud Federation and the Evolution of Cloud
Computing. Computer, 49 (11) November, pp. 96–99.
MapReduce Tutorial (2013) MapReduce Tutorial [Online].
MapReduce Tutorial. Available from:
<https://hadoop.apache.org/docs/r1.2.1/mapred_tutoria
l.html> [Accessed 7 August 2017].
Martino, B. D. (2014) Applications Portability and Services
Interoperability among Multiple Clouds. IEEE Cloud
Computing, 1 (1) May, pp. 74–77.
Miell, I. & Sayers, A. H. (2015) Docker in Practice. Shelter
Island, NY: Manning Publications.
Okrent, M. D. & Vokurka, R. J. (2004) Process Mapping in
Successful ERP Implementations. Industrial
Management & Data Systems, 104 (8) October, pp.
637–643.
Resizing Your Instance - Amazon Elastic Compute Cloud
(2017) [Online]. Available from:
<http://docs.aws.amazon.com/AWSEC2/latest/UserGu
ide/ec2-instance-resize.html> [Accessed 24 July 2017].
Silva, G. C., Rose, L. M. & Calinescu, R. (2013) A
Systematic Review of Cloud Lock-In Solutions. In:
2013 IEEE 5th International Conference on Cloud
Computing Technology and Science, December 2013.
vol. 2. pp. 363–368.
Weave Net (2017) [Online]. Available from:
<https://store.docker.com/plugins/weave-net-plugin>
[Accessed 20 December 2017].