![](bgb.png)
which is the amount observed as the safest maximum
for management by an intermediate configuration in-
stance. In addition, it was evidenced that the Orama
framework in its distributed architecture is capable of
visualizing the same behavior as the FaaS provider
when subjected to a concentrated and distributed ap-
proach of bursts of requests, as well as eventual dif-
ferences.
In the experiments, it was also evident that the
AFC FaaS delivers greater consistency in terms of av-
erage execution time and occurrence of failures, fol-
lowed by AWS Lambda and CGF, which registered
close results, and finally the AZF results with high
average execution times and cold start.
In future work, other providers will be integrated
into the Orama framework, such as IBM and Ora-
cle, in order to expand the coverage of the analy-
ses presented in this work. Furthermore, even higher
levels of concurrency can be evaluated by design-
ing experiments that include a larger number of dis-
tributed workers. Furthermore, evolutions in the
Orama framework will allow the analysis of bench-
mark results using percentiles.
REFERENCES
Back, T. and Andrikopoulos, V. (2018). Using a mi-
crobenchmark to compare function as a service solu-
tions. In ECSOCC, pages 146–160. Springer.
Barcelona-Pons, D. and Garc
´
ıa-L
´
opez, P. (2021). Bench-
marking parallelism in faas platforms. Future Gener-
ation Computer Systems, 124:268–284.
Carvalho, L. and Araujo, A. (2022). Orama: A bench-
mark framework for function-as-a-service. In Pro-
ceedings of the 12th CLOSER, pages 313–322. IN-
STICC, SciTePress.
Carvalho, L. R. and Araujo, A. P. F. (2020). Performance
comparison of terraform and cloudify as multicloud
orchestrators. In 2020 20th IEEE/ACM CCGRID,
pages 380–389.
Copik, M., Kwasniewski, G., Besta, M., Podstawski, M.,
and Hoefler, T. (2021). Sebs: A serverless benchmark
suite for function-as-a-service computing.
DUAN, Y., Fu, G., Zhou, N., Sun, X., Narendra, N. C., and
Hu, B. (2015). Everything as a Service (XaaS) on
the Cloud: Origins, Current and Future Trends. vol-
ume 00, pages 621–628.
Erinle, B. (2013). Performance testing with JMeter 2.9.
Packt Publishing Ltd. ISBN: 9781782165842.
Gartner (2022). Gartner forecasts worldwide pub-
lic cloud end-user spending to reach nearly $600
billion in 2023. [online; 01-Jan-2023; url:
https://tinyurl.com/3z72zebw].
Grambow, M., Pfandzelter, T., Burchard, L., Schubert, C.,
Zhao, M., and Bermbach, D. (2021). Befaas: An
application-centric benchmarking framework for faas
platforms.
Gupta, P. R., Taneja, S., and Datt, A. (2014). Using heat
and ceilometer for providing autoscaling in openstack.
JIMS8I-International Journal of Information Commu-
nication and Computing Technology, 2(2):84–89.
HashiCorp (2021). Terraform: Write, plan, apply. [online;
11-Aug-2021; url: https://www.terraform.io ].
Ibrahim, M. H., Sayagh, M., and Hassan, A. E. (2021).
A study of how docker compose is used to compose
multi-component systems. Empirical Software Engi-
neering, 26(6):128.
Jain, R. (1991). The art of computer systems: Tech-
niques for experimental design, measurement, simu-
lation, and modeling. ISBN:13.978-0471503361.
Kuhlenkamp, J., Werner, S., Borges, M. C., El Tal, K., and
Tai, S. (2019). An evaluation of faas platforms as a
foundation for serverless big data processing. In Pro-
ceedings of the 12th IEEE/ACM, UCC’19, page 1–9,
NY, USA. ACM.
Maissen, P., Felber, P., Kropf, P., and Schiavoni, V. (2020).
Faasdom. Proceedings of the 14th ACM International
Conference on Distributed and Event-based Systems.
MELL, P. and Grance, T. (2011). The NIST definition of
cloud computing. National Institute of Standards and
Tecnology.
Motta., M. A. D. C., Reboucas De Carvalho., L., Rosa., M.
J. F., and Favacho De Araujo., A. P. (2022). Compar-
ison of faas platform performance in private clouds.
In Proceedings of the 12th CLOSER,, pages 109–120.
INSTICC, SciTePress.
Sax, M. J. (2018). Apache Kafka, pages 1–8. Springer Inter-
national Publishing, Cham. ISBN: 978-3-319-63962-
8.
Schleier-Smith, J., Sreekanti, V., Khandelwal, A., Carreira,
J., Yadwadkar, N. J., Popa, R. A., Gonzalez, J. E., Sto-
ica, I., and Patterson, D. A. (2021). What serverless
computing is and should become: The next phase of
cloud computing. ACM, 64(5):76–84.
Somu, N., Daw, N., Bellur, U., and Kulkarni, P. (2020).
Panopticon: A comprehensive benchmarking tool for
serverless applications. In 2020 COMSNETS, pages
144–151.
Wen, J., Liu, Y., Chen, Z., Chen, J., and Ma, Y. (2020).
Characterizing commodity serverless computing plat-
forms. DOI:10.48550/ARXIV.2012.00992.
Wittig, M. and Wittig, A. (2018). Amazon web services in
action. Simon and Schuster. ISBN: 978-1617295119.
FaaS Benchmarking over Orama Framework’s Distributed Architecture
77