Performance Evaluation of Cloud-based RDBMS through a Cloud

Scripting Language

Andrea S. Char

, Guilherme F. Hoffmann

, Luiz A. Steffenel

Manuele K. Pinheiro

and Benhur de O. Stein

Universidade Federal de Santa Maria, Santa Maria, Brazil

CReSTIC Laboratory, Universit

e de Reims Champagne-Ardenne, Reims, France

CRI Laboratory, Universit

e Paris 1 Panth

eon-Sorbonne, Paris, France

Keywords:

Relational Database Management Systems, Cloud Computing, Performance Evaluation, Benchmarking.

Abstract:

Cloud computing has brought new opportunities, but also new concerns, for developing enterprise informa-

tion systems. In this work, we investigated the performance of two cloud-based relational database services,

accessing them via scripts which also execute on a cloud platform, using Google Apps Script technology.

Preliminary results show little differences between the services in their trial versions, considering limitations

imposed by the Google platform.

1 INTRODUCTION

Cloud computing has proven to be an advantageous

alternative for both businesses and end users, provid-

ing previously unimaginable services at affordable or

even free subscription plans. Engineers and devel-

opers of enterprise information systems can count on

various types of cloud services, including Infrastruc-

ture as a Service (IaaS), Platform as a Service (PaaS)

and Software as a Service (SaaS) (Mell and Grance,

2011).

In the market landscape of cloud computing,

a recent trend is Database as a Service (DBaaS),

which moves database storage and management to the

cloud (Hacigumus et al., 2002; Mahmood, 2014). Ap-

plications that reside or not in the cloud can beneﬁt

from this model, which inherits advantages of other

cloud services, such as availability, scalability and

ease of management. Another recent trend is cloud

scripting, a programming solution to automate tasks

in the cloud. This little-known alternative allows to

build entire information systems by coupling existing

cloud services.

Google is a representative example of a com-

pany offering cloud services to both the general pub-

lic (email, calendar, storage, etc.) and to develop-

ers. A useful resource for developers is Google Apps

Script (GAS) (Google Inc., 2009; Ferreira, 2014), a

cloud scripting solution to automate access to cloud

services. This programming tool takes advantage of

Google’s Platform as a Service (PaaS) layer, enabling

to build applications that run on Google’s infrastruc-

ture and use its libraries, services, and tools. In this

model, developers do not need to manage or control

any cloud infrastructure: they only deal with devel-

oping applications and performing environment set-

tings (Mell and Grance, 2011). Among its features,

Google Apps Script offers the ability to access rela-

tional cloud databases (Google Inc., 2017a).

These new possibilities of cloud data management

and programming bring, along with the beneﬁts, some

questions to be investigated. For example, cloud ser-

vices often associate performance attributes (requi-

sitions per second, maximum execution time, etc.)

with pricing models (free, pay-as-you-go, etc.) (Li

et al., 2010). Thus, knowing the performance of ser-

vices becomes important because, by ignoring it, a

developer can generate more expenses or even make

a cloud execution unfeasible (Iosup et al., 2011).

In this paper, we investigated the access times

for two different third-party cloud databases using

Google Apps Script. This can provide subsidies for

developers interested in these services. This type of

performance data was not found in the literature, re-

inforcing the motivation for this work.

Indeed, we are focusing in a scenario where the

332

Charão, A., Hoffmann, G., Steffenel, L., Pinheiro, M. and Stein, B.

Performance Evaluation of Cloud-based RDBMS through a Cloud Scripting Language.

DOI: 10.5220/0006350803320337

In Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS 2017) - Volume 1, pages 332-337

ISBN: 978-989-758-247-9

application and the data services do not reside in the

same provider. Several factors may conduct the split-

ting of services among different providers. Most of

times, dependency issues prevent the complete mi-

gration from one provider to the other, obliging com-

panies to integrate services hosted by different cloud

providers.

2 BACKGROUND

2.1 Cloud Databases

As several cloud database providers exist and this pre-

liminary work has no intention to perform a broad

comparison between these services, we decided to

concentrate in two relational database services: the

Amazon Relational Database Service (RDS), a pio-

neer service launched by Amazon in 2009, and Mor-

pheus, a recent database service offered by Morpheus

Data since 2014. Our primary criteria to choose these

services was related to their different launch dates,

which could be an indicator of maturity of the service.

Both services can be accessed through the JDBC

(Java Database Connectivity) service offered by GAS.

It is worth noting that Google also offers database

services, both relational (Cloud SQL) and NoSQL

(Bigtable and Datastore). However, as stated before,

our aim was to evaluate the performance of disjoint

services and using both cloud scripting and cloud

databases from Google would bias the results.

2.1.1 Amazon Web Services - Amazon RDS for

MySQL

Amazon Web Services is a cloud computing pioneer

offering a comprehensive set of global computing,

storage, database, analytics, application, and deploy-

ments services. As part of this suite, Amazon RDS

(Relational Database Service) is a service that facil-

itates the installation, operation and scheduling of

a MySQL, Oracle, Microsoft SQL Server or Post-

greSQL database in the cloud. It provides scalable

capacity and manages database administration tasks.

In its free program, it offers 750 hours of database

micro-instances per month.

2.1.2 Morpheus Cloud Database

Morpheus is a PaaS that started its services in 2015,

offered by the Morpheus Data. Its ﬂagship product

is an online database service that allows users to cre-

ate, deploy and host instances of four different types

of database classes through public, private and hy-

brid clouds. It offers high reliability and availability,

with each instance being deployed for free with a full

replica, and with automated daily backups. The four

database classes are:

• MongoDB for non-structured data and docu-

ments;

• MySQL for traditional RDBMS (Relational

Database Management System);

• Redis for in-memory data management;

• ElasticSearch for distributed search.

2.2 Cloud Scripting Languages

Cloud scripting is an approach to exploit cloud-based

services in a programmatic way. This can be used to

perform large scale distributed computations (Murray

and Hand, 2010; Dzik et al., 2013) or simply to coor-

dinate multiple services from cloud providers.

GAS is a prominent example of cloud scripting so-

lution. It comprises a JavaScript-based programming

language that allows you to build web applications

and integrate Google Apps and third-party services.

These scripts are written to an editor directly from the

browser and run on Google’s servers.

Google Apps Script is a relatively new technol-

ogy, launched in 2009. It is in constant development,

with new features and features being implemented

frequently. Table 1 summarizes the services acces-

sible through GAS to the present date. Some of them

are scheduled to be deprecated soon, as they are been

replaced. Note, in this table, the JDBC service, which

provides access to RDBMS through GAS.

Although GAS brings advantages such as porta-

bility, ﬂexibility and unconcern of the developer in

conﬁguring the environment, it brings with it a time

limitation that each script has to execute, which is cur-

rently set to six minutes (Google Inc., 2017b).

3 EXPERIMENTS

In order to carry out this work, the development was

divided in two stages: preparation of the environ-

ment (databases and connection with GAS) and cod-

ing of the test scripts in GAS with operations on the

databases.

3.1 Environment Setup

To start the work, we created free accounts in both

services (Amazon RDS and Morpheus). The conﬁg-

Performance Evaluation of Cloud-based RDBMS through a Cloud Scripting Language

333

Table 1: Google Services.

Google Apps Services Advanced Google Services Script Services

Calendar Admin SDK Base

Contacts AdSense Cache

Document Analytics Charts

Domain(Turned off) Apps Activity Content

Drive BigQuery HTML

Forms Calendar JDBC

Gmail Classroom Lock

Groups Drive Mail

Language DoubleCLick Campaigns Optimization

Maps Fusion Tables Properties

Sites Gmail Script

Spreadsheet Google+ UI (Turned off)

Google+ Domains URL Fetch

Mirror Utilities

Prediction XML

Shopping Content

Tasks

URL Shortener

Youtube

urations provided in each service for this type of ac-

count are shown below:

• Amazon RDS: db.t2.micro instance with one vir-

tual CPU, one ECU (Engine Control Unit), 615

MB memory, without EBS (Elastic Block Store)

optimization, low performance network.

• Morpheus: Storage space of 5 GB and 1 GB of

RAM.

The connection of the script to the databases is

done through the JDBC Service Apps Script. This

feature offers 10 different IP address ranges that the

platform can use to access the databases. These ad-

dresses serve to conﬁgure access permissions on the

cloud database.

The two databases use slightly different forms

of conﬁguring access permissions. Morpheus works

with an Access Control List (ACL), and Amazon RDS

creates the database within a Virtual Private Cloud

(VPC), and VPC has a security group, which is re-

sponsible for controlling which IP addresses will be

released to connect to the database.

Both databases use the IP address allocation

method called CIDR (Classless Inter-Domain Rout-

ing), which stores an IPv4 or IPv6 network speciﬁ-

cation. The speciﬁcation format used is composed of

the IP address followed by the number of bits of the

netmask.

3.2 Test Scripts

The scripts developed for the tests are based on the

SysBench benchmark

, a simple performance evalu-

ation tool that allows you to quickly get an impression

of the performance of systems. Part of this tool is ded-

icated to evaluating database performance and is used

by authors who have investigated the performance of

cloud databases (Schwartz et al., 2012). The original

SysBench codes are written in the Lua programming

language. For the tests in this work, codes have been

rewritten using Google Apps Script.

All codes have as principle to do basic SQL opera-

tions in a database, searching for a random value and

then performing the operation. The main goal is to

measure the time taken to process a speciﬁc amount

of database operations. The tests are divided into the

following categories of database operations: Inserts,

Updates, Selects and Deletes. The table used in the

tests has ﬁve numeric/alphanumeric ﬁelds (see Table

2). Figure 1 present a code excerpt from the GAS test

scripts. It performs a database connection, creates a

table and executes a set of database insert operations.

Table 2: Table for tests (Sysbench).

id Name Gender Age DonutsEaten

Initially, measurements were taken to identify the

maximum number of operations each database could

https://github.com/akopytov/sysbench

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

334

function d bOpe ra ti on s () {

Lo gge r . lo g (" Start " );

var age ;

var d onu ts ;

var num ;

var co nn = J dbc . g et Co nn ec ti on ( ’ jdb c : m ysql :// \

16 2 .2 52 .1 0 8. 12 7: 1 33 42 / S ys bench ’ , ’ User ’ , ’ * *** * ’ );

con n . c rea te St at em en t (). e xe cut e ( ’ \

CR EAT E TAB LE Benc hm ar k \

( id I NTEGE R , \

Nam e VAR CH AR (50) , \

Ge nde r VA RCH AR (10) , \

Age INTE GER , \

Do nu ts Ea te n INTE GER , PR IM ARY K EY ( id )) ’);

for(i =1 ; i < 100 ; i ++ ){

age = Math . fl oor (( Mat h . r and om () * 100) + 1 );

do nut s = M ath . floor (( Math . r and om () * 1 000 ) + 1);

num = Math . fl oor (( Mat h . r and om () * 3 500 ) + 1);

con n . c rea te St at em en t (). e xe cut e ( ’ \

IN SER T INTO ‘ Ben chm ark ‘ \

(‘id ‘, ‘ Name ‘ , ‘ Gende r ‘ , ‘ Age ‘ , ‘ Do nut sEate n ‘) \

VA LUE S ( ’+ i + ’, " Nam e ’ +i + ’" , " Fe mal e " , ’+ age + ’, ’+ do nut s + ’ ) ’);

}

Lo gge r . lo g (" End " );

}

Figure 1: Code excerpt from the test scripts.

make within the maximum six-minute time period im-

posed by Google Apps Script. From this, in order for

the two databases to successfully execute within the

time limit, three test sets with different amounts of

operations were performed for each type of access, in

order to obtain the response times perceived by GAS.

Times were measured using the Logger.log func-

tion, which records the time it started and ﬁnished

running the script.

4 RESULTS

Table 3 shows the maximum number of operations

that each database performed in the six-minute time

limit imposed by GAS.

It can be seen in Table 3 that Amazon RDS per-

formed a slightly higher number of operations than

Morpheus, in the interval of six minutes. Consider-

ing the limits expressed in Table 3, we set the follow-

ing number of operations for the tests of access times:

100, 1000 and 3500.

For each type of operation, we executed the scripts

10 times for each set of operations in each of the

databases, then we calculated the mean and the stan-

dard deviation for each case. Tables 4, 5, 6, 7 present

the results for each type of operation.

Observing the results obtained, it can be said that

both databases presented similar performance. Ac-

cess times were slightly lower for Amazon RDS in all

cases, but the standard deviation was lower for Mor-

pheus in some cases. In all cases, the coefﬁcient of

variation was below 25%, suggesting homogeneity in

the set of measures.

5 FINAL REMARKS

In this work, we investigated the performance of two

cloud databases, through codes written in Google

Apps Script and executed in the Google cloud. The

codes were adaptations of the SysBench benchmark,

which performs basic SQL operations on databases.

The results indicated similarity of performance

among the cloud databases, although they are in-

stances with non-identical conﬁgurations. Note that

although a cloud service (GAS) is used to perform

the access to databases, response times are feasible

for applications that perform few operations. It is also

worth mentioning that the instances used in this work

are of low processing power, because they are both

trials and free.

In general, it is noted that the greatest limitation

comes from the GAS itself, when establishing that an

Performance Evaluation of Cloud-based RDBMS through a Cloud Scripting Language

335

Table 3: Number of operations for each database in the 6-minutes limit.

Operation Number

Amazon RDS Morpheus

INSERT 4022 3763

SELECT 4271 4142

UPDATE 4022 3763

DELETE 4022 3730

Table 4: Execution times for INSERT operations.

INSERT

# of Operations Time (seconds)

Amazon RDS Morpheus

Mean Standard Deviation Mean Standard Deviation

100 9.65 0.01555634919 10.513 0.008485281374

1000 93.0389 2.5130575 92.9074 0.1704127343

3500 321.911 9.557255255 337.9359 0.9418662325

Table 5: Execution times for SELECT operations.

SELECT

# of Operations Time (seconds)

Amazon RDS Morpheus

Mean Standard Deviation Mean Standard Deviation

100 8.85745 0.2921058113 9.90745 0.02467802666

1000 86.91995 4.345807566 90.8749 0.7863027407

3500 291.8325 14.41012909 317.15945 0.1690692314

Table 6: Execution times for UPDATE operations.

UPDATE

# of Operations Time (seconds)

Amazon RDS Morpheus

Mean Standard Deviation Mean Standard Deviation

100 9.6315 0.4037579721 10.28395 0.1385222184

1000 91,56245 3,283167496 95.35045 3.625407178

3500 325.5305 18.83803176 340.2379 2.699733691

Table 7: Execution times for DELETE operations.

DELETE

# of Operations Time (seconds)

Amazon RDS Morpheus

Mean Standard Deviation Mean Standard Deviation

100 9.9109 0.04808326112 10.42595 0.05932625894

1000 90.68245 0.8153648294 93.3664 5.216326725

3500 306.7209 16.11284222 330.6804 17.72221726

execution can not exceed six minutes. In this sense,

longer tasks can beneﬁt from strategies that divide

processing into multiple runs, as is done in some ap-

plications using GAS (Ferreira, 2014).

REFERENCES

Dzik, J., Palladinos, N., Rontogiannis, K., Tsarpalis, E.,

and Vathis, N. (2013). Mbrace: Cloud computing

with monads. In Proceedings of the Seventh Work-

shop on Programming Languages and Operating Sys-

tems, PLOS ’13, pages 7:1–7:6, New York, NY, USA.

ACM.

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

336

Ferreira, J. (2014). Google Apps Script, 2nd Edition – Web

Application Development Essentials. O’Reilly Media.

Google Inc. (2009). Google Apps Script. Avail-

able: https://developers.google.com/apps-script. Last

accessed: March 2017.

Google Inc. (2017a). JDBC – Apps Script. Avail-

able: https://developers.google.com/apps-

script/guides/jdbc. Last accessed: March 2017.

Google Inc. (2017b). Quotas for Google services.

Available: https://developers.google.com/apps-

script/guides/services/quotas. Last accessed: March

2017.

Hacigumus, H., Iyer, B., and Mehrotra, S. (2002). Providing

database as a service. In Proceedings 18th Interna-

tional Conference on Data Engineering, pages 29–38.

Iosup, A., Yigitbasi, N., and Epema, D. (2011). On the

performance variability of production cloud services.

In 2011 11th IEEE/ACM International Symposium on

Cluster, Cloud and Grid Computing, pages 104–113.

Li, A., Yang, X., Kandula, S., and Zhang, M. (2010). Cloud-

cmp: Comparing public cloud providers. In Proceed-

ings of the 10th ACM SIGCOMM Conference on In-

ternet Measurement, IMC ’10, pages 1–14, New York,

NY, USA. ACM.

Mahmood, Z., editor (2014). Cloud Computing: Chal-

lenges, Limitations and R&D Solutions. Springer.

Mell, P. M. and Grance, T. (2011). The NIST deﬁnition

of cloud computing. Technical report, Gaithersburg,

MD, United States.

Murray, D. G. and Hand, S. (2010). Scripting the

cloud with skywriting. In Proceedings of the 2Nd

USENIX Conference on Hot Topics in Cloud Comput-

ing, HotCloud’10, pages 12–12, Berkeley, CA, USA.

USENIX Association.

Schwartz, B., Zaitsev, P., and Tkachenko, V. (2012). High

Performance MySQL, 3rd Edition. O’Reilly.

Performance Evaluation of Cloud-based RDBMS through a Cloud Scripting Language

337