Assessment of Private Cloud Infrastructure Monitoring Tools

A Comparison of Ceilometer and Monasca

Mario A. Gomez-Rodriguez, Victor J. Sosa-Sosa and Jose L. Gonzalez-Compean

CINVESTAV Tamaulipas, Km. 5.5 carretera Cd. Victoria-Soto La Marina, Cd. Victoria, Tamaulipas, Mexico

Keywords:

Cloud Computing, Cloud Monitoring, Resource Management, Ceilometer, Monasca.

Abstract:

Cloud monitoring tools are a popular solution for both cloud users and administrators of private cloud infras-

tructures. These tools provide information that can be useful for effective and efﬁcient resource consumption

management, supporting the decision-making process in scenarios where administrators must react rapidly to

saturation and failure events. However, the estimation of the impact on host systems in a private cloud is not

a trivial issue for administrators, specially when monitoring measurements are required in reduced periods of

times. This paper presents a performance comparison between two free and well supported cloud monitoring

tools called Ceilometer and Monasca, deployed on a private cloud infrastructure. This comparison is mainly

focused on evaluating these tools ability to obtain monitoring information in short time intervals for early

detection of resource constraints. The impact of resource consumption on the performance of host systems

produced by both tools was analyzed and the evaluation revealed that Monasca produced a better performance

than Ceilometer for evaluated scenarios and that, according to the learned lessons in this comparison, Monasca

represents a suitable option for being integrated into an adaptive cloud resource management system.

1 INTRODUCTION

Cloud computing has been widely adoped by organi-

zations and corporations for delivering services (soft-

ware, platform, infraestructure) over the internet be-

cause of the improvement of energy efﬁciency, hard-

ware and software resources utilization, elasticity,

performance isolation, ﬂexibility and on-demand ser-

vice schema (Aceto et al., 2013). In cloud model,

applications are executed in virtual machines (VMs)

that shares the resources of the physical machines

(host systems) (Brinkmann et al., 2013); as a re-

sult, the management of cloud data centers manag-

ing large amount of virtual and physical machines

becomes a complex task for administrators. The in-

crement in the volumne of trafﬁc is another impor-

tant issue in the management of cloud data center.

It includes topics such as the identiﬁcation and res-

olution of conﬂicts related to availability, reliability

and quality of service. In order to face up this chal-

lenge, precise and ﬁne-grained monitoring activities

are necessary (Aceto et al., 2013). The complexity of

the monitoring activities are commonly managed by a

monitoring system with funtional and non-functional

components. The functional components of a mon-

itoring system capture the state of functionality and

consumption of cloud resources (Smit et al., 2013),

whereas non-funcitonals include components to de-

ploy the monitoring system in a distributed, fault-

tolerant and scalable manner for monitoring a large

amount of resources (Aceto et al., 2013; Fatema et al.,

2014). Such tools help system administrators to ob-

tain information such as load generated by users and

the performance of the cloud computing resources

(Fatema et al., 2014).

Cloud monitoring involves the collection of differ-

ent metrics (e.g. CPU utilization, memory utilization,

etc.), usually from both, the virtual platform (VMs,

hypervisor, virtual networks, etc.) and the physical

infrastructure (hosts, network, hard drives, etc.) and

which are subsequently stored on a database system

in order for IT operators to deal with it and make de-

cisions that are transformed into actions.

Monitoring techniques are indispensable in order

to manage large-scale cloud resources, since with-

out an approprite monitoring some issues such as

usage-based billing and elastic scaling are impossible

(Fatema et al., 2014).

Cloud monitoring systems are key for improv-

ing decision-making processes in scenarios where ad-

ministrators must react rapidly to saturation and fail-

ure events. The extraction of monitoring informa-

tion in reduced periods of time, applying a precise

and ﬁne grained conﬁguration is a non-trivial key

Gomez-Rodriguez, M., Sosa-Sosa, V. and Gonzalez-Compean, J.

Assessment of Private Cloud Infrastructure Monitoring Tools - A Comparison of Ceilometer and Monasca.

DOI: 10.5220/0006484503710381

In Proceedings of the 6th International Conference on Data Science, Technology and Applications (DATA 2017), pages 371-381

ISBN: 978-989-758-255-4

371

for early detection of resource constraint that ad-

ministrators should deal with. This paper presents

a performance comparison, carried out in a private

cloud infrastructure, between two free and well sup-

ported cloud monitoring tools called Ceilometer and

Monasca. This comparison is mainly focused on eval-

uating these tools ability to obtain monitoring infor-

mation in short time intervals for early detection of

resource constraints. As main contributions, this com-

parison reveals that Ceilometer is a suitable and easy

to conﬁgure tool to manage the virtual layer in a pri-

vate OpenStack cloud infrastructure, but when ﬁne

grained monitoring is required its Collector process

does not have an efﬁcient use of RAM in the host sys-

tem, degrading its performance meaningfully. Unlike

Ceilometer, Monasca is easier to conﬁgure for man-

aging the physical layer in an OpenStack cloud in-

frastructure, showing that its Persister process has a

better use of RAM in the host system. Moreover, in a

ﬁne grained monitoring scenario, Monasca was able

to provide timely monitoring information, becoming

a suitable option for being integrated into an adaptive

cloud resource management system.

The rest of the paper is organized as follows:

Section 2 shows the research on cloud monitoring,

covering general purpose monitoring tools adapted

to cloud environments and cloud speciﬁc monitor-

ing tools. The characteristics of the cloud monitor-

ing tools evaluated in this paper are presented in Sec-

tion 3 whereas Section 4 shows a qualitative analysis

of Ceilometer and Monasca cloud monitoring tools,

highlighting their main features. Section 5 describes

the methodology used to conduct the performance as-

sessment of the previously described monitoring tools

as well as the analysis of the results. Finally the paper

ends giving the conclusions in Section 6.

2 RELATED WORK

The tools for monitoring of resources have been de-

veloped ad hoc for different computing models (e.g.

servers, clusters, grid and cloud). For instance, Na-

gios (Nagios Enterprises, 2017; Barth, 2008) is an

open-source monitoring tool for distributed systems

and networks which has been adapted to cloud en-

vironments. It uses a plug-in-based model to ex-

tract information about different performance met-

rics (Perez-Espinoza et al., 2015b) from resources of

physical and virtual machines. Nagios follows a cen-

tralized approach that produces a single point of fail-

ure and a bottleneck on the server storing the metrics,

which could affect the reliability and response time in

which the administrators of data centers receive moni-

toring information. Although Nagios can be extended

using DNX in order to be distributed and to sup-

port large deployments of infrastructures (Calero and

Aguado, 2015), DNX suffers from high-availability

problems as "if the master server goes down, every-

thing is down" (Nagios Enterprises, 2015). Other

monitoring systems commonly used in cluster envi-

ronments, where the changes in the infrastructure are

not as dynamic as they are in cloud scenarios (Aceto

et al., 2013) are Ganglia (Massie et al., 2004) and Col-

lectd (Cowie, 2012).

PCMONS (Chaves et al., 2011) was designed

speciﬁcally for monitoring the resources of private

cloud infrastructures. The main drawback of PC-

MONS is its centralized architecture since the moni-

tored data are stored and processed by a unique server,

impacting directly to the scalability and fault toler-

ance of the system and representing a single point of

failure (Perez-Espinoza et al., 2015a). Another pro-

posal is FlexACMS (de Carvalho et al., 2013), a mod-

ular monitoring solution designed for private clouds

based on general purpose monitoring solutions such

as Nagios and MRTG (Oetiker, 2017). This type of

tool supports the creation of monitoring slices, which

are composed of a set of monitoring metrics and as-

sociated conﬁgurations used to monitor cloud slices

on cloud platforms; however, fault tolerance and

scalability issues are not addressed in their proposal

(Perez-Espinoza et al., 2015b). Konig et al. (König

et al., 2012) implemented a P2P monitoring frame-

work to provide reliable and elastic monitoring ser-

vices for gathering information across all cloud lay-

ers. Povedano-Molina et al. (Povedano-Molina et al.,

2013) proposed a distributed cloud monitoring archi-

tecture called DARGOS which provides measures of

the physical and virtual resources using push/pull ap-

proaches. It is composed of two main components,

the Node Monitoring Agent (NMA), for collecting the

resources statistics, and the Node Supervisor Agent

(NSA), as a subscriber to monitoring information be-

ing published by NMA. DARGOS mainly copes with

extensibility, adaptability, and intrusiveness (Aceto

et al., 2013). MonPaaS (Calero and Aguado, 2015)

is another cloud monitoring tool which uses Open-

Stack as cloud stack and the Nagios tool to monitor

the cloud. It can be used by cloud providers and con-

sumers. Also, it was relesead as open-source under

a GPL license. The problem with MonPaaS is that

its last version was implemented in a very old Open-

Stack version (Folsom) and it is not well documented.

Its deployment requires that users has a great under-

standing of both, the source code of the monitoring

tool and the OpenStack architecture and their APIs.

Ceilometer (Foundation, 2017c) is a free and well

KDCloudApps 2017 - Special Session on Knowledge Discovery and Cloud Computing Applications

372

supported cloud monitoring tool native of the Open-

Stack ecosystem, which represents the main option

for administrators to deploy private/public cloud in-

fraestrucutre (Foundation, 2017b). Monasca, which

stands for Monitoring-at-Scale (LP, 2017a), is also in-

cluded in the native OpenStack ecosystem but can be

deployed in stand-alone manner. This tool deploys

agents on the monitored cloud to extract monitoring

data from cloud resources and delivers this informa-

tion to Monasca service by using push operations.

This service has gained popularity among OpenStack

users as it offers a multi-tenant management environ-

ments and alarms for events and metrics, which is not

completely offered by tools previously described.

Ceilometer and Monasca are becoming the preva-

lent monitoring tools for OpenStack users and have

been used as support for big data proposals (Zareian

et al., 2016) and network virtualization (Gardikis

et al., 2016). Although Ceilometer and Monasca

are prevalent tools among the OpenStack users and

are useful as monitoring solution supporting resource

management proposals, there are no available assess-

ment comparison studies between these tools for the

decision-making processes of administrations of pri-

vate clouds. This paper presents useful insights and

learned lessons for decision makers to take perfor-

mance considerations of both tools into account when

choosing the most suitable tool for speciﬁc monitor-

ing requirements.

3 A COMPARATIVE

PERFORMANCE STUDY OF

CLOUD MONITORING TOOLS

In this section, we describe in more detail the char-

acteristics of the cloud monitoring tools evaluated in

this paper. We also discuss the advantages and draw-

backs of both tools to establish a big picture about

monitoring features of both tools, which is described

as qualitative assessment focused on decision makers

and private cloud administrators in Section 4.

3.1 Ceilometer

OpenStack enables the creation of private and pub-

lic IaaS clouds and consists of several key ser-

vices (Foundation, 2016). These services include

the Telemetry service which contains the Ceilometer

component to provide a data collection service across

all OpenStack core components (Foundation, 2017c).

The data collected by Ceilometer can be used to pro-

vide customer billing and resource tracking. Such

data can be sent to different targets, for example,

to MongoDB (NoSQL) or Gnocchi (time series)

database. It provides four main components (Foun-

dation, 2017c):

1. Polling agent: program to poll OpenStack ser-

vices and build meters.

2. Notiﬁcation agent: program to listen to notiﬁca-

tions on message queues, convert them into events

and samples and apply pipeline actions.

3. Collector: program to gather and record (e.g. in a

database) event and metering data created by no-

tiﬁcation and polling agents.

4. API: service to query and view data recorded by

collector.

Ceilometer provides two methods which can be used

together to collect data (Foundation, 2017c):

1. Bus listener agent: takes events generated on the

notiﬁcation bus and transform them into Ceilome-

ter samples. This method is supported by the

ceilometer-notiﬁcation agent which monitors the

messages queues for notiﬁcations.

2. Polling agents: poll some API or other tool to

collect data at a regular interval. The agents

can be conﬁgured to poll the local hypervisor

(compute-agent) or remote (central-agent) APIs

(public REST APIs or host-level SNMP/IPMI

daemons). The last method is less preferred due

to the load it can generate on the API services.

The data collected by agents are manipulated and

published in what they call pipelines. A pipeline is

a set of transformers that modify the data points and

transform them. Ceilometer provides different trans-

formers which can be used to manipulate data (e. g.

combine historical data or use the temporal context)

in the pipeline. Such functionality is carried out by

the notiﬁcation agent.

The processed event and metering data captured

by notiﬁcation and polling agents are gathered by a

program called Collector and, after the data validation

(signature check) it writes the samples to the speciﬁed

target: database (MongoDB or gnocchi), ﬁle or http.

In case the collected data was stored in a database

(e.g. MongoDB), such stored data can be accessed

using a REST API rather than by accessing the under-

lying database directly (until OpenStack Mitaka ver-

sion). If Gnocchi is used, the data must be accessed

using the Gnocchi API.

3.2 Monasca

Monasca is an OpenStack project that provides an

open-source multi-tenant, highly scalable, perfor-

mant, fault-tolerant monitoring-as-a-service solution.

Assessment of Private Cloud Infrastructure Monitoring Tools - A Comparison of Ceilometer and Monasca

373

Table 1: Cloud monitoring level comparison: Ceilometer and Monasca.

Infrastructure

monitoring level

Ceilometer Monasca

Virtual Only conﬁgure the meters in the

pipeline conﬁguration ﬁle of each

compute node that will be monitored

using the Compute-agent.

Only conﬁgure the libvirt.yaml plugin

on each compute node that will be

monitored with the Monasca-agent.

This is a bit tricky plugin since we had

to modify the python source code of

such agent’s plugin to make it work.

The problem was related with the

authentication of the Nova and

Neutron clients. This plugin will allow

us to obtain different per-instance

metrics such as cpu.utilization_perc,

io.read_bytes, net.in_packets_sec,

mem.used_mb, etc.

Physical Conﬁgure an SNMP server on each

Compute node that will be monitored.

Conﬁgure the Central-agent to poll the

remote host-level SNMP daemons

running on compute nodes. Conﬁgure

the hardware meters to be polled via

SNMP in the pipeline conﬁguration

ﬁle of the node that is running the

Central-agent (typically the

controller). This is the less prefered

method since it can impose more load

on the nodes and, if the Central-agent

goes down, the hardware meters of all

compute nodes get lost.

Only conﬁgure the plugins you want to

enable (e.g. cpu.yaml, disk.yaml,

memory.yaml network.yaml) on each

compute node we want to monitor

with the monasca-agent.. Each plugin

can provide different metrics such as

cpu.yaml: cpu.percent, cpu.idle_perc,

etc.

Table 2: Hardware characteristics of cloud hosts.

Host Cores RAM (GB) Disk(s) size Freq. (Ghz)

controller 6 31 465.8G,1.8T 2

compute6 12 62 3.2T 2.5

compute7 6 23 465.8G,931.5G,931.5G 3.07

compute8 16 62 931.5G,931.5G,931.5G,931.5G 2.6

compute9 12 62 930.4G,27.3T 2.2

compute10 24 251 2.7T 2.2

compute11 24 125 2.7T,2.7T 2.2

compute12 24 125 2.7T,2.7T 2.2

Metrics can be published to the Monasca API, stored

and queried. Monasca builds an extensible platform

for advanced monitoring services that can be used by

both operators and tenants to gain operational insight

and visibilty, ensuring availabilty and stability (LP,

2017a).

The members of the Monasca team are primar-

ily composed of companies, organizations and indi-

viduals involved in development and deployment of

OpenStack. Some of the major companies involved

with developing and/or deploying Monasca are (LP,

2017a): Hewlett Packard, Enterprise Time Warner

Cable (TWC), Fujitsu, Cisco, Cray, Rackspace, SAP

NEC.

The Monasca API is the gateway for all interac-

tion with Monasca. In a typical scenario metrics are

collected by the Monasca Agent running on a sys-

tem and sent to the Monasca API. The API then pub-

lishes the metrics to the Kafka queue. From here the

Monasca Persister consumes metrics and writes them

to the Metrics database. The Monasca Threshold En-

gine also consumes the metrics and uses them to eval-

KDCloudApps 2017 - Special Session on Knowledge Discovery and Cloud Computing Applications

374

uate alarms (LP, 2017b). At this point the metrics are

in the system and can be queried using the Monasca

API, either directly or through one of other compo-

nents, such as the Horizon plugin or the Monasca

CLI (LP, 2017b). Also there exists a conﬁguration

database used for storing information such as alarm

deﬁnitions and notiﬁcation methods. This database

can be either MySQL or PostgreSQL (LP, 2017b).

Finally, Monasca is composed of the following

main components (Foundation, 2017a):

• Monitoring Agent (monasca-agent): a Python

based monitoring agent that consists of several

sub-components and supports system metrics,

such as cpu utilization and available memory, Na-

gios plugins, statsd and many built-in checks for

services such as MySQL, RabbitMQ, etc.

• Monitoring API (monasca-api): a RESTful API

for monitoring that is primarily focused on the

following concepts and areas: metrics, statistics,

alarms and notiﬁcation methods. It has both Java

and Python implementations avaialble.

• Persister (monasca-persister): a program which

consumes metrics and alarm state transitions from

the MessageQ and stores them in the Metrics and

Alarms database. It has both Java and Python im-

plementations available.

• Message Queue: a third-party component that pri-

marily receives published metrics from the Mon-

itoring API and alarm state transition messages

from the Threshold Engine that are consumed by

other components, such as the Persister and No-

tiﬁcation Engine. Currently, a Kafka based Mes-

sageQ is supported.

• Metrics and Alarms Database: a third-party com-

ponent that primarily stores metrics and the alarm

state history.

• Conﬁg Database: a third-party component that

stores a lot of the conﬁguration and other infor-

mation in the system. Currently, MySQL is sup-

ported.

• Monitoring Client (python-monascaclient): a

Python command line client and library that

communicates and controls the Monitoring API.

The Monitoring Client also has a Python library,

"monascaclient" similar to the other OpenStack

clients, that can be used to quickly build addi-

tional capabilities. The Monitoring Client library

is used by the Monitoring UI, Ceilometer pub-

lisher, and other components.

• Monitoring UI: A Horizon dashboard for visualiz-

ing the overall health and status of an OpenStack

cloud.

4 A QUALITATIVE

COMPARISON OF

CEILOMETER AND MONASCA

As we described in the previous sections, Ceilometer

and Monasca are two different monitoring tools de-

signed to monitor an OpenStack cloud. We decided

to compare such tools since OpenStack is one of the

most used cloud stack in both literature and open-

source community.

In Table 1 we can see a comparison between the

virtual and physical infrastructure monitoring levels

of both Ceilometer and Monasca. On one hand we

can see that it is easy to conﬁgure the virtual infras-

tructure monitoring on Ceilometer, whilst it has some

drawbacks to monitor the physical infrastructure. To

obtain the hardware metrics of the physical nodes it

has to run a centralized server (central-agent) on one

node to poll the SNMP servers running on compute

nodes. Such approach has the drawback that, if the

central-agent goes down, the hardware meters of all

compute nodes get lost, in addition to imposing an

extra load on the physical nodes. On the other hand,

unlike Ceilometer, in Monasca it is easy to monitor

the physical infrastructure. Only it is necessary to

conﬁgure the plugins to enable the different metrics

it provides. However, problems arise when we had

to enable the virtual infrastructure monitoring via the

libvirt.yaml plugin. Such plugin allows to obtain dif-

ferent per-instance metrics. We had to modify the

python source code of such agent’s plugin to make

it works. The problem was related with the authenti-

cation of the Nova and Neutron clients.

A summary of the most important components of

Ceilometer and Monasca are depicted in the mind

map shown in Figure 1. It also shows some relevant

properties that can be varied to perform different tests,

for example to vary the number of workers or pro-

cesses in charge of collecting metrics in the Collector

or Persistert agent in Ceilometers or Monasca respec-

tively.

5 PERFORMANCE EVALUATION

AND RESULTS

In this section, we describe the methodology used to

conduct the performance assessment of Ceilomenter

and Monasca tools in a private cloud as well as the

analysis of the results of this evaluation.

Assessment of Private Cloud Infrastructure Monitoring Tools - A Comparison of Ceilometer and Monasca

375

Figure 1: Ceilometer and Monasca mind map.

KDCloudApps 2017 - Special Session on Knowledge Discovery and Cloud Computing Applications

376

Figure 2: Cloud testbed.

5.1 Evaluation Methodology

A set of experiments were executed in a real cloud

computing environment and the cloud deployed for

this evaluation included eight nodes. Seven for Com-

pute and one Controller (see the characteristics of

each node in Table 2). Ceilometer and Monasca were

used to monitor the virtual and physical infrastruc-

ture. Figure 2 shows that Compute6 to Compute12

run the polling agents. In the Compute10 node is

also located the Ceilometer Collector and Monasca

Persister processes, whose main tasks are to obtain

(through message queues) the monitoring informa-

tion extracted by polling agents. This information is

stored in the Gnocchi database (Ceilometer conﬁgu-

ration) and the Inﬂux database (Monasca conﬁgura-

tion). The message queues used by Ceilometer Col-

lector and Monasca Persister are Rabbit and Kafka

respectively.

Depending on the conﬁguration of the monitoring

system, some times, the most recent measurements

may have not been processed and stored when high

frequency monitoring was activated. In this scenarios,

measures could not be extracted in a timely manner

with the respective API of the tool, implying that the

current stored data were obsolete, providing an incon-

sistent landscape of the cloud state. In order to ﬁnd

which of the two monitoring tools were able to obtain

the most recent measurements (almost real time), we

measured the delay between the time (UTC) of the

Controller node and the timestamp of the last mea-

surement of one metric (CPU % usage of a virtual

machine) stored in the database of the monitoring sys-

tem. All this using the respective API of each system.

The total number of tests performed with both

monitoring tools were 50 (25 for each monitoring

tool), each one executed during 1 hour, with differ-

ent polling interval and different number of work-

ers/processes of the program in charge of taking the

data from the message queue to send them to the

database. Table 3 shows the different conﬁgurations

executed on the cloud. It is worthy to mention that

in this evaluation the monitoring information that

Ceilometer and Monasca were requiring to polling

agents installed in the private cloud has to be with

CPU, RAM, Hard Disk and Network consumption of

every node in the cloud (including VMs metrics from

the hypervisors). In this evaluation, for every request

of monitoring information, Ceilometer extracted a to-

tal of 62 metrics and Monasca 92 metrics from the

different plugins included in their respective polling

agents in our private cloud scenario.

Table 3: Test conﬁgurations.

Monit.

tool

# Workers Polling

intervals

Conﬁgs.

Ceilometer 1, 2, 4, 8,

10, 30, 60,

90, 120

Monasca 1, 2, 4, 8,

10, 30, 60,

90, 120

Total 50

5.1.1 Metrics

In order to conduct our evaluation, we collected two

basic performance metrics:

Assessment of Private Cloud Infrastructure Monitoring Tools - A Comparison of Ceilometer and Monasca

377

• % RAM: This metric allows us to determine the

RAM consumption impact when Ceilometer Col-

lector and Monasca Persister processes use differ-

ent conﬁgurations and vary the time interval for

extracting monitoring information.

• Delay: This metric represents the delay produced

by Ceilometer and Monasca in the extraction of

monitoring data.

5.2 Analysis of Results

During the execution of the previously described

tests, we analyzed the physical resource consump-

tion of both monitoring systems and we found that

Ceilometer has memory problems with the agent

(Collector) in charge of sending the monitored data

to the database, whilst Monasca didn’t present prob-

lems with its analogue agent (Persister). However,

Monasca presented some limitations when the time

interval conﬁgured in polling agents was less than 10

seconds, its polling agents suddenly crashes. This be-

havior was caused by delays in the execution of plug-

ins used by the polling agents, since some plugins are

still in execution measuring metrics when the agents

tries to poll again according to their interval rate. For

this reason, we use in this evaluation time intervals of

10 or more seconds.

We started our evaluation running Ceilometer as

our monitoring tool. As we can see in Figure 3 (be-

ing the horizontal axis the time and the vertical axis

the RAM %), using a ﬁne-grained polling interval of

10 seconds and 1 Collector worker causes that the

RAM usage increases very fast (Figure 3a), whilst us-

ing longer polling intervals (Figures 3b-d) the usage

of RAM decreases. We can note that using the 10

polling frequency provokes that the RAM usage in-

creases 1 percent every 0.28 hours wich means the

monitoring system is consuming more than 2.5 GB of

RAM per hour, running the host system out of mem-

ory in almost 28 hrs. This situation implied that for

a test scenario as described in Figure 2, it was neces-

sary to start again the monitoring process every day,

limiting the continuous monitoring activity to 1 day

periods. On the other hand, the RAM consumption

of Monasca was negligible (almost 0) when using a

similar number of processes (number of workers in

Ceilometer) in its Persister agent, reason why these

graphs were not included in the paper. The intuition

behind this behavior is that Monasca Persister is free-

ing the memory used after completing every request

for monitoring information and Ceilometer Collector

is keeping this information in memory.

Figure 4 shows the timestamp delay comparison

of the following 10 conﬁgurations of both monitoring

tools: Collector workers or Persister processes: 1;

polling intervals: 10, 30, 60, 90 and 120 seconds.

Polling intervals deﬁne the time interval that will be

used by polling agents to get monitoring informa-

tion from a speciﬁc resource, e.g. CPU, RAM, HD

or Network. The delay was calculated as the differ-

ence between the current timestamp of the controller

node and the timestamp of the last measurement of

one metric (e.g., CPU % usage in a VM) stored in the

database of the monitoring system. We can see in Fig-

ure 4a that Ceilometer’s delay has a linear growth and

the smaller the polling interval (ﬁne-grained monitor-

ing) the greater the timestamp delay, e.g., in this case

the 10s conﬁguration obtained the greater delay along

the time series whilst the 120s conﬁguration obtained

the smaller delay. Monasca on the other hand pre-

sented a stabilized behavior. In Figure 4b it can be

seen that the delay is almost the same along the time

series, generating only an approximate increase of 5

seconds over the polling interval, e.g., the 10s conﬁg-

uration obtained a delay of about 15 seconds along the

time series whilst the 120s conﬁguration obtained a

delay of about 125 seconds, conﬁrming what was pre-

viously said. It is worthy to mention that every time

Controller requests for a speciﬁc metric (e.g., CPU %

usage), Ceilometer or Monasca responses with a time

series of that metric (list of all measures with its corre-

sponding timestamps taken since the monitoring pro-

cess started). This means that the time series is short

at the the initial phase of monitoring activity and is

getting larger with time.

Delay of the rest of executed conﬁgurations of

both monitoring tools is shown in Figure 5. In the

case of Ceilometer (Figures 4a and 5a) we can see that

the worst conﬁgurations were those with the smaller

polling interval (10s), whereas the bests conﬁguratons

were those with the greater polling interval (120s);

and the two better conﬁgurations were those with four

and sixteen workers and 120s polling interval (de-

picted in Figure 5a as 4_w-p_120s and 16_w-p_120s

respectively) with delay of about 500s along the time

series. This means that with the best Ceilometer’s

conﬁguration we obtained an approx. delay of 8 min-

utes during 1 hour. Monasca on the other hand ob-

tained a better performance (see Figure 5b), following

the same behavior of the ﬁrst 5 conﬁgurations previ-

ously analyzed (Figure 4b) in which the delay only

increased 5 seconds approximately on the polling in-

terval along the time series during 1 hour. This shows

that Monasca can provide a more precise landscape of

the cloud since it had a better performance (smaller

delay) in all the executed tests.

Currently the entire time series is retrieved to get

the last measurement of the metric. Since the moni-

KDCloudApps 2017 - Special Session on Knowledge Discovery and Cloud Computing Applications

378

Figure 3: One collector worker in Ceilometer execution. Polling intervals: a) 10s, b) 30s, c) 60s and d) 90s.

Table 4: Experiments summary.

Cloud stack Monitor Feature

measured

Resource

measured

Agent

name

Memory

consumption

Delay’s

growth

OpenStack

Ceilometer Timestamp

delay

RAM

Collector High Linear

Monasca Persister Low Stable

toring time of the experiments only extends to 1 hour,

such a behaviour is not a problem when we get the

time series. However, if we decided to run more ex-

periments with a longer monitoring time it would be

necessary to get only a subset of the last measure-

ments. Taking into account the previous observa-

tion, a more extended set of experiments increasing

the running time of the monitoring tools can be per-

formed to conﬁrm our ﬁndings.

Finally, Table 4 shows a summary of the previ-

ously described experiments and results. As we can

see, an OpenStack cloud was constructed to run the

set of experiments, measuring the RAM consump-

tion of the collectors programs of the monitoring tools

(Ceilometer Collector and Monasca Persister) and the

timestamp delay of the last monitored measurement;

the former allows us to detect possible bottlenecks

during the data collection stage and the latter to iden-

tify which monitor enables a timely resource monitor-

ing.

6 CONCLUSIONS

This paper provides a relevant information about

results obtained from a qualitative and quantitative

evaluation of two monitoring tools (Ceilometer and

Monasca), which are popular among the administra-

tors of cloud deployed by using OpenStack. The qual-

itative comparison of both tools shows a big picture

of the main functionality characteristics, whereas the

quantitative evaluation provides useful insights about

how the collector agents impact resource consump-

tion in host systems in a private OpenStack cloud en-

Assessment of Private Cloud Infrastructure Monitoring Tools - A Comparison of Ceilometer and Monasca

379

Figure 4: Delay in extraction of monitoring data. Collector workers/persister processes: 1. Polling intervals: 10s, 30s, 60s,

90s, 120s. a) Ceilometer and b) Monasca.

Figure 5: Delay in extraction of monitoring data. Collector workers/persister processes: 2, 4, 8, 16; polling intervals: 10s,

30s, 60s, 90s, 120s. a) Ceilometer and b) Monasca.

vironment when high frequency sensing of node re-

sources is required. As expected, the consumption of

resources (especially memory) in the host server/node

meaningfully increases when sensing is carried out in

short periods of time. The obtained results of this

evaluation make Monasca as an easier to use and suit-

able tool for monitoring private cloud infrastructure

than Ceilometer when high frequency sensing is re-

quired. Ceilometer exhibits memory problems caus-

ing delays when obtaining monitoring data; as a re-

sult, the metrics stored on Ceilometer database were

out-of-date. This is a relevant ﬁnding for decision

makers to take into account when choosing a moni-

toring tool for a private cloud infrastructure. This is

also relevant for an ongoing work in which an adap-

tive cloud resource management system is being de-

veloped and that will include Monasca as data mon-

itoring provider based on learnt lessons in this study.

The adaptive system will apply data analysis and pre-

diction algorithms that allow the cloud manager plat-

form to assign and reallocate resources in a dynami-

cally way, without user intervention.

ACKNOWLEDGEMENTS

This work was partially supported by the sectoral fund

of research, technological development and innova-

tion in space activities of the Mexican National Coun-

cil of Science and Technology (CONACYT) and the

Mexican Space Agency (AEM), project No.262891.

REFERENCES

Aceto, G., Botta, A., De Donato, W., and Pescapè, A.

(2013). Cloud monitoring: A survey. Computer Net-

works, 57(9):2093–2115.

Barth, W. (2008). Nagios: System and network monitoring.

No Starch Press.

Brinkmann, A., Fiehe, C., Litvina, A., LÃijck, I., Nagel,

L., Narayanan, K., Ostermair, F., and Thronicke,

KDCloudApps 2017 - Special Session on Knowledge Discovery and Cloud Computing Applications

380

W. (2013). Scalable monitoring system for clouds.

In 2013 IEEE/ACM 6th International Conference on

Utility and Cloud Computing, pages 351–356.

Calero, J. M. A. and Aguado, J. G. (2015). Monpaas:

An adaptive monitoring platformas a service for cloud

computing infrastructures and services. IEEE Trans-

actions on Services Computing, 8(1):65–78.

Chaves, S. A. D., Uriarte, R. B., and Westphall, C. B.

(2011). Toward an architecture for monitoring

private clouds. IEEE Communications Magazine,

49(12):130–137.

Cowie, B. (2012). Building a better network monitoring

system. Bachelor Report Degree, Computing and

Mathematical Sciences, University of Waikato, Hamil-

ton, New Zealand.

de Carvalho, M. B., Esteves, R. P., da Cunha Rodrigues,

G., Granville, L. Z., and Tarouco, L. M. R. (2013). A

cloud monitoring framework for self-conﬁgured mon-

itoring slices based on multiple tools. In Proceedings

of the 9th International Conference on Network and

Service Management (CNSM 2013), pages 180–184.

Fatema, K., Emeakaroha, V. C., Healy, P. D., Morrison, J. P.,

and Lynn, T. (2014). A survey of cloud monitoring

tools: Taxonomy, capabilities and objectives. Journal

of Parallel and Distributed Computing, 74(10):2918 –

2933.

Foundation, O. (2016). Openstack installation

guide for red hat enterprise linux and centos,

http://docs.openstack.org/mitaka/install-guide-rdo/.

Foundation, O. (2017a). Monasca,

https://wiki.openstack.org/wiki/monasca.

Foundation, O. (2017b). Openstack open source cloud com-

puting software, https://www.openstack.org/.

Foundation, O. (2017c). Welcome to the

ceilometer developer documentation!,

http://docs.openstack.org/developer/ceilometer/.

Gardikis, G., Koutras, I., Mavroudis, G., Costicoglou, S.,

Xilouris, G., Sakkas, C., and Kourtis, A. (2016). An

integrating framework for efﬁcient nfv monitoring. In

2016 IEEE NetSoft Conference and Workshops (Net-

Soft), pages 1–5.

König, B., Calero, J. A., and Kirschnick, J. (2012). Elastic

monitoring framework for cloud infrastructures. IET

Communications, 6(10):1306–1315.

LP, H.-P. E. D. (2017a). Monasca: about,

http://monasca.io/root/about/.

LP, H.-P. E. D. (2017b). Monasca: architecture,

http://monasca.io/root/architecture/.

Massie, M. L., Chun, B. N., and Culler, D. E. (2004). The

ganglia distributed monitoring system: design, im-

plementation, and experience. Parallel Computing,

30(7):817–840.

Nagios Enterprises, L. (2015). Nagios

xi - using dnx for load-balancing,

https://assets.nagios.com/downloads/nagiosxi/docs/using-

dnx-for-load-balancing-with-nagios-xi.pdf.

Nagios Enterprises, L. (2017). Nagios,

http://www.nagios.org/.

Oetiker, T. (2017). Mrtg: Multi router trafﬁc grapher.,

http://oss.oetiker.ch/mrtg/.

Perez-Espinoza, J. A., Sosa-Sosa, V. J., and Gonzalez, J. L.

(2015a). Distribution and load balancing strategies

in private cloud monitoring. In 2015 12th Interna-

tional Conference on Electrical Engineering, Com-

puting Science and Automatic Control (CCE), pages

1–6.

Perez-Espinoza, J. A., Sosa-Sosa, V. J., Gonzalez, J. L., and

Tello-Leal, E. (2015b). A distributed architecture for

monitoring private clouds. In 2015 26th International

Workshop on Database and Expert Systems Applica-

tions (DEXA), pages 186–190.

Povedano-Molina, J., Lopez-Vega, J. M., Lopez-Soler,

J. M., Corradi, A., and Foschini, L. (2013). Dargos: A

highly adaptable and scalable monitoring architecture

for multi-tenant clouds. Future Generation Computer

Systems, 29(8):2041 – 2056. Including Special sec-

tions: Advanced Cloud Monitoring Systems & The

fourth {IEEE} International Conference on e-Science

2011 - e-Science Applications and Tools & Cluster,

Grid, and Cloud Computing.

Smit, M., Simmons, B., and Litoiu, M. (2013). Distributed,

application-level monitoring for heterogeneous clouds

using stream processing. Future Generation Com-

puter Systems, 29(8):2103–2114.

Zareian, S., Fokaefs, M., Khazaei, H., Litoiu, M., and

Zhang, X. (2016). A big data framework for cloud

monitoring. In Proceedings of the 2Nd Interna-

tional Workshop on BIG Data Software Engineering,

BIGDSE ’16, pages 58–64, New York, NY, USA.

ACM.

Assessment of Private Cloud Infrastructure Monitoring Tools - A Comparison of Ceilometer and Monasca

381