![](bg3.png)
services in their own container and are divided in
the groups: node-specific services, self-contained ser-
vices and system services. While the services of
the first group, such as the load balancer and the
databases, run on specific compute nodes, services
of the second group are independent from the node
they run on. The load-receiver, container-manager
and machine-manager belong to this group and can
run on every compute node. The third group contains
services which are running in addition to the applica-
tion services on each compute node. These services
are the VM-load-transmitter and the commandline-
executor. We assume, that there is an interface, which
could be used to create and delete compute nodes.
Most cloud providers and private cloud frameworks
such as OpenStack already offer similar services.
The commandline-executor is used to create and
delete containers. This service receives commands
e. g. via a POST-REST call and executes them on
the compute node. Only this service can access the
container engine of the compute node. To ensure that
there is no unauthorized usage of the service, it is only
accessible by the container-manager.
The other service which has to run on every com-
pute node is the VM-load-transmitter. This service
monitors the utilization of the compute node. The
measurements of cpu, ram, bandwidth and used disk
space are sent to the machine-manager. These infor-
mation are used to determine if machines have to be
started or stopped.
The machine-manager stores information about
the environment. In most cases there are many dif-
ferent types of compute nodes which can be ordered
or created. They differ in the amount of processors,
their performance, ram, disk space and disk type. The
available types have to be defined and configured. The
service stores the commands to start or stop/delete
such a compute node. Besides this the service stores
also the maximum number of compute nodes of the
same type that are allowed to run. This mechanism
is intended to restrict the costs. It is also possible
to store cost limits, e. g. the maximum amount to be
spent per month.
Once the machine-manager starts a compute node,
the service stores information about it. These consist
of its IP address, but also the type of compute node
and whether the compute node is allowed to be shut-
down or not. This design feature is important since
the infrastructure requires some static parts, e. g. ma-
chines to run databases or to store certain informa-
tion in general. Besides indicating the application of
a machine, its type is also used to determine the com-
pute node, on which an instance of a service should
be executed. E. g. databases usually require fast disks
to run with good performance, while a machine that
is running compute services needs a cpu with more
performance. The machine-manager stores statistics
about the compute node e. g. cpu utilization, used
ram, bandwidth and used disk space. All information
should be stored with time stamps, as we explain in
detail later.
The container-manager controls the services. As
initial step, every new service has to be registered.
This can be part of the deployment process. For
a registered service at least the following informa-
tion should be stored: name, minimum number of
instances, maximum number of instances, the start
and stop commands, bool value if load balancing is
required and requirements for the node (e. g. the
name of a particular compute note, if specified). For
an efficient management of the instances, also in-
formation about the resource consumption should be
stored. If an instance of a service is executed, the
IP of the machine and the port where the service can
be reached, should be stored in a service registry.
The IP address also corresponds the commandline-
executor. If the average load of all instances of a ser-
vice is above or below a certain percentage in a given
time frame, the container-manger can take action and
scales the service up or down. The container-manager
sends a request to the machine-manager to decide on
which compute node the instance should be started
or stopped. In general the container-manager stores
information about the utilization of containers using
custom metrics and the number of instances of a ser-
vice, which is discussed in detail in Section 3.3.
3.2 Load Balancing
The load balancer and the load-receiver are using
the same database, preferably an in-memory database
such as redis
3
. The latter receives the utilization of the
custom metric from the instance and stores it in the
database. Redis supports sorted sets. This makes it
possible to store the information in the right order for
the load balancer, so that reading from the database
has the least effort. We are using a set for each ser-
vice, resulting in multiple scoreboards.
For load balancing a NAT proxy with feedback
method can be used. With this method the load bal-
ancer can handle requests with different durations
much better than other approaches such as round
robin. But also due to the frequently starting and
stopping of machines and containers other methods
apart from this and the round robin approach wouldn’t
work. The load balancer and the in-memory database
which stores the utilization should run on the same
3
https://redis.io
ICEIS 2017 - 19th International Conference on Enterprise Information Systems
456