Comparing On-Premise IoT Platforms: Empowering University of

Things Ecosystems with Effective Device Management

Mevludin Blazevic

and Dennis M. Riehle

Institute for IS Research, University of Koblenz, Universit

atsstraße 1, Koblenz, Germany

Keywords:

Internet of Things, Platform, Smart Campus, LoRaWAN.

Abstract:

This paper presents a comparative evaluation of on-premise Internet of Things (IoT) platforms utilizing the

Utility Analysis (UA) method within a university campus environment. The market analysis and evaluation

are conducted systematically using a scoring procedure, guided by expert interviews to identify functional

and non-functional requirements for the IoT device management in the University of Things (UoT) setup.

After a market exploration considering the exclusion criteria, ”no on-premise installation” and ”no license-

free cost models”, ﬁve IoT platforms are assessed against predeﬁned evaluation criteria derived from these

requirements. Among the evaluated platforms, the Long-Range Wide Area Network (LoRaWAN) Network

Server ChirpStack is the most suitable software solution, demonstrating superior utility value. Chirpstack may

be integrated into the university’s own Infrastructure as a Service (IaaS) cloud infrastructure, enhancing data

privacy, security, and application control. The primary objective of this study is to identify an optimal IoT

platform capable of meeting stakeholder requirements for the exploration of available solutions. Recognizing

the critical role of the IoT platform within the Smart Campus/UoT ecosystem, this research contributes to the

enhancement of a Smart Campus infrastructure by facilitating efﬁcient IoT sensor and gateway management.

1 INTRODUCTION

The Internet of Things (IoT) refers to the network of

interconnected devices embedded with sensors, soft-

ware, and other technologies, enabling them to col-

lect and exchange data. This interconnected ecosys-

tem enables seamless communication and automation

across various domains, revolutionizing how we in-

teract with and manage our physical environments.

As the IoTs expands in signiﬁcance and preva-

lence, the importance of IoT platforms as pivotal el-

ements within IoT systems grows as well. These

platform solutions differ in complexity, each offer-

ing distinct functionalities. IoT platforms are com-

prehensive software solutions designed to facilitate

the management, connectivity, and data processing of

interconnected devices within the Internet of Things

ecosystem. Consequently, organizations aiming to

integrate an IoT platform alongside their current In-

formation Technology (IT) setup encounter the chal-

lenge of choosing the most appropriate one for their

particular needs from a plethora of available software

solutions. Evaluating the functional aspects serves

as an important criterion for selection in this process

(cf. (Fahmideh and Zowghi, 2020; Guth et al., 2018;

https://orcid.org/0000-0003-0347-7392

https://orcid.org/0000-0002-5071-2589

Lempert and Pﬂaum, 2019)).

The objective of this paper is to perform a Utility

Analysis (UA) centered on on-premise IoT platforms

currently accessible in the market, aiming to estab-

lish both functional and non-functional criteria to in-

form our selection process. Additionally, the paper

endeavors to provide a guide based on best practices

for selecting an on-premise IoT platform suitable for

managing IoT devices and sensors within a University

of Things (UoT) or Smart Campus infrastructure and

beyond.

A Smart Campus or UoT can be viewed as a sub-

component of a Smart City, while both entities share

a similar framework, with a Smart Campus essen-

tially functioning as a small-scale Smart City ((Zhang

et al., 2022; Fortes et al., 2019; Silva-da N

obrega

et al., 2022; Vasileva et al., 2018)). Within our real-

world implementation of the IoT infrastructure on the

university campus, approximately 100 sensors and 7

Long-Range Wide Area Network (LoRaWAN) gate-

ways are deployed. These sensors conduct various

measurements encompassing air quality (speciﬁcally

targeting ﬁne dust pollution), meteorological parame-

ters, temperature, wind speed, and direction, Ultravi-

olet (UV) radiation, precipitation levels, and hailstone

dimensions within the outdoor campus area. Indoors,

recorded metrics include air pressure, Carbon Diox-

310

Blazevic, M. and Riehle, D.

Comparing On-Premise IoT Platforms: Empowering University of Things Ecosystems with Effective Device Management.

DOI: 10.5220/0012727000003705

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 9th International Conference on Internet of Things, Big Data and Security (IoTBDS 2024), pages 310-320

ISBN: 978-989-758-699-6; ISSN: 2184-4976

ide (CO

) levels, temperature, and humidity. Man-

aging these heterogeneous IoT datasets transmitted

via LoRaWAN requires structured management to be

able to evaluate it appropriately and use it advanta-

geously. Furthermore, an in-house hosted IoT plat-

form is important for registering and overseeing the

IoT installed devices and gateways at the university

campus alongside the collected data. This presents

several beneﬁts for a Smart Campus infrastructure, in-

cluding the potential to seamlessly integrate the plat-

form with other IoT applications like data platforms

and visualization tools. By using a free open-source

solution, it also allows the development of extensions

or adjustments to the software. In addition, it of-

fers more ﬂexibility in installation and user manage-

ment. External hosted systems like The Things Net-

work have some restrictions such as a limited num-

ber of devices or API keys. The latter is relevant for

a Smart Campus application for the integration and

interaction with other IoT applications such as data

platforms. The research objectives for this paper are

described as follows:

RO1: Identiﬁcation of requirements for an IoT plat-

form for a Smart Campus infrastructure, consid-

ering potentially existing IoT infrastructures.

RO2: Execution of a market analysis followed by

evaluating appropriate IoT platforms.

In the subsequent section, an overview of the con-

cept of IoT platforms and related work within the

Smart Campus/UoT domain is provided. Following

this, the Research Methodology for this work is pre-

sented. Subsequently, the presentation of all iden-

tiﬁed stakeholder requirements is given. Utilizing

these requirements, evaluation categories are formu-

lated for subsequent analysis. Five IoT platforms

identiﬁed from the market analysis are evaluated us-

ing UA. Finally, the ﬁndings of this study are summa-

rized, limitations are acknowledged, and an outlook is

provided for further research building upon this work.

2 BACKGROUND

2.1 IoT, Smart Campus and IoT

Platforms

A broad spectrum of deﬁnitions for the IoT is found

across scientiﬁc and non-scientiﬁc literature, as ev-

idenced by works such as those by Baiyere et al.

(2020), Chui et al. (2020), Miorandi et al. (2012),

and Rayes and Salam (2017). These deﬁnitions

commonly emphasize the integration of physical ob-

jects with digital technologies. IoT can be thus de-

scribed as a network of connections between digital

technologies and physical objects, enabling the lat-

ter to acquire computational capabilities and interact

autonomously or with human intervention ((Baiyere

et al., 2020)). Moreover, various IoT domains were

established, including Smart City, Smart Buildings,

Smart Agriculture, Smart Manufacturing, and numer-

ous others ((Garda

sevi

c et al., 2017; Ibarra-Esquer

et al., 2017). IoT sensors offer a variety of new

services accross these domains, for instance, sensor-

based smart parking solutions in smart cities ((Riehle

et al., 2023)) or smart air-quality monioring in smart

buildings ((Arz von Straussenburg et al., 2023)). Ac-

cording to Zhang et al. (2022), a Smart Campus is re-

garded as a component within the broader domain of

a Smart City, encompassing diverse subdomains such

as Smart Education, Smart Assessment, Smart Learn-

ing, Smart Teaching, and others. These subdomains

are supported by different information technologies

((Mart

ınez et al., 2021; Mircea et al., 2021)).

The increasing applications and popularity of IoT

and the resulting investments by companies in these

technologies have created strong competition in the

market, leading to multiple existing IoT standards.

In addition, connectivity requirements of IoT devices

vary depending on the use case and IoT domain,

which means that different protocols are used ((Bha-

jantri and S., 2020)), which leads to a large number of

IoT platforms and connectivity standards such as Lo-

RaWAN on the market ((Barros et al., 2022)). This di-

versity in protocols, coupled with the multitude of use

cases and software companies, leads to a large num-

ber of IoT platforms available in the market ((Barros

et al., 2022)).

An IoT platform serves as a middleware compo-

nent within a holistic IoT system, facilitating the ex-

change of data between edge devices and enterprise

applications in both directions. It represents a multi-

layered technology that facilitates the provisioning

and administration of IoT devices and services, ef-

fectively overseeing the deployment process for IoT

applications in enterprises ((Barros et al., 2022; Fah-

mideh and Zowghi, 2020; Guth et al., 2018)).

IoT platforms from different manufacturers offer

varying core functionalities tailored to diverse appli-

cation contexts, often lacking comprehensive cover-

age of all required functions. These platforms receive

the data collected by sensors from the IoT devices

through protocols such as Wiﬁ, Bluetooth, Zigbee, Z-

wave, and LoRaWAN. Serving as middleware com-

ponents, IoT platforms provide diverse functionali-

ties for data processing and transmission, separating

them from systems purely addressing IoT data stor-

age ((Wolters et al., 2023)). While most IoT platforms

Comparing On-Premise IoT Platforms: Empowering University of Things Ecosystems with Effective Device Management

311

primarily focus on establishing connections to IoT

devices and managing them through device manage-

ment functionalities, data visualization features are

not consistently integrated and are frequently supple-

mented by third-party software (cf. (Guth et al., 2018;

Fahmideh and Zowghi, 2020; Elgedawy and Shoukry,

2022; Toutsop et al., 2021))

2.2 Related Work

A brief exploration of scientiﬁc literature using Sco-

pus and speciﬁcally targeting the article titles, ab-

stracts, and keywords for ”iot platform comparison”

and ”iot platform evaluation” yields merely six ﬁnd-

ings.

In 2019, Panduman et al. (2019) conducted a

comparative examination of ﬁve distinct IoT plat-

forms within the Smart Factory domain. Alongside

the Thingsboard platform, the analysis encompassed

the KAA platform, Microsoft Azure IoT, AWS IoT, and

Thingspeak. The study’s ﬁndings indicate that Azure

IoT and the KAA platform predominantly fulﬁll the

speciﬁed criteria, albeit these criteria are speciﬁcally

tailored to the Smart Factory domain.

In their 2019 work, Berawi et al. (2019) intro-

duced Chief-Screen 1.0, a specialized IoT platform

for the construction domain. The authors gathered re-

quirements and design considerations through group

and expert interviews, integrating them into the plat-

form’s development. However, they did not compare

Chief-Screen 1.0 with other platforms.

Kugler et al. (2021) discuss the creation of a

method for selecting IoT platforms along given cri-

teria such as security, pricing and usability. They also

propose a workﬂow for setting up and deploying these

platforms. However, they do not offer an in-depth

comparison of existing IoT platforms on the market.

Like Kugler et al. (2021), Garc

ıa-Valls and

Palomar-Cos

ın (2022) focus on presenting an evalua-

tion methodology for IoT platforms, using healthcare

as an illustrative domain. They offer a workﬂow for

selecting and assessing these platforms, speciﬁcally

conducting performance evaluations on cloud-based

platforms in their application example. Furthermore,

no market analysis is carried out in their study.

Nasser et al. (2023) also focus on constructing a

model for evaluating IoT platforms. Unlike previous

studies, their model encompasses both technical and

business criteria. Input and design parameters for the

evaluation model were established through a literature

review. However, the study does not include a com-

parative analysis of the existing IoT platforms in the

market.

3 METHODOLOGY

When selecting the most suitable IoT platform from

numerous options, scoring procedures such as utility

value analysis are leveraged. Scoring models involve

quantitatively assessing objects or subjects based on

weighted criteria. UA, a type of scoring, compares

decision alternatives against each other ((K

uhnapfel,

2021; Cascio, 1996)). Adapted for this research work,

the procedure steps outlined by K

uhnapfel (2021) are

as follows:

In step 1, the university describes its objective and

outlines the prerequisites for an IoT platform. The ob-

jective clariﬁes the rationale behind employing UA.

The requirements and design considerations for the

selecting process considering the current IoT infras-

tructure are gathered through expert interviews and

brainstorming techniques (cf. (Geschka and Geschka,

2014; Soest, 2023)) speciﬁed requirements are con-

solidated into a requirements table, which serves to

establish inclusion criteria for IoT platforms follow-

ing the guidelines for requirements engineering in

software development by Kassab (2022) and Lam-

sweerde (2009).

In step 2, ﬁve decision alternatives are chosen

based on predeﬁned exclusion criteria. These criteria

are deﬁned through interviews with university stake-

holders.

In Step 3, the evaluation criteria (decision crite-

ria) are determined based on the identiﬁed require-

ments. The criteria are essential for the success of this

methodology and should therefore be chosen care-

fully and refer to the properties of software and il-

lustrate the usefulness of a decision alternative. It is

important to note that there are criteria that are viewed

objectively and criteria that can be viewed subjec-

tively.

Step 4 includes the weighting of the decision crite-

ria based on subjective assessment using percentages,

where the sum of the individual weights is 100%. In

order to ﬁnd out the importance of the respective cri-

teria, the paired comparison method ((David, 1963))

was used.

In Step 5, a scale is deﬁned for evaluating the

achievement of the goal criterion. It was decided to

employ a numerical ordinal scale ranging from 0 to 4.

Step 6 includes evaluating the decision criteria.

Here, points are awarded based on the predeﬁned

scoring system. To conduct the evaluation, data

was sourced from the manufacturer documentation of

each IoT platform and insights gleaned from the test

setups of the on-premise installations.

In Step 7, the utility value or score is calculated.

The points awarded for each criterion are multiplied

IoTBDS 2024 - 9th International Conference on Internet of Things, Big Data and Security

312

by their respective weightings to determine the crite-

rion’s utility value for a decision alternative. The total

utility value is then calculated by summing up these

individual utility values.

Step 8 involves making the decision and explain-

ing the result. Here, a decision is made based on the

calculated overall utility values and summarized in a

short discussion.

4 REQUIREMENTS

ELICITATION

To assess an appropriate IoT platform for the uni-

versity’s application context, a thorough requirement

analysis is necessary. This involves conducting ex-

pert interviews and employing brainstorming tech-

niques ((Geschka and Geschka, 2014; Soest, 2023)).

In this context, the administrators of the IoT devices

that were previously managed via The Things Net-

work over the past three years are considered ex-

perts. With increasing experience working with The

Things Network in the Smart Campus area, essential

needs have surfaced that surpass the capabilities of

The Things Network. Of utmost signiﬁcance is the

need to accommodate storage for any number of sen-

sors, gateways, and Application Programming Inter-

faces (API) keys. Unfortunately, this is not possi-

ble with The Things Network as only a limited num-

ber of devices and API keys can be stored. An in-

house hosted on-premise IoT platform offers the ad-

vantage by enabling the creation of multiple platform

instances tailored for educational purposes, speciﬁ-

cally catering to students studying IoT. Therefore, an

open-source solution is necessary. These instances

provide a hands-on experience during seminars or ex-

ercises within lectures. Furthermore, the ﬂexibility to

spin up additional instances on the university’s cloud

infrastructure further enhances accessibility and scal-

ability, empowering students to explore IoT concepts

in a practical and interactive environment. The re-

quirements for an IoT platform are closely linked to

the operation of the Smart Campus LoRaWAN infras-

tructure. For this reason, relevant terms such as EUI

can be found in the LoRaWAN requirement tables (1

and 2). The Extended Unique Identiﬁer (EUI) or De-

vEUI is a 64-bit identiﬁer that is provided to a manu-

facturer of IoT devices and is standardized. The prin-

ciple is comparable to the concept of MAC addresses

for addressing computers on Ethernet. The only dif-

ference is the length of the identiﬁer. The IEEE Reg-

istration Authority is responsible for assigning an EUI

area for manufacturers.

The requirements are categorized into functional

and non-functional aspects. Functional requirements

outline speciﬁc functions that the IoT platform must

prioritize implementing (Table 1). Non-functional re-

quirements, on the other hand, pertain to performance

standards and quality attributes of the software, en-

suring the necessary functionality is delivered (Table

2). The goal was to prioritize the identiﬁed require-

ments based on their importance. This prioritization

is crucial for evaluating the deﬁned categories during

the subsequent UA. This process is signiﬁcant due to

the presence of exclusion criteria, which, if not met,

can result in immediate rejection of the software as

they are considered essential or strongly desired by

the university environment. Vice-versa, if other re-

quirements of lesser signiﬁcance are unmet, the se-

lection process does not necessarily lead to rejection

but could still inﬂuence decision-making among sev-

eral options. To organize these priorities effectively,

the requirements table is segmented into ”must-have,”

”should-have,” and ”nice-to-have” categories. Must-

have requirements hold the highest priority and en-

compass the exclusion criteria. While not indispens-

able for the success of the IoT platform, should-have

requirements should be implemented if feasible, pro-

vided they do not compromise any must-have require-

ments. Nice-to-have requirements, while less critical,

can still contribute signiﬁcant value if met, and thus,

their fulﬁllment should not be entirely disregarded.

In total, 14 functional and 21 non-functional re-

quirements are deﬁned and categorized into various

domains. For instance, functional requirements were

denoted by numbers like ”FR1” for abbreviation,

and non-functional requirements as ”NFR”. As de-

picted in Table 1, functional requirements span across

ﬁve distinct domains: (FR1) General requirements

regarding the functions of the IoT platform, (FR2)

Data Management), (FR3) Functional requirements

regarding interoperability, (FR4) Functional require-

ments for error handling and exception cases and

(FR5) Functional requirements for managing users

and access rights. General requirements for the

IoT platform (FR1) include basic functions that are

commonly provided also by other platforms such as

the gateway registration via an EUI, the data recep-

tion from LoRaWAN gateways and sensor registra-

tion. Under (FR2) Data Management we declared

the (FR2.1) On Premise-Hosting as a must-have. For

data security and compliance, access control to IoT

devices and compliance requirements such as the Eu-

ropean data protection regulation can be met as data

https://lora-developers.semtech.com/documentation/

tech-papers-and-guides/the-book/deveui/

Comparing On-Premise IoT Platforms: Empowering University of Things Ecosystems with Effective Device Management

313

Table 1: Functional requirements for an IoT platform.

No. Requirement Name Description Priority

FR1 General requirements regarding the functions of the IoT platform

FR1.1 Gateway registration

via EUI

The IoT platform must provide registration of IoT gateways via an

EUI.

Must-have

FR1.2 Data reception from

LoRaWAN gateways

The IoT platform must enable receiving data via registered IoT gate-

ways.

Must-have

FR1.3 Livelog The IoT platform is intended to provide a live log of the data ﬂowing

through the gateways.

Should-have

FR1.4 Sensor registration via

EUI

The IoT platform must provide registration of the organization’s IoT

sensors via an EUI.

Must-have

FR1.5 Payload Formatter The IoT platform can provide a function to format the received data

into a desired format such as JavaScript Object Notation.

Nice-to-have

FR2 Data management

FR2.1 On Premise-Hosting The IoT platform must offer the option of being hosted on-premise

on a cloud.

Must-have

FR3 Functional requirements regarding interoperability

FR3.1 Lightweight Direc-

tory Access Protocol

(LDAP) interface

The IoT platform should have an Lightweight Directory Access Pro-

tocol (LDAP) interface to access the organization’s own user IDs.

Should-have

FR3.2 API interface The IoT platform must have an API interface for retrieval of data by

other applications via an API key.

Must-have

FR3.3 Webhook The IoT platform must have webhook integration, to automatically

send data to the organization’s cloud.

Must-have

FR3.4 MQ Telemetry Trans-

port (MQTT) interface

The IoT platform must provide an MQ Telemetry Transport (MQTT)

interface for data transmission.

Must-have

FR4 Functional requirements for error handling and exception cases

FR4.1 Automatic notiﬁcation If an IoT gateway or IoT sensor fails, the IoT platform should send

a notiﬁcation.

Should-have

FR4.2 Monitoring The IoT platform is intended to provide a monitoring function to

provide an overview of the device status of the end devices.

Should-have

FR5 Functional requirements for managing users and access rights

FR5.1 User management The IoT platform must have user management with access rights

management.

Must-have

FR5.2 Authorization concept The IoT platform is intended to assign rights to groups to which users

can be assigned.

Should-have

FR5.3 Assignment of rights to

API key

The IoT platform should enable the assignment of rights to API keys

to restrict access from the respective external application.

Should-have

FR5.4 Local user login In addition to Lightweight Directory Access Protocol (LDAP), the

IoT platform should also be able to create local users.

Should-have

governance. Furthermore, the university may have

control over effective and efﬁcient data storage and

backup procedures due to the on-premise hosting.

Lastly, access only to authorized users can be granted

by having full control over the application hosting.

Furthermore, within FR3, which encapsulates in-

teroperability requirements, the signiﬁcance of API

interfaces, webhooks, and MQ Telemetry Transport

(MQTT) was underscored. Stakeholders highlighted

in the requirements survey the need for processing

sensor data further in a custom-built IoT data plat-

form, along with data visualization, particularly us-

ing the open-source software Grafana. Consequently,

the existence of suitable interfaces becomes imper-

ative. Subsequently, FR4 and FR5 outline require-

ments concerning error handling and user manage-

ment, respectively. Within these sections, only the

inclusion of logic for user storage, whether through

installation on the local IoT platform database or

Lightweight Directory Access Protocol (LDAP), is

designated as a mandatory criterion.

5 IoT PLATFORM SELECTION

The IoT platforms were evaluated using the UA

according to Cascio (1996) and K

uhnapfel (2021),

which was explained in more detail in Section 3.

The aim of this UA is to select, from a pool of po-

tentially suitable IoT platforms available in the mar-

ket, the one that aligns with the requirements outlined

in section 4. Speciﬁc criteria are deﬁned, and failure

to meet any of these criteria will result in the exclu-

sion of an IoT platform from consideration. In addi-

IoTBDS 2024 - 9th International Conference on Internet of Things, Big Data and Security

314

Table 2: Non-Functional requirements for an IoT platform.

No. Requirement Name Description Priority

NFR1 Security

NFR1.1 Access control The system may only be used after successful authentication via the

LDAP of the organization or accessible by a locally created user.

Must-have

NFR1.2 Encryption The IoT platform must implement encryption standards to protect

data from unauthorized access.

Must-have

NFR1.3 Password veriﬁcation The IoT platform can include functionality to assess the strength of

passwords entered by users.

Nice-to-have

NFR1.4 Password Policies The IoT platform can provide functionality to conﬁgure password

policies.

Nice-to-have

NFR2 Reliability

NFR2.1 Fail-safety The system must be 99% available. Must-have

NFR3 User friendliness

NFR3.1 User-friendly interface The system is designed to offer a user-friendly interface to simplify

working on the IoT platform.

Should-have

NFR3.2 Mobile interface The user interface should be accessible on mobile devices. Should-have

NFR4 Scalibility

NFR4.1 Number of API keys The IoT platform must enable the creation of an unlimited number

of API keys.

Must-have

NFR4.2 Number of IoT gate-

ways

The IoT platform must enable the registration of at least 30 gate-

ways.

Must-have

NFR4.3 Number of users The IoT platform must be free of restrictions regarding the number

of users created.

Must-have

NFR4.4 Number of devices The IoT platform should enable the registration of at least 1000 de-

vices.

Must-have

NFR4.5 Number of Groups (Or-

ganizations) / Multite-

nancy

The IoT platform should be free of restrictions regarding the number

of groups or organizations created

Should-have

NFR5 Performance

NFR5.1 Latency The IoT platform must have low latency for real-time data process-

ing in IoT applications.

Must-have

NFR6 Legal requirements

NFR6.1 Selection of frequency

ranges

The IoT platform must have a frequency range conﬁguration on

which the IoT gateways receive data.

Must-have

NFR6.2 Data protection Personal sensitive data must be collected and kept following data

protection regulations.

Must-have

NFR7 Technical requirements

NFR7.1 LoRaWAN network ar-

chitecture

The IoT platform must be based on the LoRaWAN Network archi-

tecture.

Must-have

NFR7.2 Network server The IoT platform must provide a network server for routing the data

from the IoT gateway.

Must-have

NFR7.3 Device drivers The IoT platform should offer the possibility of providing the orga-

nization’s device drivers to reuse them.

Nice-to-have

NFR7.4 Device driver properties Device drivers should incorporate features for payload handling, de-

code/encode functionalities, and settings conﬁguration.

Should-have

NFR8 Financial constraints

NFR8.1 Use of non-proprietary

software

The IoT platform must be installed and accessible without the need

for licensing fees.

Must-have

tion, the criteria were weighted according to a per-

centage in order to include their importance in the

evaluation.

As a result of the expert interviews, two signiﬁ-

cant exclusion criteria were identiﬁed which are (1)

No On-Premise Hosting and (2) Paid license mod-

els. Installing an on-premise IoT platform on the uni-

versity’s own cloud infrastructure offers several ad-

vantages over relying on external platforms like “The

Things Network”. Firstly, it provides greater con-

trol and autonomy over data privacy and security, as

sensitive information remains within the university’s

trusted environment. Secondly, it enables customiza-

tion and tailoring of the IoT platform to speciﬁc needs

and requirements without dependency on external ser-

vice providers. Additionally, it reduces reliance on

third-party services, mitigating the risk of service dis-

ruptions or changes in terms of service as happened to

The Things Network infrastructure in the past. Over-

all, deploying an on-premise IoT platform enhances

Comparing On-Premise IoT Platforms: Empowering University of Things Ecosystems with Effective Device Management

315

data governance, ﬂexibility, and reliability for uni-

versities that seek robust and scalable IoT solutions.

After considering the two exclusion criteria, the op-

tions for suitable IoT platforms can be signiﬁcantly

reduced. Following a brief market survey, only ﬁve

IoT platforms are found to meet both criteria. On-

premise hosting can additionally improve latency due

to the infrastructure being located onsite. Moreover,

it removes the reliance on a stable internet connec-

tion for accessing the IoT platform, which is essential

for cloud-based services. Following the completion

of the market analysis, the following IoT platforms

were identiﬁed, taking into consideration the previ-

ously mentioned exclusion criteria: (1) Thingsboard

(2) ChirpStack

, (3) SiteWhere

, (4) Mainﬂux

, (5)

The Things Stack Sandbox

Table 3: Cross table for the pair comparison method.

Criteria

1 2 3 4 5 6 7 8 9 Σ

1 1 1 1 1 1 1 1 1 8

2 3 2 2 6 7 8 2 3

3 4 3 3 7 8 3 4

4 5 4 7 8 9 2

5 6 7 8 9 1

6 7 8 9 2

7 8 9 5

8 8 7

9 4

According to the requirements identiﬁed in the

requirements table, the following decision criteria

were formed for evaluation: (1) Range of Functions,

(2) User Management, (3) Interoperability, (4) Er-

ror Handling, (5) Security, (6) User-Friendliness, (7)

Scalability, (8) IoT Protocol Support, and (9) Techni-

cal Aspects.

(1) Range of Functions: This criterion describes all

functions that are offered by the IoT platform, fulﬁll-

ing the requirements across all priority levels.

(2) User Management describes the extent to which

it is possible to manage users, groups, and the corre-

sponding allocation of rights within the IoT platform

is evaluated here. This includes determining the feasi-

bility of establishing an LDAP connection and creat-

ing local users, as well as the ability to assign permis-

https://thingsboard.io/docs/getting-started-guides/

what-is-thingsboard/

https://www.chirpstack.io/docs/architecture.html

https://sitewhere1.sitewhere.io/overview.html

https://mainﬂux.readthedocs.io/en/latest/architecture/

https://www.thethingsindustries.com/docs/reference/

sions regarding API keys to regulate access for both

users and external applications.

(3) Interoperability: This aspect examines the IoT

platform’s capability to communicate with other de-

vices and systems, encompassing the availability of

interfaces aligned with requirement FA3 as outlined

in the Table 1.

(4) Error Handling: This criterion evaluates the ex-

tent to which monitoring functions are integrated

within the IoT platform and the extent to which no-

tiﬁcations are sent in the event of a sensor or gateway

failure, for example. This also includes notiﬁcations

in the event of interrupted data ﬂow or the failure of a

service within the platform.

(5) Security: This evaluation criterion includes all im-

plemented security precautions within the IoT plat-

form, which protects against unauthorized access to

the data by third parties. This refers, for example,

to the need for a strong password through password

policies and encrypting all trafﬁc.

(6) User-Friendliness: To facilitate ergonomic user

interactions within the platform, a user-friendly

Graphical User Interface (GUI) is essential. This cri-

terion ensures that this requirement is fulﬁlled.

Table 4: Calculation of the critical weight as a percentage.

Criteria

Score

Conversion

Weight

Range of functions 8

score

22%

IoT Protocol Support 7 19%

Scalability 5 14%

Technical Aspects 4 11%

Interoperability 4 11%

User Management 3 8%

Error Handling 2 6%

User-Friendliness 2 6%

Security 1 3%

Total 36 100%

(7) Scalability: Given that IoT use cases for enhanc-

ing research and education at the university are con-

tinually evolving, and the IoT infrastructure is ex-

pected to expand with additional components, the

scalability of the IoT platform becomes vitally impor-

tant. This scalability accommodates a growing num-

ber of sensors, gateways, users, and API keys.

(8) IoT Protocol Support: This criterion is used to

evaluate whether the IoT platform for the University

of Things infrastructure includes an appropriate se-

lection of IoT protocols. The presence of MQTT, Lo-

RaWAN, and Webhook is considered essential.

(9) Technical Aspects: This evaluation considers mul-

IoTBDS 2024 - 9th International Conference on Internet of Things, Big Data and Security

316

tiple technical requirements, including compatibility

with the LoRaWAN network architecture and features

like Over-the-Air Activation (OTAA) for IoT sensors.

Additionally, it assesses the platform’s capability to

set the license-free frequency range.

The weighting of the decision criteria was carried

out in a cross-tab using the paired comparison method

((David, 1963)) according to the principle “Is more

important than” (see Table 3). The far right column

sums up how often a criterion exceeded another crite-

rion in its relevance.

Table 4 shows the result of the criteria weighting

using the cross-tab, sorted according to the ranking

of their relevance. The percentages were rounded to

whole numbers.

Once the criteria are prioritized based on their sig-

niﬁcance, an appropriate scale for evaluation is es-

tablished. An ordinal scale ranging from 0 to 4 is

designated for all criteria. If a criterion is not ful-

ﬁlled, it receives zero points. If the criterion is inade-

quately met with signiﬁcant deﬁciencies, one point is

awarded. Adequate fulﬁllment with deﬁciencies war-

rants three points, while a good extent of fulﬁllment

merits three points. Finally, four points are granted

for excellent fulﬁllment. The evaluation of decision

criteria is depicted in Table 5. If information regard-

ing an IoT platform was unavailable from the test en-

vironment or manufacturer documentation, the crite-

rion was labeled as ”No information.”

The calculation of the utility value for each de-

cision alternative is presented in Table 6. In sum-

mary, the open-source software ChirpStack emerged

as the top-rated option, achieving a utility value of

3.63. This platform is speciﬁcally designed for con-

necting LoRaWAN end devices and encompasses es-

sential core functions such as device connectivity,

management, data transformation, event handling,

and API management. Additionally, it offers cross-

sectional functionalities including user management,

encryption, and platform administration. ChirpStack

is deemed the optimal choice due to its similarity to

The Things Stack Sandbox, serving as a LoRaWAN

network server.

Moreover, ChirpStack includes an application

server, join server, and geolocation server for end de-

vice positioning. It boasts scalability without limita-

tions on gateways, devices, and users, ensuring com-

patibility with future expansions of the IoT system

and increasing use cases on campus. The integration

into existing UoT infrastructure and interoperability

with external applications allows for continual exten-

sion of functionality by connecting additional IoT ap-

plications.

While Thingsboard also presents a viable alterna-

tive, it offers a comprehensive suite of IoT functions

tailored to industrial IoT, potentially exceeding ex-

plicit university requirements. However, it’s worth

noting that the developers of Thingsboard also pro-

vide paid models. The Community Edition, an open-

source version of Thingsboard, is available under the

Apache 2.0 License, albeit without any support of-

ferings. Nonetheless, ChirpStack provides the option

for integrating Thingsboard functionalities if needed

in the future.

Apart from Chirpstack, The Things Stack Sand-

box is provided as Software as a Service (SaaS)

through the cloud by The Things Industries, the man-

ufacturer also provides it as an open-source option for

installation on-premise. This alternative is closely be-

hind ChirpStack in the rating with a utility value of

3.54. It offers the same core functions as ChirpStack

and The Things Stack Sandbox, but The Things In-

dustries emphasizes that The Things Stack Sandbox

is limited in its features compared to the paid enter-

Table 5: Evaluation of the decision criteria.

No. Criteria Thingsboard

Community

Edition

Chirpstack SiteWhere Mainﬂux The Things

Stack

Sandbox

1 Range of Functions 2 4 4 2 4

2 User Management 1 2 1 2 4

3 Interoperability 3 4 4 4 3

4 Error Handling 3 1 2

Not

speciﬁed

5 Security 3 3

Not

speciﬁed

3 3

6 User-Friendliness 2 4 2 1 4

7 Scalability 4 4 2

Not

speciﬁed

8 IoT Protocol Support 4 4 1 4 4

9 Technical Aspects 4 4 1 4 4

Comparing On-Premise IoT Platforms: Empowering University of Things Ecosystems with Effective Device Management

317

Table 6: Calculation of the utility value of the decision alternatives.

Criteria

Weight

Thingsboard Chirpstack SiteWhere Mainﬂux

The Things

Stack Sandbox

Rating Score Rating Score Rating Score Rating Score Rating Score

1 22% 2 0.44 4 0.88 4 0.88 2 0.44 4 0.88

2 8% 1 0.08 2 0.16 1 0.08 2 0.16 4 0.32

3 11% 3 0.33 4 0.44 3 0.33 3 0.33 3 0.33

4 6% 3 0.18 1 0.06 2 0.12 0 0 1 0.06

5 3% 3 0.09 3 0.09 0 0 3 0.09 3 0.09

6 6% 2 0.12 4 0.24 2 0.12 1 0.06 4 0.24

7 14% 4 0.56 4 0.56 2 0.28 0 0 4 0.42

8 19% 4 0.76 4 0.76 1 0.19 4 0.76 4 0.76

9 11% 4 0.44 4 0.44 1 0.11 4 0.44 4 0.44

Σ 100% 3 3.63 2.11 2.28 3.54

prise version. In addition, according to the manu-

facturer, this is suitable for small-scale testing pur-

poses and not for use within a larger productive en-

vironment. In contrast to The Things Stack Sand-

box, ChirpStack is well-suited for larger production

environments. ChirpStack is primarily available as

an open-source solution, lacking paid enterprise mod-

els offering guaranteed support. Compared to The

Things Stack Sandbox, ChirpStack has, in addition to

the lack of the ability to create user groups, also less

granular setting options for user rights, in contrast, it

offers more integrations in the applications and the

option to create multiple tenants. Apart from a few

differences, both IoT platforms are very similar, and

basically, both represent a suitable decision alterna-

tive for the UoT environment. There is still the pos-

sibility that The Things Industries will impose further

restrictions on the free versions in the future to switch

users to the paid models.

In addition to the required functions, Thingsboard

also supports the modeling of entities and relation-

ships from the real world, i.e. assets, devices, and

their relationships with each other. Furthermore, it

provides real-time data visualization within customiz-

able dashboards. These dashboards support ”multi-

tenancy,” ensuring that users from different tenants

cannot access data or dashboards from other tenants.

This function is crucial in large IoT systems and is

also available in ChirpStack. Another key feature of

Thingsboard is its ”rule engine,” a powerful and so-

phisticated tool for processing incoming data. This

refers to the ﬁltering, enrichment, and transformation

of data and corresponding actions that can be carried

out such as triggering alarms, executing REST API

calls, and sending push notiﬁcations. Within alarm

management, users can create, delete, and conﬁrm

alarms. Mainﬂux, an open-source IoT platform, ranks

fourth in the utility analysis. While it lacks native

support for LoRaWAN, it offers a LoRaWAN adapter

as a bridge to the LoRaWAN network. However,

the adapter necessitates an existing LoRaWAN net-

work server, such as ChirpStack, to function. There-

fore, Mainﬂux isn’t a viable choice for the university

without such implementation. Nonetheless, similar to

Thingsboard, it can be integrated with a ChirpStack

instance later on to enhance its capabilities.

The SiteWhere IoT platform encompasses all es-

sential core functionalities, including device connec-

tivity, management, data transformation, event han-

dling, API management, and user administration.

However, details regarding the maximum capacity for

devices and gateways are absent from the documen-

tation. While the scalability of microservices via Ku-

bernetes replication is outlined, there’s no mention of

LoRaWAN network integration or support. Given the

university’s reliance on LoRaWAN for its existing IoT

infrastructure, the absence of this feature renders Site-

Where unsuitable as a decision alternative.

6 CONCLUSIONS

This study involved the development of requirements

for an IoT platform tailored to a University of Things

setting. It drew upon insights from operating the uni-

versity campus’s existing IoT infrastructure and in-

terviews with IoT administrators. Subsequently, a

market analysis was conducted based on both func-

tional and non-functional requirements, followed by

the evaluation of ﬁve on-premise installation IoT plat-

forms using utility analysis.

The ﬁrst research objective involves the collection

of requirements for an IoT platform for the manage-

ment of IoT sensors and gateways for a smart cam-

IoTBDS 2024 - 9th International Conference on Internet of Things, Big Data and Security

318

pus/UoT infrastructure. The process involved utiliz-

ing both operational experiences and insights from in-

stalling the existing IoT infrastructure on the univer-

sity campus. Additionally, the expertise of IoT ad-

ministrators responsible for managing the campus IoT

infrastructure was leveraged. The requirements were

categorized into functional and non-functional types,

identifying 21 non-functional and 16 functional re-

quirements. During the requirement gathering phase,

the university’s IoT experts and stakeholders priori-

tized effective device management, hosting, and user

management. Notably, there was no speciﬁc request

for IoT data visualization and ﬁltering within the IoT

platform, as a separate IoT data platform will be de-

veloped for this purpose. A multitenancy function

was not declared a must-have for user and device

management, as the IoT infrastructure is viewed as

a private domain. Nevertheless, multitenancy is de-

clared a should-have to be able to host the IoT plat-

form for possible future research partners with IoT

gateways and devices. By leveraging its own Infras-

tructure as a Service (IaaS) cloud infrastructure and

hosting an open-source on-premise IoT platform with

multitenancy capabilities, the university not only en-

sures data sovereignty and security but also estab-

lishes a robust foundation for accommodating IoT de-

vices from diverse domains such as Smart City or

Smart Region projects. This setup enables integra-

tion and collaboration between various stakeholders

within the university’s ecosystem, fostering IoT appli-

cations and knowledge exchange while maintaining

control over data privacy and governance as well as

European data protection regulations ((Blazevic and

Riehle, 2023)).

The second research question involved conduct-

ing a market analysis focusing on on-premise IoT

platforms. Utilizing exclusion criteria based on

on-premise and license-free cost models, ﬁve IoT

platforms underwent UA. Consequently, Chirpstack

emerged as the preferred choice for implementation

within the Smart Campus/University of Things envi-

ronment, fulﬁlling all essential and several additional

criteria. The Things Stack Sandbox demonstrated

adequacy solely in the realm of user management.

Given that access to the IoT platform for device man-

agement is primarily required by IoT administrators,

elaborate group and user management functionalities

are considered unnecessary in the end.

In future research projects, we recommend opti-

mizing and reﬁning the requirements for an IoT plat-

form, as well as evaluating the use of Chirpstack

over a longer period. In addition, the requirements

and evaluation criteria should not only be deﬁned for

smart campus infrastructures but also expanded in a

more abstract sense so that they can also be used

for other IoT domains such as smart cities or smart

health. Thus, these requirements can be useful for

projects when selecting IoT platforms. The beneﬁt of

the multi-tenant function would also be interesting for

smart region projects that rely on the use of an open-

source platform for cost and data protection reasons.

Since an IoT platform like Chirpstack is already in-

stalled in an in-house cloud infrastructure, this effort

is no longer necessary for smart region organizations.

ACKNOWLEDGEMENTS

This research has been supported by the Deutsche

Forschungsgemeinschaft (DFG) under Research

Grant No. 432399058.

REFERENCES

Arz von Straussenburg, A. F., Blazevic, M., and Riehle,

D. M. (2023). Measuring the actual ofﬁce workspace

utilization in a desk sharing environment based on

iot sensors. In Gerber, A. and Baskerville, R., ed-

itors, 18th International Conference on Design Sci-

ence Research in Information Systems and Technol-

ogy, DESRIST 2023, Pretoria, South Africa, number

13873 in Lecture Notes in Computer Science (LNCS),

pages 69–83, Cham. Springer Nature.

Baiyere, A., Topi, H., Wyatt, J., Venkatesh, V., and Donnel-

lan, B. (2020). Internet of Things (IoT) – A Research

Agenda for Information Systems. Communications of

the Association for Information Systems, 47(1):21.

Barros, T. G. F., Da Silva Neto, E. F., Neto, J. A. D. S.,

De Souza, A. G. M., Aquino, V. B., and Teixeira,

E. S. (2022). The Anatomy of IoT Platforms—A

Systematic Multivocal Mapping Study. IEEE Access,

10:72758–72772.

Berawi, M. A., Sunardi, A., and Ichsan, M. (2019). Chief-

Screen 1.0 as the Internet of Things Platform in

Project Monitoring & Controlling to Improve Project

Schedule Performance. Procedia Computer Science,

161:1249–1257.

Bhajantri, L. B. and S., G. (2020). A Comprehensive Sur-

vey on Resource Management in Internet of Things.

Journal of Telecommunications and Information Tech-

nology, 4(2020):27–43.

Blazevic, M. and Riehle, D. M. (2023). University of

Things: Opportunities and Challenges for a Smart

Campus Environment based on IoT Sensors and Busi-

ness Processes. In Proceedings of the 8th Interna-

tional Conference on Internet of Things, Big Data and

Security IoTBDS - Volume 1, pages 105–114.

Cascio, W. F. (1996). The Role of Utility Analysis in the

Strategic Management of Organizations. Journal of

Human Resource Costing & Accounting, 1(2):85–95.

Chui, M., L

ofﬂer, M., and Roberts, R. (2020). The Internet

of Things. McKinsey Quarterly.

Comparing On-Premise IoT Platforms: Empowering University of Things Ecosystems with Effective Device Management

319

David, H. A. (1963). The method of paired comparison.

Hafner Publishing Company, New York.

Elgedawy, I. and Shoukry, L. (2022). SANAD: A Com-

prehensive IoT Reference Architecture for Integrated

Enterprise Platforms. In 2022 IEEE Global Confer-

ence on Artiﬁcial Intelligence and Internet of Things

(GCAIoT), pages 182–187.

Fahmideh, M. and Zowghi, D. (2020). An exploration

of IoT platform development. Information Systems,

87:101409.

Fortes, S., Santoyo-Ram

on, J., Palacios, D., Baena, E.,

Mora-Garc

ıa, R., Medina, M., Mora, P., and Barco,

R. (2019). The Campus as a Smart City: University of

alaga Environmental, Learning, and Research Ap-

proaches. Sensors, 19(6):1349.

Garc

ıa-Valls, M. and Palomar-Cos

ın, E. (2022). An Evalu-

ation Process for IoT Platforms in Time-Sensitive Do-

mains. Sensors, 22(23):9501.

Garda

sevi

c, G., Veleti

c, M., Maleti

c, N., Vasiljevi

c, D.,

Radusinovi

c, I., Tomovi

c, S., and Radonji

c, M.

(2017). The IoT Architectural Framework, Design

Issues and Application Domains. Wireless Personal

Communications, 92(1):127–148.

Geschka, H. and Geschka, M. (2014). Ideenﬁndung und

Konzeptentwicklung.

Guth, J., Breitenb

ucher, U., Falkenthal, M., Fremantle, P.,

Kopp, O., Leymann, F., and Reinfurt, L. (2018). A De-

tailed Analysis of IoT Platform Architectures: Con-

cepts, Similarities, and Differences. In Di Martino, B.,

Li, K.-C., Yang, L. T., and Esposito, A., editors, Inter-

net of Everything: Algorithms, Methodologies, Tech-

nologies and Perspectives, Internet of Things, pages

81–101. Springer, Singapore.

Ibarra-Esquer, J., Gonz

alez-Navarro, F., Flores-Rios, B.,

Burtseva, L., and Astorga-Vargas, M. (2017). Track-

ing the Evolution of the Internet of Things Con-

cept Across Different Application Domains. Sensors,

17(6):1379.

Kassab, Mohamad, P. A. L. (2022). Requirements Engi-

neering for Software and Systems. Auerbach Publica-

tions, New York, 4 edition.

Kugler, S., Czwick, C., Nguyen, H. P. L., and Anderl,

R. (2021). Method for the Targeted Selection and

Installation of an IoT - Platform. In Trzcielinski,

S., Mrugalska, B., Karwowski, W., Rossi, E., and

Di Nicolantonio, M., editors, Advances in Manufac-

turing, Production Management and Process Control,

Lecture Notes in Networks and Systems, pages 116–

123, Cham. Springer International Publishing.

uhnapfel, J. B. (2021). Scoring und Nutzwertanalysen:

Ein Leitfaden f

ur die Praxis. Springer Fachmedien,

Wiesbaden.

Lamsweerde, A. (2009). Requirements engineering: from

system goals to UML models to software speciﬁca-

tions. John Wiley & Sons, Ltd.

Lempert, S. and Pﬂaum, A. (2019). Vergleichbarkeit der

Funktionalit

at von IoT-Software-Plattformen durch

deren einheitliche Beschreibung in Form einer Tax-

onomie und Referenzarchitektur. HMD Praxis der

Wirtschaftsinformatik, 56(6):1178–1203.

Mart

ınez, I., Zalba, B., Trillo-Lado, R., Blanco, T., Cambra,

D., and Casas, R. (2021). Internet of Things (IoT)

as Sustainable Development Goals (SDG) Enabling

Technology towards Smart Readiness Indicators (SRI)

for University Buildings. Sustainability, 13(14):7647.

Miorandi, D., Sicari, S., De Pellegrini, F., and Chlamtac,

I. (2012). Internet of things: Vision, applications and

research challenges. Ad Hoc Networks, 10(7):1497–

1516.

Mircea, M., Stoica, M., and Ghilic-Micu, B. (2021). Inves-

tigating the Impact of the Internet of Things in Higher

Education Environment. IEEE Access, 9:33396–

33409.

Nasser, A. A., Al-Ashwal, M. M. Y., Al-Khulaidi, A. A. G.,

Al-Naqeep, A. N., and Al-jober, M. (2023). A Hybrid

Business-Technical Model for Evaluating IoT Plat-

forms’ Functionality, Reliability, and Usability. In-

ternational Journal of Engineering Trends and Tech-

nology - IJETT, 71(10):39–59.

Panduman, Y. Y. F., Sukaridhoto, S., and Tjahjono, A.

(2019). A Survey of IoT Platform Comparison for

Building Cyber-Physical System Architecture. In

2019 International Seminar on Research of Infor-

mation Technology and Intelligent Systems (ISRITI),

pages 238–243.

Rayes, A. and Salam, S. (2017). Internet of things from

hype to reality. Springer Basel.

Riehle, D. M., Arz von Straussenburg, A. F., Blazevic, M.,

and Wolters, A. (2023). Sensor-based use case ad-

vancements in smart parking - pioneering the next

generation of smart city applications. In Proceedings

of the 34th Australasian Conference on Information

Systems ACIS 2023, 5-8 December 2023, Wellington,

New Zealand, Atlanta, GA. AIS eLibrary.

Silva-da N

obrega, P. I., Chim-Miki, A. F., and Castillo-

Palacio, M. (2022). A Smart Campus Framework:

Challenges and Opportunities for Education Based on

the Sustainable Development Goals. Sustainability,

14(15):9640.

Soest, C. v. (2023). Why Do We Speak to Experts? Re-

viving the Strength of the Expert Interview Method.

Perspectives on Politics, 21(1):277–287.

Toutsop, O., Kornegay, K., and Smith, E. (2021). A Com-

parative Analyses of Current IoT Middleware Plat-

forms. In 2021 8th International Conference on Fu-

ture Internet of Things and Cloud (FiCloud), pages

413–420.

Vasileva, R., Rodrigues, L., Hughes, N., Greenhalgh, C.,

Goulden, M., and Tennison, J. (2018). What Smart

Campuses Can Teach Us about Smart Cities: User Ex-

periences and Open Data. Information, 9(10):251.

Wolters, A., Blazevic, M., and Riehle, D. M. (2023). On-

premise internet of things (iot) data storage: Compar-

ison of database management systems. In Proceed-

ings of the 8th International Conference on Internet

of Things, Big Data and Security IoTBDS - Volume 1,

pages 140–149.

Zhang, Y., Yip, C., Lu, E., and Dong, Z. Y. (2022). A

Systematic Review on Technologies and Applications

in Smart Campus: A Human-Centered Case Study.

IEEE Access, 10:16134–16149.

IoTBDS 2024 - 9th International Conference on Internet of Things, Big Data and Security

320