ConCERTS: An IoT Cybersecurity Research Range for Education,

Experimentation, and Security Research

Dave McKay

, Matthew Bush

, Marko Kovacevic

and Atefeh Mashatan

Cybersecurity Research Lab, Ted Rogers School of Information Technology Management, Toronto Metropolitan University,

350 Victoria St, Toronto, Canada

{dave.mckay, matthew.bush, marko.kovacevic, amashatan}@torontomu.ca

Keywords:

Cybersecurity Range, IoT Security, Simulated IoT Devices, Scripted IoT, Reproducible IoT Datasets.

Abstract:

Internet-of-Things (IoT) connected devices in enterprise and residential environments are weak points in net-

work security. Security testing over a variety of IoT network conﬁgurations is hampered by device, infras-

tructure, and architectural heterogeneity that is often associated with IoT deployments. Though IoT testing

environments have been developed, they often lack the ﬂexibility, control, and visibility required for varied and

repeatable education, research, and testing scenarios. This paper aims to address this gap by both exploring

the shortcomings of existing IoT test environments and proposing ConCERTS - a Conﬁgurable Cybersecurity

Education Range and Testing Stack as a novel architectural approach for IoT testbeds and cybersecurity re-

search ranges.

1 INTRODUCTION

The Internet of Things (IoT) is rapidly changing our

lives and its inﬂuence will only continue to grow.

The number of IoT enabled devices is predicted to

reach 24.1 billion by 2030 (Xenofontos et al., 2022).

Though these devices vary immensely in computing

power and purpose, they are all capable of receiving

and transmitting data and are therefore vulnerable to

attacks. IoT devices also have access to the physi-

cal world through a combination of sensors, actua-

tors, and controllers. The ubiquity of IoT and its ac-

cess to the physical world makes these devices partic-

ularly important to secure. Unfortunately, increased

IoT connectivity is creating a number of security con-

cerns and potential attack surfaces (Modarresi and

Symons, 2020). Potential attacks and vulnerabilities

that could compromise a number of devices have al-

ready been discovered (Khoury et al., 2021). Man-

aging the security of these devices is made even more

difﬁcult by the sheer number of device types and net-

work conﬁgurations that exist. IoT devices are small,

ubiquitous, low-powered, forgettable, and prime tar-

gets for malware. Most consumers never update the

https://orcid.org/0009-0007-4450-8614

https://orcid.org/0000-0002-1894-3424

https://orcid.org/0000-0001-5918-2337

https://orcid.org/0000-0001-9448-9123

ﬁrmware on the smart light bulb or camera in their

home (Tawalbeh et al., 2020). The combination of in-

ternet accessible IoT devices and unpatched ﬁrmware

greatly increases the attack surface of IoT networks.

The Mirai Botnet is an example of malware that took

advantage of lax IoT security to take over a huge num-

ber of devices worldwide which were subsequently

used in distributed denial of service (DDoS) attacks

(De Donno et al., 2018). To improve education and

advance IoT security, researchers often turn to cre-

ating various IoT testbeds and ranges. While many

testbeds and ranges are accessible to researchers, they

have shortfalls in control, ﬂexibility, reproducibility,

or visibility. This can be remedied through creating

a custom test environment or manually reconﬁguring

an existing one, these solutions often require a pro-

hibitive degree of time and technical expertise. To

address the challenges present in traditional IoT re-

search and education, a novel architectural approach,

named Conﬁgurable Cybersecurity Education Range

and Testing Stack (ConCERTS), is presented here

along with two demonstration scenarios.

This paper explores several existing IoT ranges

and testbeds that have applications for both security

research and education. A literature review has been

undertaken to identify gaps in the design considera-

tions undertaken in developing them. The missing

considerations are identiﬁed as requirements for the

proposed ConCERTS architecture. The architecture

McKay, D., Bush, M., Kovacevic, M. and Mashatan, A.

ConCERTS: An IoT Cybersecurity Research Range for Education, Experimentation, and Security Research.

DOI: 10.5220/0013370900003899

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 11th International Conference on Information Systems Security and Privacy (ICISSP 2025) - Volume 1, pages 71-82

ISBN: 978-989-758-735-1; ISSN: 2184-4356

enables an open environment in which researchers

have complete visibility, control, reproducibility, and

ﬂexibility over all core building blocks of IoT - from

the control server, down to the physical sensors and

actuators. ConCERTS enables researchers to create

fully localized and extensible test scenarios, capture

activity data and play it back. A basic device, such

as a Raspberry Pi, can be connected to a variety of

physical sensors and can be have its behaviour re-

motely conﬁgured to allow one device to simulate

several very different IoT devices. Controllers for

these simulated IoT devices run locally and can be

programmed to enable many use cases (e.g., access

control, inventory, etc.). Device activity scripting al-

lows researchers to perform reproducible experiments

and generate comprehensive datasets that are not lim-

ited by the heterogeneity of the network or human and

environmental interactions. The viability of the pro-

posed architecture has been tested by implementing

two scenarios, a door lock card reader and a Heating

Ventilation and Air Conditioning (HVAC) system.

The structure of this paper is as follows. Section

2 details the background and reports on the literature

review of existing approaches for testbeds and ranges.

Section 3 outlines the key requirements identiﬁed as

necessary for IoT ranges and testbeds for research and

training. Section 4 proposes an architecture for Con-

CERTS and details the implementation of the demon-

strations. Section 5 provides a discussion on relevant

outcomes of the approach taken. Section 6 outlines

the next steps and potential future work related to this

project. The conclusions of this work are drawn in

Section 7.

2 BACKGROUND AND RELATED

WORK

IoT devices are popular for DDoS attacks be-

cause the randomness of where the devices reside

makes it difﬁcult to detect abnormal trafﬁc patterns

for anti-DDoS systems (De Donno et al., 2018).

Security researchers have even developed a ran-

somware that locks and controls your smart thermo-

stat (Franceschi-Bicchierai, 2016). Many IoT de-

vices on Shodan often include vulnerable ﬁrmware

or default usernames and passwords (Albataineh and

Alsmadi, 2019). These devices are prime targets for

attackers and are an entry point into networks. Het-

erogeneity is another signiﬁcant concern for IoT net-

works with many devices using different communica-

tion protocols and control interfaces (Hamad et al.,

2019). IoT devices do not always possess robust

or easily accessible monitoring capabilities, nor do

they natively possess the ability to aggregate and ana-

lyze homogeneous data centrally within the network.

Many of these devices also need to be conﬁgured and

connected independently. Not only does this com-

plicate security for existing networks, but it makes

provisioning test networks for research and educa-

tion much more difﬁcult. Many IoT devices may

even store personal or enumerable data in plaintext

(Lodge, 2016). While some IoT devices cryptograph-

ically sign updates to prevent exploits, these updates

can be compromised during the transport or deploy-

ment phases, or in the situations where an attacker has

physical access to the device (Maxxz, 2016). Kolias

et al. (Kolias et al., 2016) highlight security and pri-

vacy threats to commercial IoT use cases, including

leakage of sensitive user information, including per-

sonally identiﬁable information, as well as unautho-

rized function execution. Numerous architectural vul-

nerabilities were discovered, including insecure com-

munications, protocols, and network services. Ad-

ditionally, the use of Cloud services meant placing

trust and control in the hands of a third party. The

design, development, and testing of an IoT product

or system requires a target environment that closely

aligns with real-world use cases. Aligning with IoT

architecture, IoT simulators and ranges need to sup-

port scenarios consisting of heterogeneous elements,

be computationally efﬁcient, and support custom re-

quirements (Sharif and Sadeghi-Niaraki, 2017). Tra-

ditional IoT research focuses on large-scale systems

anchez et al., 2013) (Teranishi et al., 2015) or se-

curity testing of individual commercial IoT devices

(Hale et al., 2018). Most ranges are too complex for

the individual researcher to quickly conﬁgure and run

their own educational or research scenarios. Addi-

tionally, the size and scope of most IoT ranges make

granular control and visibility difﬁcult.

2.1 Review of IoT Testbeds and Ranges

The following is a survey of existing IoT testbeds and

ranges with a description of their architecture, pur-

pose, and an indication of missing features that would

provide control, ﬂexibility, reproducibility, or visibil-

ity.

FIT IoT-Lab: The Very Large Scale IoT Testbed is

one of the most robust open source IoT testbeds pro-

viding access to the range via the internet. Despite

being deployed at six different sites across France,

these testbeds are all interconnected and available at

the same web portal. Additionally, all testbeds pos-

sess common REST interfaces and Command Line

Interface tools. The distinct beneﬁt this testbed pro-

vides is a platform to test and understand networking

ICISSP 2025 - 11th International Conference on Information Systems Security and Privacy

and IoT speciﬁc architecture. In contrast to the range,

FIT is a cloud-based platform as a service (PaaS) that

uses specialized IoT hardware, more complex WSNs,

operating systems and does not have a security focus

(Adjih et al., 2015).

OpenTestBed (Mu

noz et al., 2019), an open-

source IoT testbed built with Raspberry Pi and Open-

Mote B shields, uses a group of 80 motes (IoT Nodes)

that emulate traditional IoT devices. Using Wi-Fi

and WSNs, these devices simulate real-world de-

ployments through minimal setup. What is particu-

larly alluring about OpenTestBed is that it provides

the opportunity to load one’s own ﬁrmware onto de-

vices and observe behaviour when that ﬁrmware is

running. When considering infrastructure, set up

time, and support servers, traditional testbeds can be

quite costly. OpenTestBed acts as a low-cost alterna-

tive (Mu

noz et al., 2019). OpenTestBed devices com-

municate over WiFi to a single controller, are self-

contained, and require only a power connection for

installation. It can be set up within a lab environment

and uses Raspberry Pi for each node. OpenTestBed

focuses on IoT development using Wireless Sensor

Networks and custom ﬁrmware. In addition, it does

not provide any specialized logging concerning de-

vice usage or security.

Jones et al. presented a testbed that incorporates

a number of low-powered sensor tags in a mesh net-

work topology (Jones et al., 2018). The network can

operate with Wi-Fi and 6LoWPAN. The sensor tags

communicate sensor data and intrusion detection sys-

tem (IDS) alerts to an AWS cloud environment via

message queue telemetry transport (MQTT) for pro-

cessing, storing, and displaying data. The primary use

case of the range is testing an edge-based IDS. The

authors validated the effectiveness of the testbed by

running several denial-of-service attacks and explor-

ing the collected data. While the IDS being used in

this case is edge-based, the authors do not discuss ex-

ploring the security of cloud or edge-based applica-

tions themselves. The data being logged in this case

does not include application states and is limited to

things like RAM usage, and CPU usage.

Siboni et al. describe their efforts to create a

testbed for testing individual IoT devices in (Siboni

et al., 2019). The testbed can run multiple security

tests on a given device and automatically record the

results for researchers to use. The testbed consists

of a number of modular components for data collec-

tion, repeatable testing, and management. The phys-

ical testbed contains an orchestrating machine that

communicates with an analysis machine and a control

and communication machine located inside a shielded

room. The testbed is notably capable of context-based

testing where it generates various environmental stim-

uli such as lighting or audio changes. This testbed

however is limited to device testing only and so can-

not be used for full network tests or application-level

testing.

SecLab is another physical testbed that has a par-

ticularly interesting architecture (Schwaiger et al.,

2022) . Interchangeable embedded nodes are con-

trolled directly by a USB connection to an agent ma-

chine. The agent machine is capable of loading pro-

grams onto the embedded nodes and orchestrating

when the embedded nodes are active. The researcher

playing the role of the attacker in this system can

communicate with embedded nodes via REST API

calls. This enables a control network that allows much

greater control, ﬂexibility, and reproducibility for se-

curity tests. This makes simple network level testing

much easier but does not account for some compo-

nents of a complete IoT environment such as inter-

facing with cloud applications or more robust edge

computation.

CyberIoT is a web-based security training plat-

form (Shariﬁ et al., 2021). CyberIoT allows the

creation of scenarios that operate within cloud-based

sandboxes. To do this, the platform has a user-

friendly web presentation, applications for scenarios

and scripting, data collection and monitoring, and in-

frastructure and sandbox provisioning. The platform

allows communication with a public API that sends

commands to a cloud controller. Nodes can instanti-

ate new virtual machines on demand and the network

controllers are responsible for conﬁguring and allo-

cating VLANs and IP addresses. While this is cost-

effective for training purposes, it lacks the ability to

test networks with physical IoT devices. Addition-

ally, there are no details about running a scenario as a

test of the system, so it is unclear what sort of speciﬁc

capabilities the range has in practice.

Thom et al. detail building an IoT testbed for

both security research and education (Thom et al.,

2021). They employ a combination of physical and

virtual devices as well as a software deﬁned network-

ing (SDN) segment to create a robust testing environ-

ment. They have a miniature smart city component

complete with electric trains, street lighting, and re-

alistic trafﬁc signals. There are also several honeypot

devices that act as decoys which aid both in devel-

oping security skills and conducting research. The

SDN segment allows for testing the development of

security techniques in an SDN environment. Trafﬁc

monitoring and data collection are performed at mul-

tiple points in the network. A disadvantage to this

system is it cannot be deployed to capture and replay

real world data as the network structure is largely in-

ConCERTS: An IoT Cybersecurity Research Range for Education, Experimentation, and Security Research

ﬂexible. There is also no mention of IoT applications

interacting with cloud or edge services.

Nock et al. aim to address the difﬁculty of cyber-

security training by creating an IoT range with a mod-

ular front and back-end (Nock et al., 2020). Their

approach supports both virtual devices and a physi-

cal network of Zolertia RE-MOTE IoT devices in a

wireless mesh network. These devices can be conﬁg-

ured to work with a number of sensors, but the au-

thors place no great emphasis on this feature. These

two components are coupled by a RESTful API which

allows users to ignore the complexity and focus on

security training scenarios without limiting the addi-

tions that can be made to support new IoT technolo-

gies in the back end. They emulate an IoT network

using Cooja which users can then build scenarios on

top of using Contiki-NG. Users can run red/blue team

scenarios and simulate cyberattacks to promote secu-

rity education. While running experiments is possi-

ble, the range is far more oriented toward education

and training.

Abu Waraga et al. detail their efforts to create an

IoT-based security testbed for running repeatable tests

on devices (Abu Waraga et al., 2020). The testbed

captures data on a central administrative machine run-

ning the Kali Linux operating system that oversees

all network trafﬁc. The authors also test devices with

mobile applications but data that is collected is limited

by the access that closed-source commercial applica-

tions provide. The testbed captures network data on

application packets, but it cannot be compared with

application-level logs in this case. The researchers

also provide two sample analyses to validate their

testbed using a smart lightbulb and wireless camera.

After an extensive manual review of the devices, au-

tomated testing is performed. A report is generated,

with “vulnerable or “not vulnerable” declarations for

each case. While this testbed identiﬁes vulnerabili-

ties within commercial devices, it does not enable the

researcher to create their own vulnerabilities or de-

vice scenarios. Additionally, the closed-source nature

of many commercial applications makes it difﬁcult to

explore cloud and application interactions in this con-

text.

IoT Cyber Range (IoT CR) enables the user to out-

line and work on customizable IoT networks. Virtu-

alized, scalable, and modular - IoT CR can run mul-

tiple scenarios at once (Nock et al., 2020). IoT CR

uses Contiki-NG, an embedded OS for IoT networked

devices, and Cooja, a full network stack emulator

designed for Contiki-NG. Contiki-NG enables ease

of transfer of OS from the virtualized environment

to physical devices. Because the devices are simu-

lated rather than emulated, many virtualized devices

can run within a virtualization server because of the

low power requirements needed (Nock et al., 2020).

However, the virtual nature of the devices eliminates

common attacks through wireless protocols. Addi-

tionally, the focus is on PaaS without the possibility

of a do-it-yourself (DIY) implementation, setup, and

experimentation. IoT-CR’s focus is a virtual range

where security professionals can simulate and teach

attack and defense strategies.

Leaf (Ficco and Palmieri, 2019) is an open-source

cybersecurity training platform that focuses on IoT

edge computing scenarios that closely aligns with

real-world production deployments. Tasks are as-

signed to users or the scenario by organizers. These

tasks must be completed so that points can be as-

signed to users or teams. The data tracked for the

scenario includes timelines for events and task com-

pletion which allows detailed scenario reviews. While

the authors’ platform provides tools for provisioning

and running scenarios mimicking an enterprise set-

ting, it does not aid in conﬁguring the infrastructure.

This approach grants great ﬂexibility, but it likely

makes the initial setup of scenarios very cumbersome.

Other full stack simulators, which simulate com-

plete real-world deployments, include DPWSim

(Han et al., 2014) (Device Proﬁles for Web Services),

and iFogSim [10] (Fog Computing). Big Data Pro-

cessing Simulators like IOTSim (Zeng et al., 2017)

and SIMIoT (Sotiriadis et al., 2014) focus on cloud

computing and data center mechanics. Network sim-

ulators including Cooja (Osterlind et al., 2006), used

in Nock’s Cyber Range (Nock et al., 2020) and Cup-

Carbon (Mehdi et al., 2014), a smart city IoT simu-

lator focus on protocol research within IoT, typically

with wireless sensor networks. Many of these are far

too complex for individual researchers to completely

control their test environment.

3 REQUIREMENTS

This section presents a set of requirements for an

IoT range to be developed for the purpose of edu-

cation, testing, and research from the perspective of

the stated goals of control, ﬂexibility, reproducibility,

and visibility. The Conﬁgurable Cybersecurity Edu-

cation Range and Testing Stack (ConCERTS) project

has requirements that have been distilled from exist-

ing range and testbed best-practices and gaps in the

review. Table 1 provides a comparison between the

features of some existing IoT testbeds and ranges and

those identiﬁed as requirements for the ConCERTS

architecture.

The goal of user control should involve the re-

ICISSP 2025 - 11th International Conference on Information Systems Security and Privacy

Table 1: Existing Testbeds/Ranges and Comparison with ConCERTS.

Testbed/Range

Requirements

Abu

Waraga

ﬁt-

iot-

lab

FICCO Jones Open

Testbed

IoT

SecLab Cyber

IoT

Siboni Thom Con

CERTS

Open Source ✓ ✓ ✓ ✓ ✓ ✓ ✓

Composable ✓ ✓ ✓ ✓ ✓ ✓ ✓

Operable ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Virtual ✓ ✓ ✓ ✓ ✓ ✓

Simulated ✓ ✓ ✓ ✓ ✓

Commercial

IoT

✓ ✓ ✓ ✓

Logging ✓ ✓ ✓ ✓ ✓ ✓ ✓

Capture from

commerical

devices

✓ ✓ ✓ ✓

Scripting ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Replay ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Convert logs to

scripts

✓

searcher or student in all aspects of the range. As both

a security testing tool and educational device, Con-

CERTS should be accessible, composable, and opera-

ble, prompting requirements for open source develop-

ment and user controlled operations. The user should

be given full access to the source code, they should

be able to combine devices in any mix they require,

and they should be able to control the activity, actions,

logging, scripting, and replay for any device and the

whole network of devices.

Flexibility in the system is required so that the

range can be used for a variety of research and ed-

ucational purposes over a variety of scenarios, with a

variety of devices. Again, having an open source sys-

tem allows for the ﬂexibility of modifying the code or

extending the system architecture. The system should

work in virtual environments, physical simulations,

physical simulations with real-world sensors, IoT de-

vices under control from the simulation, independent

IoT devices, or any combination of these. This can

provide ﬂexibility in the range to be used for such

varied use cases as a testbed for a new device, for cre-

ating data sets, anomaly detection, and learning IoT

networking skills. Each device, collection of devices,

or controller should have the ability to be conﬁgured

on how it reacts to events. The ability to easily re-

conﬁgure virtual or simulated devices with or without

sensors allows users to avoid the costly acquisition of

many devices and instead have a combination of vir-

tual, inexpensive reusable simulated devices or com-

mercial IoT devices based on the what the researcher

or student has on hand or in their budget.

An important requirement for experimentation

and to test learned skills is to ensure reproducible

results with the system. The goal of reproducibility

entails having logging of events and activity, script-

ing to simulate events and activity, and the ability to

turn logged events and activity into scripts. The user

should have the option to write scripts for desired net-

work activity or to capture real-world generated ac-

tivity to play back. A combination should also be

supported. Allowing the system to convert log ﬁles

from actual activity allows for a physical way to gen-

erate scripts that match real-world events instead of

just constructed scenarios.

To observe and learn from the range activity, all

of the actions must be visible to the researcher or stu-

dent. The system requires the ability to log all events

that trigger system activity and to log the activity as

well. The logs need to be accessible and preferably

in a common format so that they can be combined for

analysis. The system should provide a facility to cap-

ture the activity from commercial IoT devices as those

devices may not provide access to logs or provide full

transparency of events and activities in their logs.

ConCERTS: An IoT Cybersecurity Research Range for Education, Experimentation, and Security Research

4 PROPOSED ARCHITECTURE

The following architecture has been proposed with

the goals of improving control, ﬂexibility, repro-

ducibility, and visibility over the existing systems

from Table 1. The intent is to provide an IoT range

that can be used for training, experimentation, and re-

search. The requirements for hardware and software

indicate that the system needs to intermingle virtual,

simulated, and commercial devices running IoT appli-

cations and to provide a system to manage the scenar-

ios to be set up and operated in the range. These re-

quirements are best served by the architecture by con-

sidering them as independent layers. The scenario,

application, and hardware layers as shown in Figure 1.

The scenario layer provides the control over the

conﬁguration of the IoT network, the scripting, and

gathering logs of the application activity. The de-

sign of this layer supports reproducibility using script-

ing and converting logs to scripts. The ﬂexibility re-

quirement is supported by allowing the user to con-

ﬁgure the devices, network, and application scenario.

The control is gained through delivering conﬁgura-

tions and scripts to devices and starting and stopping

scripted scenarios. And the visibility is handled in

this layer by gathering the common format logs from

each device for analysis as well as having the facility

to query each device for its state during script execu-

tion.

The scenario layer communicates with the Con-

CERTS devices in the application layer through the

use of a common REST API. There is a basic set of

API calls to conﬁgure the device, read the conﬁgura-

tion, read logs, load a script, run, and stop the script,

and query the current state of the device, sensors, and

actuators. The scenario layer can store all of this in-

formation in a database so that the scenario can eas-

ily be run multiple times. This scenario data can be

edited to reconﬁgure the scenario with new or mod-

iﬁed devices, and the events can be edited to change

the activity that is triggered.

The application layer is where the virtual, simu-

lated, and commercial IoT device’s applications op-

erate. Each device has support for generating logs,

running scripts, responding to events, and generating

activity. Simulated devices can be virtual or simu-

lated on hardware with real-world sensors. For a pure

virtual device the device is run entirely in software

and may be a virtual machine, container, or just run

on any type of device that will support the code base

and connectivity requirements. Virtual devices can

only be used for scripted scenarios. The scripts can

be custom crafted to provide a simulation of real-

world activity, or they can be used to re-run scripts

that have been generated from log ﬁles of real-world

activity. The simulated devices are intended to be

run on single-board computers (SBC) like a Rasp-

berry Pi. The value of a simulated device on an SBC

is the ability to connect the device to sensors, ac-

tuators, or dev kits. The sensors can be electronic

components that measure conditions like temperature,

light, or motion. The actuators are electronic com-

ponents that activate devices like door locks, elec-

trical motors, or lights. Commercial devices can be

included in the ConCERTS devices by wrapping the

device in a simulated device proxy using a man-in-

the-middle (MITM) scenario where the simulated de-

vice captures all activity from the commercial device,

logs it, and allows for scripting, replays, and modiﬁ-

cations of behaviour. In this manner, real-world ac-

tivity can be captured at the application layer to be

used for scripts at the scenario layer. The application

layer is where the device controllers are located. The

controllers provide decision making support to the de-

vices. These may be used as bridges between sensors

and actuators. For example, if the temperature in a

room goes below a predetermined level, the furnace

is activated, and if it goes above a certain threshold,

the air conditioning is activated. The controllers can

be conﬁgured from the scenario layer. In this exam-

ple the predetermined thresholds for heating and air

conditioning and what devices to send the command

to are part of the conﬁguration for the controller.

The hardware layer is composed of commercial

devices, commercial devices where the ﬁrmware or

software has been modiﬁed to support ConCERTS,

simulated devices that use a dev kit to customize ad-

vanced communication behaviour, and simulated de-

vices that read from sensors or activate actuators. Al-

lowing for hardware devices increases the ﬂexibility

of the ConCERTS architecture. ConCERTS can be

used for capturing real-world activity from devices

situated in a network with genuine interactions with

the physical environment and people, or it can be

setup to be triggered manually in a scripted scenario

where people trigger the sensors in a composed or im-

provised manner. A commercial device can be encap-

sulated by ConCERTS devices to provide a simula-

tion that acts as a device proxy for that one device

so that to the device the proxy appears as the real-

world environment it expects to communicate with.

The ﬁrmware on physical devices can be extracted

from a UART interface or by pulling the ﬁrmware

chip. The ﬁrmware can be modiﬁed or extended to

support ConCERTS REST API calls or to support a

MITM device proxy set up where a simulated Con-

CERTS device can capture all of the activity and al-

low the device to participate in ConCERTS scenarios.

ICISSP 2025 - 11th International Conference on Information Systems Security and Privacy

Figure 1: An architectural diagram of the ConCERTS components.

4.1 Architecture Components

This section identiﬁes the various architecture

components that exist within the layers as

shown in Figure 1. The source code is pub-

lished on GitHub (McKay and Singh, 2024)

https://github.com/CRLTeam/ConCERTS-device.

4.1.1 Scenario Manager

The scenario manager operates at the scenario layer

and is responsible for simplifying conﬁguration,

scripting, and communication for scenarios. Figure 1

identiﬁes the Settings and Scripts Control as well as

the Scenario Database. It is responsible for sending

and receiving REST API calls to and from various

controllers and devices on the network. These REST

API calls serve a variety of purposes.

Device and controller conﬁguration and manage-

ment are initially handled by the Settings Control of

the Scenario Manager. This component provides de-

vice and service controllers with the speciﬁc informa-

tion they need to set up nodes on the network. This

information includes application code conﬁguration,

sensor conﬁguration, logging instructions, and other

modules executed by controllers.

The event Scripts Controller is responsible for the

unique level of consistency and reproducibility that

ConCERTS offers. It communicates with the various

devices and controllers in the network and orches-

trates events according to its predetermined scripts.

It loads the conﬁguration and scripting information

from the scenario database allowing scenario design-

ers to easily provision consistent scenarios.

4.1.2 Scenario Analysis

The Elastic Search server is responsible for conﬁgur-

ing Elastic agent policies, querying device logs, and

aggregating data. Elastic can also be conﬁgured to

perform intrusion detection and other SIEM functions

beyond simple data collection. The datastore is a

PostgreSQL database that integrates log and activity

data collected from Elastic agents, conﬁguration data

and scripts from the scenario manager, and other mis-

cellaneous data the user wishes to collect outside of

Elastic.

4.1.3 Controller Simulator

The controller simulator facilitates the development

of simple controllers that can support the basic logic

to respond to events from devices, make a decision

on that event, and send out events to other devices.

In one of the demonstration scenarios in this paper,

the controller simulates a building’s HVAC system. It

takes inputs from a thermostat and turns on or off fan,

furnace, and AC unit devices.

ConCERTS: An IoT Cybersecurity Research Range for Education, Experimentation, and Security Research

Figure 2: Replay attack device conﬁguration.

4.1.4 Device Simulator

The device simulator is a software application that

simulates a simple IoT device in a very small amount

of plain code. It can respond to events from a REST

call, an event generated in a simulation script, or from

a sensor. This makes the device simulator a very sim-

ple application to understand and to modify. The de-

vice contains the scenario agent that holds the REST

calls for the scenario manager to call it, and it has an

Elastic agent so that the Elastic server can contact it

and retrieve the event logs.

4.1.5 Device Proxy

A commercial device can be encapsulated by a device

proxy so that it can participate with the application

and scenario layers of ConCERTS. The proxy catches

the outgoing signals and can redirect them through the

device just like activity from a sensor. Depending on

the abilities of a device, it can be passive or it can be

directed to trigger events from the scripting agent.

4.2 Architecture Demonstration:

Replay Attack

A replay attack on an IoT device is executed by cap-

turing commands that were sent to an IoT device and

playing them back maliciously. Using this type of

attack provides a simple demonstration of running a

security breach within an Enterprise IoT range envi-

ronment. The demonstration setup uses several Con-

CERTS components as depicted in Figure 2. The Sce-

nario Manager provides the conﬁguration of the Card

Reader and Door Lock devices and the Card Man-

ager controller. It deﬁnes the behaviour of each de-

vice and identiﬁes them to the controller. The devices

each run the Device Simulator. The Card Reader was

run on a Raspberry Pi and the Door Lock on a Win-

dows desktop to demonstrate ﬂexibility in the options

for device platforms. The Card Manager controller

was run in a Docker container on a Linux server as a

further demonstration of options. The Wireshark tool

was used to capture the network activity and the re-

play attack was executed using a curl command.

The demonstration makes good use of the Sce-

nario and Application layers. This simulates the ac-

tivity that may occur at a typical ofﬁce front door.

When a card is read at the Card Reader device, the

card’s identity number is passed on to the Card Man-

ager controller. The controller checks its database to

see if the identity number is allowed to unlock the

door. If it is, the Card Manager sends a command

to the Door Lock device to unlock the door for a set

period of time to allow someone to enter. The card

reading events are written as a script that holds the

card reading event information. Each event has the

card identity number and an amount of time to wait

until the running the next event. This can be used to

simulate the cards being presented to the card reader

over the course of time. The script triggers the card

reader to send the card number to the controller that

can then command the door lock to open. Only the

card reader device needs to run a script in this sce-

nario as the events trigger the controller and the lock

device.

In this scenario, the command packets are unen-

crypted. A separate attacker node was conﬁgured

on the network using a device operating with a Kali

Linux distribution and Wireshark installed. As a train-

ing scenario, this is where the trainee would begin

the attack. The attacker uses Wireshark to capture

network trafﬁc activity. The attacker can look for

network packets that contain an HTTP POST. This

ICISSP 2025 - 11th International Conference on Information Systems Security and Privacy

Figure 3: Virtual, simulated with sensors and actuators, and real-world device demonstration.

is a way to spot REST API calls used to commu-

nicate with and perform commands related to rele-

vant IoT devices. If the attacker is unsure how the

command will present itself, they can simply ﬁlter by

source or destination device. The attacker then opens

a detailed view of the individual packet and can view

the full request Uniform Resource Identiﬁer (URI).

The attacker now has the URI, including the port re-

quired. Finally, the attacker can view the data and

variables portion of the command and reconstruct the

command. The attacker can then construct and for-

ward the data to the relevant device using a curl com-

mand. This command allows the attacker to bypass

the card reader device and card manager controller

by sending a reconstructed open command directly

to the door lock device. All events are logged lo-

cally on each device and controller, and upon scenario

completion they are aggregated in a central scenario

database where they can be reviewed and analyzed.

This demonstration was set up in four steps. In

the ﬁrst step, the card reader, door lock, and card

card manager were all installed. The card reader

and door lock required installing node.js and cloning

the Github repository onto the Raspberry PI and the

desktop computer, then they were ready to run. The

card manager was installed by running an image of

a docker container of the Github repository on a vir-

tual machine running Linux under the ProxMox vir-

tual environment. For the second step, each device

was conﬁgured by making REST calls to the device’s

IP address using curl commands. The conﬁguration

information contained settings for the card reader to

access the card manager, the card manager to access

the door lock, and the valid card numbers that will

unlock the door. The third step was to create a ba-

sic script. For this scenario, the script is a JSON data

structure that has an array of card numbers, and how

long to wait until the next card swipe. An example

would be:

{

script: [

{card:1234, wait:120},

{card:2227, wait:500}

]

}

This would scan card 1234 then wait 2 minutes

and scan card 2227. The script is sent to the card

reader as JSON data with a REST call using a curl

command. The ﬁnal step was to use another REST

API curl command to start the script. Now the re-

searcher can execute the script as many times as they

like to gather the data that they need. This scenario

can be set up and run in less than an hour by a stu-

dent that has basic knowledge in the Linux CLI, and

running applications. It is simpler if all of the virtual

devices are run in the same way - either all on one

computer or on multiple Raspberry Pi devices.

4.3 Architecture Demonstration:

Hybrid Architecture

The second demonstration was developed to explore

the versatility of the ConCERTS hardware layer by

connecting real world sensors and actuators to the

simulated devices, and running them in combination

with a commercial IoT device that is providing the

inputs to a controller. This scenario models a Heat-

ConCERTS: An IoT Cybersecurity Research Range for Education, Experimentation, and Security Research

ing, Ventilation, and Air Conditioning (HVAC) sys-

tem that would commonly be found in an ofﬁce build-

ing as depicted in Figure 3. In this demonstration sce-

nario, as the temperature rises on the thermostat above

a set value, the thermostat triggers the controller to

turn off the heating, and when the temperature rises

even higher, turn on the AC. When the temperature

drops below a certain value, the controller tells the

AC to turn off and as it goes lower it triggers the

heater to turn on. To build this demonstration sce-

nario a commercial thermostat IoT device from Hon-

eywell was used to connect to a simulated HVAC con-

troller. The simulated HVAC controller was conﬁg-

ured to trigger events for heating and cooling by send-

ing commands to the AC and heating devices. The

heating device uses a physical inductive heating coil

to heat a piece of metal positioned near the thermo-

stat. The air conditioning device uses a physical fan

to blow room temperature air across the piece of metal

to cool it down. The trigger events are stored in the

logs of each device and the logs are pulled by the sce-

nario manager. The scenario manager can convert the

logs to scripts to allow the system to replay the exact

events as they happened with or without the hardware

connected. This example demonstrates ConCERTS

devices using physical actuators and complementing

commercial IoT hardware to capture real-world event

data from devices and using that data to replicate the

exact scenario as many times as required using Con-

CERTS scripting simulations.

5 DISCUSSION

The driving goals of improved control, ﬂexibility, re-

producibility, and visibility has produced a highly

ﬂexible tool that can simulate or compliment an IoT

cybersecurity research range. The review of exist-

ing approaches revealed that many existing ranges

and testbeds required technical expertise to use in a

meaningful capacity. ConCERTS aims to remedy this

with its unique architecture and software stack that

relies on high-level programming languages and uni-

form communication formatting. It should be noted

that ConCERTS does not presently intend to per-

form hardware or ﬁrmware level testing and that these

low-level details are hidden to enhance usability. In-

stead, ConCERTS focuses on application, network,

and service-level testing and education. The goal for

ConCERTS is to engage the IoT cybersecurity com-

munity to add more simulated devices, to create data

sets along with the scripts to reproduce them, and to

create workshops around setting up and running Con-

CERTS scenarios. To keep this open, ConCERTS has

been released as an open source project under the per-

missive MIT license.

In opting for open source software and conﬁg-

urable hardware, ConCERTS avoids the problem of

proprietary applications obfuscating valuable test data

and reducing user control. Proprietary software may

communicate with cloud services or require several

interactions with infrastructure that is difﬁcult to

recreate without a tool like the ConCERTS Device

Proxy. Scripted events provide a degree of abstraction

that allows ConCERTS to simulate complex scenar-

ios without necessarily recreating the entire environ-

ment being tested. In general, event-based simulation

makes it much easier to consistently recreate scenar-

ios for speciﬁc areas of research and education.

In working with students, we have anecdotally

shown that in two hours they have the ability to add

new device types to the code base and set up hybrid

devices that use sensors. The demonstration the stu-

dents worked on was a motion detector connected to a

lighting controller that drove a light. The motion de-

tector was a simple proximity sensor and the light was

just an LED with both connected to Raspberry Pis,

but with ConCERTS this can be used to capture real-

world activity, and turn those logs into scripts. By just

waving their hands in front of the proximity detector,

the students were able to turn the activity logs into

scripts, allow for adjustments of those scripts, and re-

play the scripts to watch the commands turn the LED

light on and off just like when it was manually acti-

vated.

Consistent communication methods between de-

vices and controllers and event-based scripting en-

ables a high level of visibility into the scenarios. The

controller passes on events to the application-level

logic with logs being collected at both levels. This

architecture allows a user to aggregate network, appli-

cation, and event-level data. This means ConCERTS

can reliably associate speciﬁc events and states that

are normally decoupled in a traditional testing envi-

ronment. For example, cloud applications are usually

hidden from the user and cannot be monitored as part

of a holistic testing environment. With this ability to

view events on the service controller that simulates

cloud applications, ConCERTS can gain a more com-

prehensive view of IoT security and see how actions

at various levels affect the entire system.

The demonstration scenarios show how this archi-

tecture can be easily used to set up, conﬁgure, and

run cybersecurity training scenarios. The replay at-

tack scenario only required conﬁguring two devices

with simple applications for reading cards and un-

locking doors and providing access to an operating

system preloaded with security tools. Conﬁguration

ICISSP 2025 - 11th International Conference on Information Systems Security and Privacy

was as simple as sending predeﬁned REST API calls

to the relevant devices and following instructions for

how to perform the attack. It is easy to imagine that

further development of IoT applications and services,

coupled with a UI, would allow users to simply pro-

vision a number of educational scenarios.

6 FUTURE WORK

Future work on ConCERTS will consist of engag-

ing the cybersecurity community to develop more

devices, controllers, and features. Development ef-

forts will look to further develop integrated log-

ging capabilities that support analysis. Associating

both encrypted and unencrypted network packets with

event-level logs would provide the ability to compare

application-level behaviour with network behaviour.

Additionally, to enhance ease of use, ConCERTS will

be looking to utilize Ansible scripts to conﬁgure set-

tings that cannot be abstracted at the controller level.

This would allow for the installation of additional

software outside of the device simulator, perform re-

mote updates, and conﬁgure network settings for sev-

eral devices at the same time. Another future addition

would be to support the extraction and modiﬁcation of

ﬁrmware from a device to be run on an emulator such

as QEMU as part of the device simulation software.

This would allow for commercial devices to partic-

ipate in the ConCERTS network while allowing for

testing low-level vulnerabilities. Finally, the inten-

tion is to expand the set of example demonstrations

to show the full capacity that ConCERTS brings to

educational, research, and experimental applications.

7 CONCLUSION

The architectural requirements of ConCERTS was

driven by a review of existing approaches and the

desire to improve control, ﬂexibility, reproducibility,

and visibility. The architecture removes the complex-

ity of studying IoT networks, applications, and ser-

vices by abstracting away much of the lower-level

conﬁguration. As both a physical range and a soft-

ware stack, it is hoped that ConCERTS can not only

be used to further research goals for the authors of

this paper, but that others will be able to provision

their own networks for testing and educational pur-

poses. The range has the unique advantage of hav-

ing a low entry cost, a focus on usability, and a novel

degree of control over the whole network and sce-

nario. In simulating real-world applications and use

cases, the range allows for the necessary applica-

tions, datastores, and sensors to represent real devices

such as security card door locks, HVAC systems and

more. Event-based scripting allows for effortless re-

producibility and a unique ability to associate and ag-

gregate logging data. ConCERTS offers researchers

the opportunity to make, hack, and break devices and

IoT systems with full control over conﬁguration and

data. As IoT security research presses onward, it is

crucial that tools like ConCERTS provide researchers

the right tools for the job.

ACKNOWLEDGEMENTS

The authors express their sincere gratitude to the

anonymous reviewers for their insightful and detailed

feedback. This research was supported in part by the

Natural Sciences and Engineering Research Council

(NSERC RGPIN-2019-06150] of Canada, the Canada

Research Chairs program (CRC-2021-09-01), the On-

tario Ministry of Colleges and Universities Small In-

frastructure Fund (SIF 40704), and the Canada Foun-

dation for Innovation’s John R. Evans Leaders Fund

(CFI JELF 40704).

REFERENCES

Abu Waraga, O., Bettayeb, M., Nasir, Q., and Abu Talib, M.

(2020). Design and implementation of automated iot

security testbed. Computers & Security, 88:101648.

Adjih, C., Baccelli, E., Fleury, E., Harter, G., Mitton,

N., Noel, T., Pissard-Gibollet, R., Saint-Marcel, F.,

Schreiner, G., Vandaele, J., and Watteyne, T. (2015).

Fit iot-lab: A large scale open experimental iot

testbed. In 2015 IEEE 2nd World Forum on Internet

of Things (WF-IoT), pages 459–464.

Albataineh, A. and Alsmadi, I. (2019). Iot and the risk

of internet exposure: Risk assessment using shodan

queries. In 2019 IEEE 20th International Symposium

on ”A World of Wireless, Mobile and Multimedia Net-

works” (WoWMoM), pages 1–5.

De Donno, M., Dragoni, N., Giaretta, A., and Spognardi,

A. (2018). Ddos-capable iot malwares: Comparative

analysis and mirai investigation. Security and Com-

munication Networks, 2018:1–30.

Ficco, M. and Palmieri, F. (2019). Leaf: An open-source

cybersecurity training platform for realistic edge-iot

scenarios. Journal of Systems Architecture, 97:107–

129.

Franceschi-Bicchierai, L. (2016). Hackers make the ﬁrst-

ever ransomware for smart thermostats.

Hale, M., Lotfy, K., Gamble, R., Walter, C., and Lin, J.

(2018). Developing a platform to evaluate and assess

the security of wearable devices. Digital Communica-

tions and Networks, 5.

ConCERTS: An IoT Cybersecurity Research Range for Education, Experimentation, and Security Research

Hamad, S. A., Zhang, W. E., Sheng, Q. Z., and Nepal,

S. (2019). Iot device identiﬁcation via network-ﬂow

based ﬁngerprinting and learning.

Han, S. N., Lee, G. M., Crespi, N., Heo, K., Van Luong, N.,

Brut, M., and Gatellier, P. (2014). Dpwsim: A simu-

lation toolkit for iot applications using devices proﬁle

for web services. In 2014 IEEE World Forum on In-

ternet of Things (WF-IoT), pages 544–547.

Jones, T., Dali, A., Rao, M. R., Biradar, N., Madassery,

J., and Liu, K. (2018). Towards a layered and secure

internet-of-things testbed via hybrid mesh. In 2018

IEEE International Congress on Internet of Things

(ICIOT), pages 17–24. IEEE.

Khoury, R., Vignau, B., Hall

e, S., Hamou-Lhadj, A., and

Razgallah, A. (2021). An analysis of the use of cves

by iot malware.

Kolias, C., Stavrou, A., Voas, J., Bojanova, I., and Kuhn, R.

(2016). Learning internet-of-things security ”hands-

on”. IEEE Security Privacy, 14(1):37–46.

Lodge, D. (2016). Steal your wi-ﬁ key from your doorbell?

iot wtf!

Maxxz, J. (2016). Backdooring the frontdoor hacking a

”perfectly secure” smart lock.

McKay, D. and Singh, A. (2024). Concerts-device github

repository.

Mehdi, K., Lounis, M., Bounceur, A., and Kechadi, T.

(2014). Cupcarbon: A multi-agent and discrete event

wireless sensor network design and simulation tool.

Modarresi, A. and Symons, J. (2020). Technological hetero-

geneity and path diversity in smart home resilience: A

simulation approach.

noz, J., Rincon, F., Chang, T., Vilajosana, X., Ver-

meulen, B., Walcarius, T., van de Meerssche, W., and

Watteyne, T. (2019). Opentestbed: Poor man’s iot

testbed. In IEEE INFOCOM 2019 - IEEE Confer-

ence on Computer Communications Workshops (IN-

FOCOM WKSHPS), pages 467–471.

Nock, O., Starkey, J., and Angelopoulos, C. M. (2020). Ad-

dressing the security gap in iot: Towards an iot cyber

range. Sensors, 20(18).

Osterlind, F., Dunkels, A., Eriksson, J., Finne, N., and

Voigt, T. (2006). Cross-level sensor network simu-

lation with cooja. In Proceedings. 2006 31st IEEE

Conference on Local Computer Networks, pages 641–

648.

Schwaiger, P., Simopoulos, D., and Wolf, A. (2022). Au-

tomated iot security testing with seclab. In NOMS

2022-2022 IEEE/IFIP Network Operations and Man-

agement Symposium, pages 1–6. IEEE.

Sharif, M. and Sadeghi-Niaraki, A. (2017). Ubiquitous sen-

sor network simulation and emulation environments:

A survey. Journal of Network and Computer Applica-

tions, 93:150–181.

Shariﬁ, A. Z., Vanijja, V., Pal, D., and Anantasabkit, W.

(2021). Cyberiot: An initial conceptualization of a

web-based cyber range for iot. In 2021 International

Conference on Computational Performance Evalua-

tion (ComPE), pages 091–096. IEEE.

Siboni, S., Sachidananda, V., Meidan, Y., Bohadana, M.,

Mathov, Y., Bhairav, S., Shabtai, A., and Elovici, Y.

(2019). Security testbed for internet-of-things devices.

IEEE transactions on reliability, 68(1):23–44.

Sotiriadis, S., Bessis, N., Asimakopoulou, E., and

Mustafee, N. (2014). Towards simulating the internet

of things. In 2014 28th International Conference on

Advanced Information Networking and Applications

Workshops, pages 444–448.

anchez, L., Mu

noz, L., Galache, J., Sotres, P., Santana,

J., Gutierrez, V., Ramdhany, R., Gluhak, A., Krco, S.,

Theodoridis, E., and Pﬁsterer, D. (2013). Smartsan-

tander: Iot experimentation over a smart city testbed.

Computer Networks.

Tawalbeh, L., Muheidat, F., Tawalbeh, M., and Quwaider,

M. (2020). IoT privacy and security: Challenges and

solutions. 10(12):4102.

Teranishi, Y., Saito, Y., Murono, S., and Nishinaga, N.

(2015). Jose: An open testbed for ﬁeld trials of large-

scale iot services. 62:151–159.

Thom, J., Das, T., Shrestha, B., Sengupta, S., and Arslan,

E. (2021). Casting a wide net: An internet of things

testbed for cybersecurity education and research. In

2021 International Symposium on Performance Eval-

uation of Computer and Telecommunication Systems

(SPECTS), pages 1–8. IEEE.

Xenofontos, C., Zografopoulos, I., Konstantinou, C., Jol-

faei, A., Khan, M. K., and Choo, K.-K. R. (2022).

Consumer, commercial, and industrial iot (in)security:

Attack taxonomy and case studies. IEEE Internet of

Things Journal, 9(1):199–221.

Zeng, X., Garg, S. K., Strazdins, P., Jayaraman, P. P., Geor-

gakopoulos, D., and Ranjan, R. (2017). Iotsim: A

simulator for analysing iot applications. Journal of

Systems Architecture, 72:93–107. Design Automation

for Embedded Ubiquitous Computing Systems.

ICISSP 2025 - 11th International Conference on Information Systems Security and Privacy