![](bg2.png)
others focus on accuracy (e.g. inspections). An appli-
cation should therefore be able to quickly implement
new detection and analysis features to meet these re-
quirements. Due to the lack of modifiability present
in current software solutions, applications built on
top of aerial thermal imaging remain largely niche
and vendor/equipment-locked (Divya, 2017). Appli-
cations would benefit from a software framework fo-
cused on modifiability and interoperability, to aid in
developing technology-agnostic end-to-end solutions
connecting thermal cameras to different image analy-
sis / detection modules.
This paper presents the requirements for such a
framework and proposes a suitable software architec-
ture. To test the viability of the architecture, a proof-
of-concept prototype is implemented and evaluated
against the initial requirements. As a use case, de-
tection of large crowds (so-called mobs) during open-
air events is investigated. Monitoring crowds dur-
ing open-air events is important, as mobs can cre-
ate potentially dangerous situations through bottle-
necks, blocking escape routes, etc. Detection of mobs
can also be used for rapid identification of fights
and brawls. Through monitoring and detecting these
mobs, these situations can be identified before they
become problematic (Steffen and Seyfried, 2010).
The remainder of this paper is organized as fol-
lows. Section 2 presents similar projects on the topic
of modifiable image analysis frameworks and thermal
object detection. Section 3 presents the requirements
of the framework and the software architecture de-
signed from these requirements. Section 4 presents
the implementation of the framework prototype. The
mob detection experiment is described in Section 5.
Evaluation of the framework and results of the mob
detection experiment are presented in Section 6. Fi-
nally, Section 7 draws conclusions from this research
and indicates where future efforts in this field should
go to.
2 RELATED WORK
The Irish start-up DroneSAR (Slattery, 2017) devel-
oped a search-and-rescue (SAR) drone platform al-
lowing users to stream live images and video from a
drone as it conducts a search for missing persons. The
platform works with any camera, visual and thermal
but focuses on drones from vendor DJI, DroneSAR’s
industry partner. The solution proposed in this paper
is not restricted to only DJI drones and can be em-
ployed for different use-cases.
Amazon introduced the Amazon Kinesis
VideoStreams platform in January 2018 as a
new service for the Amazon Web Services (AWS)
cloud platform. It allows users to stream live video
from devices to the AWS cloud and build applications
for real-time video processing (Inc., 2018). While the
Amazon Kinesis VideoStreams platform certainly has
interesting features, it imposes a vendor lock-in as it
relies solely on Amazon Cloud services, whereas the
open-source solution proposed here can be deployed
on a variety of (cloud) resources and it can even be
deployed locally when there is no up-link to the cloud
available.
The VIPER project by EAVISE and KU Leu-
ven investigated how thermal and video images could
be used for real-time detection of persons using
object detection algorithms based on deep learn-
ing (Goedeme, 2017). Our work evaluated a mob de-
tection experiment based on the YOLOv3 (Redmon
and Farhadi, 2018) object detection algorithm, using
a deep neural network. In previous work, a drone
was used to detect objects in a predefined incident
area such as barrels containing explosive fluid, fires,
firefighters, etc. (Tijtgat et al., 2017a; Tijtgat et al.,
2017b). The framework proposed in this paper offers
an abstraction allowing developers to connect micro-
service based flows in terms of input (e.g. thermal
video, regular video, images), processing (e.g. object
detection, classification, localization) and output (e.g.
REST API, custom applications).
3 REQUIREMENTS AND
SOFTWARE ARCHITECTURE
3.1 Function Requirements
Three general actors are identified for the framework:
an end-user that wants to build an image process-
ing application for a specific use case, input sen-
sor developers integrating new cameras / streams into
the framework, and stream processing module devel-
opers integrating new analysis/detection algorithms
(e.g. analysis, detection, localization, decision sup-
port) into the framework so that end-users can use
them to build their applications. An end-user should
be able to construct an image processing application
by interconnecting predefined camera and analysis
modules. Hence-built applications should be adapt-
able to new use cases by adding or replacing select
modules, retaining the overall functionality of the ap-
plication. Camera and analysis module developers
should be able to add and distribute new camera in-
tegration modules (e.g. new type of thermal camera
with support for per-pixel labeling of temperatures)
Pluggable Drone Imaging Analysis Framework for Mob Detection during Open-air Events
65