Smart Device Prototype for Automated Emergency Calls

Using SIP and TTS to Reach Legacy Emergency Services

Mihai Buf

, Marco Manso

and Barbara Guerra

Orange Romania, 47-53 Lascar Catargiu Blvd., Bucharest, Romania

Rinicom, Morecambe Road, Riverway House, Lancaster, U.K.

Keywords: Internet of Things, Smart Devices, Prototype, Telematics, Text-to-Speech, Emergencies, Emergency

Services.

Abstract: The Internet-of-Things (IoT) promises to transform our society into smart environments, incorporating

smart objects that cooperate to fulfil specific goals. Amongst its many applications, emergencies can also

benefit from IoT principles and use of automation for a better emergency response and reducing the number

of fatalities. Smart devices can be used to detect emergency events (e.g., fire, presence of hazardous gases).

In this paper, we present a prototype applying the IoT paradigm to the concept of automated calls. The

prototype is capable to measure environmental parameters - such as smoke, temperature and gas - to

determine the occurrence of a serious incident (e.g., fire in room) and automatically initiate an emergency

call. To make our approach interoperable with most platforms and operational practices (including

emergency services that mostly rely on voice calls), the system generates an audio call using preformatted

messages and a text-to-speech engine. Our approach brings the benefits of automated calls without requiring

significant investments to existing infrastructures (including those used by emergency services).

1 INTRODUCTION

The Internet-of-Things (IoT) promises to transform

our society into smart environments,

environmentally aware and well informed about its

wellbeing, interests and security. IoT refers to the

extension of the Internet paradigm to the world of

objects and places, each able to communicate their

own data and access aggregated information from

other objects and places. Building blocks of the

future IoT, smart objects cooperate to fulfil specific

goals (Fortino et al., 2013) finding applications

across many societal domains, including safety and

security of citizens

Their presence is becoming ubiquitous and

global: Gartner forecasted that 8.4 billion connected

things will be in use worldwide by 2017 and will

reach 20.4 billion by 2020 (Gartner, 2016).

The emergency services sector is not immune to

the penetration of IoT concepts and applications.

Emergency Services (ES) deal with urgent life

threatening situations that require a swift response.

While they mainly rely on voice calls (via the 112

number for most of Europe), initiatives like eCall,

part of the eSafety initiative of the European

Commission (European Commission, 2014),

represents a most notable and successful effort in

bringing automated calls and data exchange with ES

into reality.

The eCall is applied to the specific event of a car

accident: if the car's sensors detect a crash, the car

automatically calls the nearest emergency centre.

Even if no passenger is able to speak, e.g. due to

injuries, a 'Minimum Set of Data' is sent, which

includes the exact location of the crash site, the time

of incident, the accurate position of the crashed

vehicle and the direction of travel (European

Commission, 2014). The eCall concept was

designed having in mind circuit switched emergency

calls, specifically working in 2G and 3G networks

using an in-band modem (3GPP, 2017). Despite, it is

expected that eCall cuts emergency services

response time by up to 50% in the countryside and

40% in urban areas. It is estimated that eCall can

reduce the number of fatalities by at least 4% and

the number of severe injuries by 6% (European

Commission, 2017).

Further applications can be deployed to the

benefit of citizens. As proposed in (Manso et al.,

2017), these include eHealth, Smart Building

Buf, M., Manso, M. and Guerra, B.

Smart Device Prototype for Automated Emergency Calls - Using SIP and TTS to Reach Legacy Emergency Services.

DOI: 10.5220/0006631401150121

In Proceedings of the 7th International Conference on Sensor Networks (SENSORNETS 2018), pages 115-121

ISBN: 978-989-758-284-4

115

applications and the next generation eCall that fully

exploit IoT concepts, IP and packet switch networks.

Despite novel approaches for ES, such as those

taken by the EU funded action NEXES (Next

Generation Emergency Systems) (NEXES, 2015),

which embrace IP technologies and data exchange,

ES worldwide mainly rely on circuit switched voice

calls.

This paper extends the work presented in (Buf et

al., 2017) by describing the implementation and

demonstration of a smart device conveying

environmental data relevant to determine the

occurrence of emergency incidents (related with

fires or combustible gas leaks). In such a case, the

system automatically initiates an emergency call and

conveys a message in audio format, thus being able

to operate with any ES deployed nowadays.

The remainder of the paper is structured as

follows: it starts by presenting two cases where

automated emergency calls can be used to improve

safety of citizens and property; then the system

concept and architecture is described, including the

implementation of the smart device system; the

"Workflow" section presents the sequencing

between the relevant steps, followed by the structure

of the emergency messages; in the "Demonstration"

section we present two examples of the detection of

emergency events that initiate an emergency call; the

paper finalises with the conclusion section.

2 THE NEED FOR AUTOMATED

EMERGENCY CALLS

As part of the NEXES Action, several use-cases

were described involving the use of automated alerts

to enable a prompt response to an emergency and

save lives (Manso et al., 2017). In one scenario, a

tank at a small chemical plant develops a breach and

hazardous materials began leaking out. A security

guard, responsible to monitor incidents, failing to

follow the appropriate security precautions, is

quickly taken ill upon inhalation of noxious fumes

and loses consciousness. Since the chemical plant

uses smart devices, the leak is detected and an

automated alert is initiated. Because no human

intervention is possible (the guard is unconscious), a

call to emergency services is automatically

established. The emergency call taker develops

awareness of the emergency situation and dispatches

relevant personnel and resources, including

specialist containment and decontamination crews

and personal protective equipment.

This futuristic scenario highlights the potentially

vital role of telematics in specific emergency

scenarios. The use of telematics and automated

calling means that in situations in which citizens are

incapacitated, emergency services can still be timely

contacted and properly informed about the incident,

thus enhance the emergency response in terms of

response time, the safety of the citizen and, in this

situation, the safety of first responders as well.

Looking into incidents at homes, we see these

are a major cause of injuries and deaths: according

to the U.S. National Fire Protection Association

(National Fire Protection Association, 2017), in

2015 there were more than 360k house fires causing

more than 2500 deaths, 11k injuries and several

billions $USD in damage. 3 out of 5 home deaths

occurred in houses without smoke alarms. The

utilisation of smart devices and alert systems could

have a significant high positive impact in improving

human safety and reducing financial losses.

3 SYSTEM CONCEPT AND

DESCRIPTION

The concept described in this paper consists in

deploying IoT systems to perform sensing and

incident detection together with components capable

to trigger specific actions - specifically initiate an

emergency call - and convert digital messages into

audio in a form that can be received and understood

by any ES nowadays.

Smart devices are deployed in a cluster

configuration connected to a Hub (within a LAN or

via a router). The smart devices incorporate sensors

capable to measure physical quantities of interest

and detecting key events (e.g., gas concentration and

smoke detection). The devices are considered to be

resource constrained (i.e., limited processing

capabilities, low power consumption) thus they

perform limited functionalities. They are connected

to a Gateway/Hub that collects all sensor data and

perform computational intensive functions. In this

regard, the Hub has the capability to perform data

fusion and correlation to increase detection

reliability (e.g., correlate fire and smoke sensor data

to confirm a relevant event), generate event

messages into audio (using its TTS engine) and, for

our specific use-case, initiate emergency calls using

the public network (other approaches could instead

opt for calling the owner phone or a service

operator).

An overview of the system is depicted in Figure

1 and is described next.

SENSORNETS 2018 - 7th International Conference on Sensor Networks

116

Figure 1: System Overview.

3.1 Hub System Implementation

The Hub main components are described next.

• MQTT Server

The MQTT Server manages publishers, subscribers

and data exchange. It is responsible for message

management, based on the publish/subscribe

paradigm, disseminating messages to subscribers

when these are published.

In our configuration, each sensor (i.e., publisher)

has a specific topic to which it publishes messages

(i.e., sensor data or alert). Via MQTT, the Hub

allows authorised users to subscribe to sensor

messages to e.g., monitor the area and be notified in

case of emergencies. Internally, the "Logic and

Processing", described later, also subscribes to

sensor messages.

• SIP Client

The Hub includes a SIP client in order to be able to

initiate SIP calls in case emergencies are detected.

The SIP Client performs all required steps for setup

(e.g., registration) and manage calls (e.g., initiation,

establishment and termination).

• TTS

The TTS engine allows to convert text messages to

audio. It requires significant processing and storage

capabilities (typically above 1GHz processor and

several Mbytes of storage for audio voice files)

typically not present in sensor devices.

• Web-server

The Web-server allows the remote configuration and

management of the Hub (and the system). It allows

the user or administrator perform functions like

system configuration (e.g., security, setup users,

setup sensors), generate reports, configure recipients

of alerts, "live" monitor the system, etc. It also

allows to setup emergency events (e.g., fire),

workflows and associated messages (see Sections 4

and 5).

• Logic and Processing

The "Logic and Processing" is a central element of

the Hub. It receives sensor data (via MQTT) and

processes it to determine the occurrence of an

emergency event and, if so, triggers related actions

such the automatic initiation of an emergency call.

The implemented interfaces and protocols are listed

below.

• All-over-IP

The Internet Protocol (IP) is chosen as common

protocol between all devices and components within

the system. IP has become a dominant technology

worldwide successfully linking billions of devices

over the Internet and within private networks. IP is

used to interconnect smart devices, the Hub, web-

clients and initiation of calls (using SIP).

• Sensor Data Exchange Middleware: MQTT

The Message Queue Telemetry Transport (MQTT)

is a message broker middleware based on the

publish-subscribe paradigm standardised as ISO/IEC

PRF 20922 (International Organization for

Standardization, 2016). It has become very popular

in the IoT community given its lightweight Client

Server publish-subscribe messaging transport

protocol, making it fit for resource-constrained

devices.

Smart devices share information using MQTT,

by publishing information to specific topics. The

Hub receives data from sensors by subscribing to

those topics.

• Management Interface: HTTP

The HUB system management is performed over the

Hypertext Transfer Protocol (HTTP) supported by

most web browsers.

VoIP-PBX

Gateway

Cluster 2

Cluster 1

Legacy

PSAP

WEB

Interface

Gateway / Hub

R outer

sensor

S1.2

MQTT

Server

TT S

WEB

Server

SIPclient

Internet

sensor

S1.1

sensor

S2.1

sensor

S2.2

LAN

SIP

HTTP

Logicand

Processing

MQTT

messages

Voicecall

Smart Device Prototype for Automated Emergency Calls - Using SIP and TTS to Reach Legacy Emergency Services

117

• Emergency Calls: SIP, TTS and PBX

Gateway

Our concept involves the automated establishment

of an emergency call to emergency services

(specifically, a Public Answering Safety Point

(PSAP)) in case of an emergency. For this purpose,

the Hub supports the SIP protocol, which is widely

used for IP telephony all over the world.

Additionally, to "reach" the legacy public switched

telephone network (PSTN), a VoIP-PBX gateway is

added to convert SIP signalling and VoIP to a form

supported by PSTN. To generate audio, the Hub

uses a TTS engine.

3.2 Smart Device Implementation

The implemented smart device is depicted in Figure

2. The left picture shows the device casing and

sensors used. The picture at the right shows the

connections between the several components. The

components that constitute the smart device are the

following:

• Flame Sensor (Keyes KY-026 model) used to

detect the presence of fire;

• Temperature and Humidity Sensor (DHT-11

model) used to measure room temperature and

humidity (which may provide indication of

increased temperature and likelihood of a fire);

• Gas Sensor (MQ-2 model) used to detect the

presence of combustible gas, such as Liquefied

Petroleum Gas (LPG), Propane and Hydrogen.

• Arduino UNO Microcontroller Board used to

manage, control and read data from sensors. It

also runs a lightweight MQTT client. A Wi-Fi

shield was added to allow wireless connectivity

with the Hub (described in 3.1).

• Power Supply unit (9v).

The smart device handles only "lightweight" tasks,

such reading sensors data and periodically

publishing them as MQTT messages to the MQTT

server hosted in the Hub. The system workflow is

described next.

4 WORKFLOW

The workflow related with our implementation of

automated calls based on smart devices is depicted

in Figure 3. The process can be split into two parts:

the first one related with the smart device and a

second one related with the Hub.

The workflow starts with the initialisation

procedure where the system setup and configuration

is performed, including establishment of MQTT

topics (to where sensors publish messages), SIP

client registration, and the fulfilment of required

authentication steps.

After initialisation, the system enters a

continuous loop. In it, each smart device measures

their parameters of interest (for example,

temperature, gas concentration or smoke detection)

and publish the value using MQTT according to the

defined criteria (e.g., raw measurement,

measurement when above a certain threshold value

or event) and to the appropriate topic.

The Hub reads the sensors' values, by receiving

MQTT messages published in topics it subscribed.

The Hub then processes received messages to

determine if there is an emergency situation (when

out of "value safe side"), which can be based on

direct sensor data (e.g., a smoke sensor produces a

Figure 2: Smart Device Prototype Implementation.

MQ2

Gas

Sensor

DHT11

Temperature

Humidity

KY-026

Flame

Sensor

MQ-2

Gas

Sensor

DHT11

Tempera ture

Humidity

KY-026

Flame

Sensor

Power

supply

9V/1A

ArduinoUNOR3+

WiFi Shield +

ProtoShield

WiFi

Antenna

SENSORNETS 2018 - 7th International Conference on Sensor Networks

118

Figure 3: Automated Call Workflow.

message with value "true" for fire indication), a

threshold value for sensor data (e.g., the sensor

temperature is above 60ºC) or the result from

correlating data between multiple sensors (e.g.,

temperature sensor report values above 60ºC and

smoke sensor indicates presence of fire). It is critical

that this step is highly reliable otherwise, in case of

false positives, false emergencies are re-ported

(resulting in unnecessary deployment of resources)

or, in case of true negatives, emergencies are missed

(resulting in loss of material and possibly lives).

In case an emergency is detected, an emergency

call is automatically initiated (in addition or

alternatively, a local alarm can be triggered to pre-

defined recipients).

The following steps are taken when establishing

an emergency call:

- Based on the emergency type and using a

database of predefined messages (message DB),

a text message is composed containing pertinent

information about the emergency, which includes

type of incident, date&time and sensor

information (where relevant), but also caller

information (name, address and location);

- Initiate a SIP outgoing call to the emergency

services (which will use the VoIP/PBX Gateway

to reach the PSTN and subsequently the PSAP);

- Once the call is initiated, activate the TTS engine

to convert the text message to audio (and thus

automatically report the emergency in a human

understandable form). To ensure the message is

received and understood by the PSAP (human)

operator, it can be replayed a number of times (in

our case, we set to 3 times);

- Terminate the call.

5 MESSAGES

During an emergency call, audio is generated from a

text message that is built based on the type of event.

Given the context of our application, the generated

message needs to be simple, easy to understand and

contain all relevant information for emergency

services to react. As such, the text message should

contain:

- The indication that it is an automatic message;

- Information about the type of emergency;

- Information about the subscriber/user (subscriber

name);

- Location information (civic address);

- Optionally, associated sensor information when

useful.

Example of pre-formatted messages, for situations of

fire detection and abnormal temperature detection,

are presented in Figure 4. Note that dynamic fields

inside brackets are filled based on configuration

values or sensor data.

Fire emergency predefined message:

sipspeakfire: linphonecsh

generic "speak default This is an

automated emergency call from {{subscriber

name}} home, address is {{civic address}}.

Fire detected inside home. Please help."

High abnormal temperature predefined message:

sipspeakfire: linphonecsh

generic "speak default

Environment alarm at {{subscriber name}}

home, address is {{civic address}}.

The temperature is {{tdata}} degrees.

The humidity is {{hdata}}%. Please help."

Figure 4: Predefined Messages.

Smart Device Prototype for Automated Emergency Calls - Using SIP and TTS to Reach Legacy Emergency Services

119

6 DEMONSTRATION

The prototype was installed in a fictional location,

which was called John Doe's house. The smart

device was located in the kitchen.

The Hub consisted of a RaspberryPi model 3

running the Raspbian operating system. Additional

modules used were the MQTT server (based on

mosquitto), eSpeak TTS engine and the Linphone

SIP client. Asterisk (VoIP-PBX Gateway) was used

to reach the public telephony and establish voice

calls.

For purposes of managing the devices, logging,

visualisation and triggers, the open-source home

automation platform "Home Assistant" was used.

See Figure 5 for an example of the provided output.

Figure 5: Device Data Visualisation via "Home

Assistance".

The system was configured to raise an alarm if

specific criteria are verified, which, in our case, was

simplified to a sensor value being above a pre-

defined value. Therefore, if the system detects a

situation where a sensor data exceeds a given

threshold, it automatically initiates an emergency

call. Note that, for demonstration purposes and since

emergency calls may only be placed in real cases, a

personal phone number was used as a recipient for

the generate automated alerts.

In our test scenario, we simulated two events:

- The first relates to the presence of smoke (i.e.,

fire indicator), by using a lighter. As it can be

seen in Figure 6, the smoke sensor value had a

significant increase close to 15h00.

- The second relates to the increase in temperature

above a "safe" value (in our case, just above 35º

C). We simulated a temperature increase using a

heat generator (hair dryer) close to the

temperature sensor. An increase in the measured

temperature (above the 35 ℃ threshold) close to

17h00 is visible in Figure 6.

Both events triggered the automatic initiation of an

emergency call. A message was composed following

the format presented in Figure 4, which includes

relevant details pertaining to the emergency

including name of owner, address (i.e., location) and

type of emergency. The system then establishes a

SIP call and uses the TTS engine to convey the

message in audio form to the specified recipient.

7 CONCLUSION

The IoT paradigm and increasing presence of smart

devices across all sectors of society continues to find

new applications including in emergencies. The

eCall initiative brought the concept of automated

calls in situations of serious car accidents, aiming to

improve emergency response time and reduce the

number of fatalities. It is reasonable to expect that

other types of emergencies may benefit from this

concept as well.

Figure 6: Sensor Data Visualisation Over Time.

SENSORNETS 2018 - 7th International Conference on Sensor Networks

120

In this paper, we extended our first work in (Buf

et al., 2017) by describing the implementation and

demonstration of a smart device capable to detect

fire and abnormal temperature values that can be

used to determine the occurrence of an emergency

incident. In such a case, the system generates a

preformatted message incorporating specific

emergency-related information that is converted to

audio format using a TTS-engine. In this way, our

application considers the legacy nature of ES and

PSAP specifically, which mainly operates with

audio-based calls, bringing the advantage of IoT

without requiring significant changes (and

investments) to the ES infrastructure and systems.

However, considering the forecasted evolution

for next generation ES to be implemented over the

coming decades, as envisaged and explored in the

NEXES Action, we will explore in future work the

use of IP and data exchange to data-enabled PSAPs,

following concepts such as the next generation eCall

(Gellens and Tschofenig, 2016) and smart

environments (Manso et al., 2017).

ACKNOWLEDGMENTS

This paper has been prepared as part of the NEXES

Research and Innovation Action, which has received

funding from the EU’s Horizon 2020 research and

innovation programme under Grant Agreement No.

653337.

REFERENCES

Fortino, G., Guerrieri, A., Lacopo, M., Lucia, M., Russo,

W. (2013). An Agent-Based Middleware for

Cooperating Smart Objects, PAAMS (Workshops),

2013, Pgs: 387-398.

Gartner, Inc. Forecast: Internet of Things — Endpoints

and Associated Services, Worldwide, 2016.

European Commission (2014). eCall: Time saved = lives

saved. [online] Available at: https://ec.europa.eu/

digital-single-market/ecall-time-saved-lives-saved

[Accessed 1 Oct. 2017].

3GPP (2017). TR 26.967 V14.0.0. eCall Data Transfer;

In-band modem solution (Release 14). [online] 3GPP.

Available at: https://portal.3gpp.org/desktopmodules/

Specifications/SpecificationDetails.aspx?specificationI

d=1506 [Accessed 1 Oct. 2017]

European Commission (2017). Road safety statistics:

What is behind the figures? [online] Available at:

http://europa.eu/rapid/press-release_MEMO-17-

675_en.htm [Accessed 1 Oct. 2017].

Manso, M., Guerra, B., Carjan, C., Sdongos, E.,

Bolovinou, A., Amditis, A., Donaldson, D. (2017).

The Application of Telematics and Smart Devices in

Emergencies. In Springer (Ed.), Integration,

Interconnection, and Interoperability of IoT Systems

(pp. 169-197). doi: 10.1007/978-3-319-61300-0

NEXES (2015). The NEXES Concept. [online] Available

at: http://nexes.eu/the-nexes-concept/ [Accessed 1 Oct.

2017].

Gellens, R. and Tschofenig, H. (2016), Next-Generation

Pan-European eCall, draft-ietf-ecrit-ecall-07.txt, The

Internet Engineering Task Force, February 19, 2016.

Available at: https://tools.ietf.org/html/draft-ietf-ecrit-

ecall-07 [Accessed 1 Oct. 2017].

Buf, M., Manso B., and Manso, M. (2017) Smart Devices

for Automated Emergency Calls in Proceedings of 3rd

EAI International Conference on Interoperability in

IoT, Valencia, Spain. 7th November 2017.

National Fire Protection Association (2017). Fast facts

about fire. [online] Available at:

http://www.nfpa.org/Public-

Education/Campaigns/Fire-Prevention-Week/Fast-

facts-about-fire [Accessed: 26 Nov. 2017].

International Organization for Standardization (2016).

ISO/IEC 20922:2016. Information technology -

Message Queuing Telemetry Transport (MQTT)

v3.1.1. [online] Available at: https://www.iso.org/

standard/69466.html [Accessed 1 Oct. 2017].

Smart Device Prototype for Automated Emergency Calls - Using SIP and TTS to Reach Legacy Emergency Services

121