Activity-monitoring in Private Households for Emergency Detection:

A Survey of Common Methods and Existing Disaggregable Data Sources

Sebastian Wilhelm

Deggendorf Institute of Technology, Technology Campus Grafenau, Hauptstrasse 3, Grafenau, Germany

Keywords:

Human Activity Recognition (HAR), Human Presence Detection (HPD), Ambient Assisted Living (AAL),

Activity Monitoring, Presence Detection, Ambient Sensor, Emergency Detection, Smart Home, Survey.

Abstract:

Ambient-Assisted Living (AAL) technologies can enable the elderly people to live a self-determined life in

their own home environment instead of hospitals and retirement homes for a longer period of time. Hence,

AAL systems are not only used for everyday support but also for the detection of potential emergency situ-

ations and for triggering notiﬁcation chains. For this purpose the people are usually continuously monitored

within their residents by ambient or wearable sensors to detect deviations in their daily behavior.

This work surveys common used technologies for Human Activity Recognition (HAR) / Human Presence De-

tection (HPD), which is the basis for emergency detection. Furthermore, by examining various home automa-

tion software, existing data sources from the residential infrastructure, are identiﬁed that would be suitable for

detecting personal activities.

1 INTRODUCTION

Due to the demographic change – which means a ris-

ing life expectancy and a decrease in the birth rate

– the German society is aging increasingly, so el-

derly care is one of the major challenges for soci-

ety in the near future (Hoffmann, 2016; Fischer and

amer, 2016; Paulus, 2015). Parra et al. (Parra

et al., 2015) noted that elderly people usually have

several health afﬂictions. 85% of the elderly have at

least one chronic disease, 65% even two or more. Fur-

thermore, it is important to know that most of elderly

people (61% of men and 75% of women) live alone

or with his/her partner. The ones who live alone gen-

erally have more accidents. In fact, 30% of them have

one fall per year and the 50% of them even suffer

more than one fall. (Parra et al., 2015). Therefore,

it is essential to monitor the health status – i.e., the

activity of the individual – in order to recognize pos-

sible emergency situations in the home environment

(Hoffmann, 2016; Munstermann, 2015).

There are already various market solutions and re-

search projects for supporting elderly in their every-

day life and monitoring people in their home environ-

ment to detect possible emergency situations in the

households which are denoted as Ambient-Assisted

https://orcid.org/0000-0002-4370-9234

Living (AAL) systems (Uddin et al., 2018). This

work focuses on AAL systems for emergency detec-

tion. In contrast to common emergency call systems,

where the resident himself or herself has to actively

request help, AAL systems are supposed to recognize

automatically when a potential emergency situation

exists.

Therefore, most common AAL systems use data

from various sensors, which are installed specially for

this purpose within the residence or are worn on the

body of the person to be monitored (Munstermann,

2015).

In the scope of our work within the research

project BLADL an alternative approach to monitor

residents inside their home environment by reusing

existing data sources from the residential infrastruc-

ture instead of installing additional sensors is investi-

gated. This allows AAL technologies to be integrated

even more unobtrusively into the everyday lives of el-

derly people.

In private households there are already numerous

data sources such as smart meters, digital water me-

ters, weather stations, routers, mobile phones or voice

assistants available. Intelligent algorithms (e.g., Ma-

chine Learning (Alpaydin, 2020), Deep Learning (Le-

Cun et al., 2015)) can be used to disaggregate this data

and conclude on personal activities. This, in turn, al-

lows the creation of comprehensive activity proﬁles

Wilhelm, S.

Activity-monitoring in Private Households for Emergency Detection: A Survey of Common Methods and Existing Disaggregable Data Sources.

DOI: 10.5220/0010180002630272

In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF, pages 263-272

ISBN: 978-989-758-490-9

263

of the residents. Deviations from the typical activity

proﬁle can in turn indicate possible emergency situ-

ations (Floeck and Litz, 2009; Clement et al., 2013;

Reyes-Ortiz et al., 2016; Parra et al., 2015).

The main contribution of this paper is to survey

which data sources are currently used for HAR / HPD

and which further data sources are available in the in-

frastructure of private households, that can potentially

be used for HAR / HPD.

The remaining of the paper is structured as fol-

lows: common methods for HAR / HPD are pre-

sented in Section 2. With Section 3 existing data

sources from the residential infrastructure, are iden-

tiﬁed which are potentially suitable for detecting hu-

man activities. In addition different software for home

automation were examined. Finally, the paper is con-

cluded in Section 4.

2 COMMON METHODS

Most of the current available AAL systems for emer-

gency detection use sensor data to monitor the activi-

ties of the residents. In practice, motion detectors, fall

detectors, fall mats or window/door contact sensors

are used for such applications. They are often called

ambient sensors because they can be integrated into

the home environment. Vital sensors are another type

of sensor that can be used for emergency detection.

These are worn directly on the body by the person to

be monitored. This enables even more detailed moni-

toring of the person and emergencies can be detected

even faster (Munstermann, 2015).

In the domain of activity recognition the terms

’action’ and ’activity’ are commonly used. Chen et

al. (Chen et al., 2012) deﬁned the term ’action’ as a

simple ambulatory behavior executed by a single per-

son and typically lasting for short duration of time

(e.g., opening a door). The term ’activity’ refers to

complex behaviors consisting of a sequence of actions

and/or interleaving or overlapping actions (e.g., mak-

ing a meal). In most cases, the sensors only detect ’ac-

tions’. For further processing in the AAL domain of-

ten the information about the speciﬁc ’activity’ is re-

quired. To draw conclusions from ’actions’ to ’activ-

ities’, researchers have created various probabilistic

models. The Hidden Markov Model (HMM), Hidden

Semi Markov Model (HMM) and the conditional ran-

dom ﬁeld (CRF) are among others the most popular

modeling techniques (Bakar et al., 2015; Kim et al.,

2010; Ghasemi and Pouyan, 2015). Thereby, the main

challenges are: (i) recognizing concurrent activities

(ii) recognizing interleaved activities (iii) ambiguity

of interpretation (iv) multiple residents. (Kim et al.,

2010).

Activities are often categorized as Activities of Daily

Living (ADLs) or Instrumental Activities of Daily

Living (IADLs). ADLs includes the fundamental

skills typically needed to manage basic pyhsical need

(e.g., personal hygiene, dressing, toileting). IADLs

includes more complex activities related to indepen-

dent living in the community (e.g., managing ﬁnances

and medications). Milnac and Feng (Mlinac and

Feng, 2016) noted that the impairment of IADLs can

indicate cognitive impairment and mild dementia.

Since our work focuses to the recognition of acute

emergency situations – not on the monitoring of the

state of help of the residents – it is not necessary to

conclude from the detected ’actions’ to ’activities’ or

further categorize them as ADL or IADL.

In their survey papers Rashidi and Mihailidis

(Rashidi and Mihailidis, 2013) and Uddin et al. (Ud-

din et al., 2018) listed the most common sensors for

action detection in the AAL domain.

Table 1 lists ambient sensors used in smart environ-

ments, Table 2 shows typical wearable and mobile

sensors.

Table 1: Ambient sensors used in smart environments

(Rashidi and Mihailidis, 2013; Uddin et al., 2018; Eldib

et al., 2016).

Sensor Measurement

PIR

Motion

Active Infrared Motion / Identiﬁcation

RFID

Object Information

Pressure Pressure on Mat, Chair, etc.

Smart Tiles Pressure on Floor

Magnetic Switches Door Opening/Closing

Ultrasonic Motion

Camera Activity

Photo Sensor / Activity

Visual Sensor

Microphone Activity

Temperature Sensor Temperature of a room

Water ﬂow sensor Flow of water

Force Sensor Movements and falls

Smoke Sensor Smoke in the environment

Passive Infrared Motion Sensor

Radio Frequency Identiﬁcation

Calvaresi et al. (Calvaresi et al., 2016) and Ud-

din et al. (Uddin et al., 2018) noted that in general

these sensors are used independently of each other,

although the use of multi-component ambient sensor

technologies would increase the quality of the sys-

tems. Systems using combined sensor technology

most often use a combination of Passive Infrared Mo-

tion Sensor (PIR) and video cameras. The next most

common combination is a combination of pressure

HEALTHINF 2021 - 14th International Conference on Health Informatics

264

Table 2: Typical wearable and mobile sensors (Rashidi and

Mihailidis, 2013).

Sensor Measurement

Accelerometer Acceleration

Gyroscope Orientation

Glucometer Blood Glucose

Pressure Blood Pressure

Gas Respiration

ECG

Cardiac Activity

EEG

Brain Activity

EMG

Muscle Activity

EOG

Eye Movement

Pulse Oximeter Blood Oxygen Saturation

GSR

Perspiration

Thermal Body Temperature

Electrocardiography

Electroencephalography

Electromyography

Electrooculography

Galvanic Skin Response

and PIR sensors (Uddin et al., 2018).

Further to the solutions for HAR mentioned in

Table 1 and Table 2, investigations are also carried

out reusing existing data sources to draw conclusions

about user activity, which are listed below:

(i) WiFi-Signal:

Several investigates on the extraction of activity-

related information from Wireless Local Area

Network (WiFi) signals was already made. Wang

et al. (Wang et al., 2015) and Pu et. al (Pu

et al., 2013) examined the Channel State Infor-

mation (CSI) of the WiFi signal for ﬂuctuations

caused by the reﬂection of human bodys. This

enables them to detect individual movements of

residents (e.g., sitting down, walking, raising an

arm). In contrast to examining the CSI Gu et al.

(Gu et al., 2016) examined the Received Signal

Strength Indication (RSSI) due to the simplicity

of extracting this data with the same results.

Another approach to the use of WiFi signals for

HAR was investigated by Xie et al. (Xie et al.,

2016). They developed an app which allows

the localisation of a smartphone within the resi-

dence by examining which Access Points (APs)

are available with which signal quality. This ap-

proach was evaluated in an urban space.

(ii) Smartphone:

Smartphones have become an alternative for

wearable sensing due the diversity of sensors they

internally support. Furthermore, the devices offer

capacities for networking and processing (Reyes-

Ortiz et al., 2016; Parra et al., 2015). Current re-

search is primarily based on data from accelerom-

eters and gyroscopes to draw conclusions about

speciﬁc user actions. The investigations focuses

on the optimization of algorithms of individual

activities using various machine learning meth-

ods (e.g., Deep Belief Network (DBN) (Hassan

et al., 2018), Supporting Vector Machine (SVN)

(Reyes-Ortiz et al., 2016)).

In contrast the possibilities of using the internal

sensors – especially camera and microphone –

in an ambient environment are also being inves-

tigated (Parra et al., 2015; Chen et al., 2012).

Parra et al. (Parra et al., 2015) analysed the use

of smartphones for AAL and eHealth use cases.

They identiﬁed the following possible applica-

tions: (i) heart rate monitoring (ii) breathing and

pulse (iii) moods monitoring (iv) detecting stress

(v) positioning and localization (vi) spirometry

sensing (vii) sleep monitoring.

(iii) Smart Meter:

Clement et al. (Clement et al., 2012; Clement

et al., 2013) have developed an approach for dis-

aggregating smart meter measurements to draw

conclusions about the use of individual technical

devices. This in turn can detect a certain action of

a resident.

(iv) Home Weather Station:

Wilhelm et al. (Wilhelm et al., 2020a) ana-

lyzed the carbon dioxide (CO2) readings of smart

weather stations to determine the presence or

absence of people indoors. The authors based

their work on the fact that humans produce CO2

through their respiration, which is then distributed

throughout the room. As a result, if one (or more)

persons are in a room, a signiﬁcant increase in

CO2 concentration in the room can be noted. If a

person is no longer in a room, the CO2 concentra-

tion decreases due to inﬁltration and the absence

of the person can be indicated.

3 FURTHER DATA SOURCES

With this paper we analysed which further, already

existing data sources from the residential infrastruc-

ture, are potentially suitable to detect human activi-

ties / presence within the residence. Therefore, we

have systematically identiﬁed available and accessi-

ble data sources by examining different home automa-

tion software and evaluating them individually based

on the literature. We limited ourselves to open source

solutions under the assumption that the community

has already programmed interfaces for the majority of

potential data sources that can be present in a house-

hold. We ﬁrst investigated which of the open source

Activity-monitoring in Private Households for Emergency Detection: A Survey of Common Methods and Existing Disaggregable Data

Sources

265

Table 3: Open Source Home Automation Software (as per May 30th, 2020). Ordered by #GitHub Repositories.

Name Language Google-Trends Rank #GitHub Repos. #Forum posts #Forum users

HomeAssistant C++ 8 3317 7043 2717

Homebridge C++ 4 3306 2534 194

OpenHAB Java 1 2323 555000 36015

Node-Red C++ 9 1657 182368 22973

Domoticz Java 3 1424 n.A n.A.

IoBroker PHP 6 1151 170000 8100

Jeedom Python 5 789 48232 6985

FHEM Eagle 2 753 n.A. n.A.

MajorDoMo C++ 7 394 104864 4371

Pimatic Python 11 394 973000 64400

EventGhost Pyhon 12 103 n.A. 13

HomeGenie Javascript 12 62 11416 1220

AGO Control Java 10 53 1049 220

Calaos NodeJS 12 45 40900 1500

MyController Perl 13 40 1030507 22430

MisterHouse NodeJs 13 22 439900 27000

OpenMotics NodeJS 13 12 n.A. 2238

LinuxMCE Perl 13 10 n.A. n.A.

Gladys Assistant NodeJS 13 9 n.A. 2462

piDome Java 13 4 2800 562

Smarthomatic NodeJS 13 3 120000 8900

OpenNetHome PHP 13 1 114406 5418

solutions are most common by noting the following

aspects (as per May 30th, 2020):

(i) Google-Trends Ranking (based on the Google

Trends - Service) (ii) #GitHub Repositories (iii) #Fo-

rum posts (iv) #Forum users.

The applications are listed in Table 3.

We noticed that HomeAssistant has the most GitHub

repositories and OpenHAB is the leader in Google

Trends ranking. Therefore we investigated these two

applications in more detail, since we assume that the

developer community is most active on these systems

and most integrations to existing data sources have al-

ready been developed.

In the context of reusing existing data sources for

HAR / HPD, the investigations of Perkowith et. al

(Perkowitz et al., 2004), Philipose et al. (Philipose

et al., 2004) and Wyatt et al. (Wyatt et al., 2005)

are particular noteworthy. The authors have shown

that it is possible infer the speciﬁc activity of a person

from the interaction with certain objects (e.g., coffee

maker). Therefore, they tagged different objects with

Radio Frequency Identiﬁcation (RFID) tags and at-

tached RFID readers on the wrists of the residents.

Table 4 lists the data sources that have been iden-

tiﬁed and are potentially suitable for HAR / HPD.

It was decided not to include mobile devices (e.g.,

Smartphone or Smart Watch) as these are more likely

to be classiﬁed as wearable sensors.

4 CONCLUSION AND FUTURE

WORK

This paper outlines that there exists numerous data

sources in private households, which can potentially

be used for HAR / HPD and so allow monitoring peo-

ple within their residence without installing additional

sensors. However, common systems use primarily

proprietary sensor technology for detecting human

activity / presence. Only the reuse of smartphone

(Chen et al., 2012; Reyes-Ortiz et al., 2016; Parra

et al., 2015; Hassan et al., 2018), WiFi (Wang et al.,

2015; Pu et al., 2013; ?; Xie et al., 2016), smart meter

(Clement et al., 2012; Clement et al., 2013) or home

weather station (Wilhelm et al., 2020a) data to detect

activities / presence of persons within the home envi-

ronment is already investigated in literature.

The results of this survey can now be used in fur-

ther work to develop disaggregation algorithms for in-

dividual data sources and to investigate them in detail

on their suitability for HAR / HPD or for emergency

recognition as described by Hamper (Hamper, 2020).

Thus AAL systems can be developed, which are de-

tect potential emergencies within the resident without

the need for proprietary sensors.

HEALTHINF 2021 - 14th International Conference on Health Informatics

266

Table 4: Further potentially suitable data sources for HAR / HPD.

Data Source Features Potentials for HAR / HPD

air

conditioning

ventilation

(i) status of the system (e.g., fan speed)

(ii) the readings of integrated sensors –

usually temperature and humidity sen-

sors

A change of state (e.g., switching on/off) can indi-

cate a human interaction, unless it is automated or

sensor controlled.

In addition, the measurement data can potentially be

disaggregated analogous to Wilhelm et al. (Wilhelm

et al., 2020a).

air quality

and tem-

perature

sensors

(i) temperature (ii) humidity Signiﬁcant changes in indoor temperature or humid-

ity can be caused by humans, which allows the de-

tection of activity. The reason for such a change

may be that windows or doors have been opened

(Fitzner and Finke, 2012).

In addition, the measurement data can be disaggre-

gated analogous to Wilhelm et al. (Wilhelm et al.,

2020a) due to the fact, that the human body radiates

a natural warmth (Kessel et al., 2010).

alarm sys-

tem

(i) alarm state / zone alarm (ii) armed

away / stay indicator (iii) motion

(iv) door / windows contact (v) water -

alarm (vi) ﬁre - alarm (vii) system state

Alarm systems are designed to detect human activ-

ity (e.g., by motion detectors), the data can therefore

also be used for HAR / HPD in the AAL area.

aquarium

monitoring

system

(i) water temperature (ii) PH / NH

/ oxygen level (iii) light level /

kelvin

It is a natural process that the water quality in ﬁsh

water aquariums changes constantly. One example

is the so-called ’Nitrogen Cycle’, whereby the NH

concentration in the water increases. A signiﬁcant

reduction in NH

concentration can be reached by

ﬁlling up fresh water (Saint-Erne, 2017) – which

represents human activity and can therefore be iden-

tiﬁed by the measurement results of the aquarium

monitoring system .

bed (i) state (ii) pressure Smart beds offer the possibility to read out whether

a person is lying in bed and also determine the pres-

sure on the mattress. Thus the human activity can be

determined directly. Furthermore, pressure changes

show that the person lying in bed is still alive.

body scale (i) weight (ii) fat ratio When a person uses the body scale, which generates

data, human activity can be detected directly.

car (i) door-lock (ii) charge state / battery

level (iii) climate (iv) inside-tempera-

ture (v) location (vi) speed (vii) engine

state (viii) service

When a person interacts with the car, e.g., by driv-

ing it, locking it, or starting/ending a charging pro-

cess, a clear human action can be determined from

the data.

clock /

alarm clock

(i) alarm-state (ii) timer (iii) radio Changing the settings (e.g., alarm time) or changing

the status of the alarm clock (e.g., turning off an

alarm clock) requires human interaction.

coffee

machine

(i) state (ii) operating mode Using the coffee machine (e.g., by brewing a coffee)

can be read from the data of smart coffee machines

and therefore provide information about human ac-

tivities.

computer (i) state (ii) detail usage information The use of a computer can be recorded and analyzed

with great precision. Every interaction with the de-

vice indicates a certain human activity.

Activity-monitoring in Private Households for Emergency Detection: A Survey of Common Methods and Existing Disaggregable Data

Sources

267

Table 4: Further potentially suitable data sources for HAR / HPD (cont).

dishwasher (i) state (ii) operating mode (iii) pro-

gram

Switching the dishwasher on and off or even inter-

rupting a running washing process indicates human

activity.

doorbell/door

intercom

system

(i) events when the doorbell was

pressed (ii) movements in front of the

door (iii) pictures outside the door

(iv) actions at the door relay

Since the doorbell itself is placed in front of the

apartment, it is only conditionally suitable for HAR

/ HPD within the residential area. However, actions

at the intercom system or at the door relay clearly

indicate human activity in the residence. In addi-

tion, the images from video-based door intercoms

can be analyzed to determine when a person leaves

or enters the house (Tan et al., 2006).

EV charging

stations

(i) vehicle loading (ii) vehicle state

(iii) vehicle locked (iv) wall-box state

(v) authenticated entity

When a loading process is started or stopped, hu-

man interaction is mandatory. In addition, changes

in the vehicle state (e.g. unlocking or locking) also

indicate human activity.

fridge

/freezer

(i) state (ii) operating mode (iii) cur-

rent temperature (iv) target tempera-

ture

Changes in device settings (target temperature) or

operating mode directly indicate human activity.

garage door (i) door status (ii) sun reﬂection

(iii) switch status (iv) vehicle status

Opening or closing the garage door

is always due to human activity.

In addition, individual garage door opener systems

offer the possibility to query whether the vehicle is

in the garage. If this state changes, human activity

must also be assumed.

heating

system /

heat pump

(i) current room temperature (ii) tar-

get room temperature (iii) tempera-

ture boiler (iv) hot water temperature

(v) temperature ﬂow / return (vi) oper-

ating mode

The heating system offers several opportunities to

draw conclusions about human activity. First, a

change of settings (e.g., target temperature) usually

means that a person has executed an activity. Fur-

thermore, the (warm) water consumption in apart-

ments can be disaggregated to draw conclusions

about human activity (e.g., person was showering).

Modern heating systems are additionally equipped

with extensive sensor technology, which monitors

the indoor air quality. These could also be disaggre-

gated analogous to Wilhelm et al. (Wilhelm et al.,

2020a) to draw conclusions about human activity.

Hi-Fi sys-

tem/ media

receiver

(i) operating state (ii) amp power

(iii) amp settings (iv) zone settings

(v) current played per zone (vi) current

mode per zone

State changes like turning on / turning of or change

the amp power or settings indicates direct human

interaction with the Hi-Fi system / media receiver.

home

weather

station

(i) temperature (indoor / outdoor)

(ii) humidity (indoor / outdoor)

(iii) CO

(indoor / outdoor) (iv) Air

Quality Index (v) PM2.5 level (vi) rain

(vii) wind (viii) UV index (ix) system

state

Even if the outdoor information is not relevant to the

HAR / HPD, signiﬁcant changes in indoor temper-

ature or humidity can be caused by humans, which

allows the detection of activity. The reason for such

a change may be that windows or doors have been

opened (Fitzner and Finke, 2012).

Furthermore, the carbon dioxide measurements

(CO2) can be used to determine the presence or ab-

sence of people indoors as already investigated by

Wilhelm et al. (Wilhelm et al., 2020a).

irrigation/

smart gar-

dening

system

(i) zone mode (ii) current operating

(iii) temperature (iv) humidity (v) wind

speed (vi) water level

Changes in settings or operating mode (if not auto-

mated) indicate human activity.

HEALTHINF 2021 - 14th International Conference on Health Informatics

268

Table 4: Further potentially suitable data sources for HAR / HPD (cont).

lamp (i) switch state (ii) dimmer (iii) color Changes of state (switch on / off / dimmer or change

color) indicate switching actions by humans, unless the

lamps are controlled automatically.

lawn mower (i) state (ii) mowed (iii) error The systems basically work autonomously, but manual

status changes can be made which can be interpreted as

human activity.

lock (i) status of the locks (ii) registering ac-

tions – i.e. accesses

Status changes or access events indicates clear human

activity.

microwave (i) state (ii) operating mode (iii) timer The use of the microwave can be indicated by the resulting

change in state.

oven (i) state (ii) operating mode (iii) current

temperature (iv) target temperature

Changes of state (e.g. operating mode or adjusting the

target temperature) follows human interaction with the

device.

power plug (i) switch state (ii) current power (iii) en-

ergy

Switch state changes indicated direct human interaction

with the device.

The current power information can also be used to draw

conclusions as to whether the connected device is cur-

rently active and thus to detect human interaction with

the connected device (e.g., coffee machine) (Wilhelm

et al., 2020b; Wilhelm et al., 2021).

printer (i) state (ii) jobs (print/scan) State changes (except for automated programs like clean-

ing) or new jobs indicates human activity.

radiator

thermostat

(i) current temperature (ii) target tempera-

ture (iii) state

Signiﬁcant changes in the indoor temperature can be

caused by humans, which allows the detection of activity

(e.g., opening a window) (Fitzner and Finke, 2012). Fur-

thermore, the change of the target temperature indicates

direct human interaction.

(rain)

water pump

(i) water-level (ii) operating mode

(iii) source used (iv) switch state

Water ﬂow is mostly due to human activities (e.g., ﬂush-

ing toilets, garden irrigation). The corresponding data

can be disaggregated to identify individual tapping points

and thus to conclude on a concrete human-induced action

(Froehlich et al., 2011).

remote con-

trol

(i) received remote (ii) received command If remote control commands are received, this is due to

human activity.

roller shutter (i) position Changing the position of the roller shutters indicates hu-

man activity.

router /

network-

switches /

access points

(i) state (ii) power (iii) connected

devices (iv) deep packet inspections

(v) WiFi-link-quality

Various conclusions about human activities can be drawn

from the general network information by analyzing the

data consumption of individual end devices. Furthermore

can the WiFi information be disaggregated as described

by Wang et al. (Wang et al., 2015), Pu et. al (Pu et al.,

2013), Gu et al. (Gu et al., 2016) or Xie et al. (Xie et al.,

2016).

smart meter

(gas)

(i) gas delivery (ii) gas valve position

(iii) error code

Depending on the installed heating system and/or gas-

powered appliances in the household, the smart meter

offers the possibility of drawing conclusions about indi-

vidual activities. For example, when hot water is being

used or when cooking is being done.

smart meter

(power)

(i) current power delivery (ii) current

power production (iii) power failures

(iv) voltage (L1, L2, L3) (v) current (L1,

L2, L3) (vi) switch position (vii) error code

Power consumption data can be disaggregated, which al-

lows conclusions to be drawn to determine which in-

dividual devices are currently active in the household.

This, in turn, allows conclusions to be drawn about in-

dividual human activities (Clement et al., 2012; Clement

et al., 2013; Wilhelm et al., 2020b).

Activity-monitoring in Private Households for Emergency Detection: A Survey of Common Methods and Existing Disaggregable Data

Sources

269

Table 4: Further potentially suitable data sources for HAR / HPD (cont).

smart meter

(water)

(i) water delivery / heating delivery (ii) wa-

ter meter valve position (iii) error code

Water ﬂow is mostly due to human activities (e.g., ﬂush-

ing toilets, showers). The corresponding data can be

disaggregated to identify individual tapping points and

thus to conclude on a concrete human-induced action

(Froehlich et al., 2011).

smart speaker (i) commands / interactions (ii) volume

(iii) music played (iv) Bluetooth connec-

tions (v) reminders (vi) routines (vii) mes-

sages (viii) connected devices

When humans interact with intelligent speakers, hu-

man activity can be detected based on the com-

mands/interactions given.

smoke detec-

tor

(i) smoke alarm state (ii) manual test active

(iii) ui color (iv) battery state (v) connec-

tion state

According to Rashidi and Mihailidis (Rashidi and Mihai-

lidis, 2013) and Uddin et al. (Uddin et al., 2018) smoke

detectors are already commonly used for HAR. It is also

possible to read out when a manual device test is per-

formed so that human activity can be detected.

surveillance

camera

(i) picture / recording (ii) motion

(iii) sound (iv) state

If surveillance cameras are installed indoor, they can

directly detect human activities within the residence.

For example, image recognition algorithms can be used

(Rashidi and Mihailidis, 2013; Uddin et al., 2018; Tan

et al., 2006).

Human activities can also be monitored via the internal

microphones of some surveillance cameras (Rashidi and

Mihailidis, 2013; Uddin et al., 2018).

swimming

pool

(i) temperature (pool / spa / air) (ii) pump

state (iii) spa pump state (iv) heater state

(v) PH (vi) pool light

Changes in the operating status of the pump, heating or

lighting may be caused by human activity.

Under certain circumstances it is even possible to infer

human activity in the pool from the change in water qual-

ity (Wyczarska-Kokot, 2015).

switch / dim-

mer / relay

(i) switch state (ii) input (iii) current power Switching state changes indicate direct human interac-

tion, unless it is an automatically controlled system (e.g.

by a timer).

The current power information can also be used to draw

conclusions as to whether the connected device is cur-

rently active and thus to detect human interaction with

the connected device (e.g., coffee machine) (Wilhelm

et al., 2020b).

telephone

system

(i) calls (incoming / outgoing) (ii) con-

nected to (iii) missed calls

When incoming calls are accepted or outgoing calls are

made, human activity can usually be assumed.

Special features such as automatic answering machines

must be considered.

tv / beamer (i) state (ii) commands (iii) current play-

ing (iv) mute / volume (v) source (vi) (only

beamer) lamp

Changes in state, source or the current playing program

indicates in general a human action.

vacuum robot (i) state (ii) battery level If the devices are not set to an automatic program, a status

change indicates human activity.

washing ma-

chine / tumble

dryer

(i) state (ii) program (iii) phase (iv) pro-

grammed start-time

State changes or programming a new start-time are caused

by human interaction with the devices and so indicates

human activity.

water softner (i) alam / alert (ii) current ﬂow (iii) water

hardness (inlet / outlet) (iv) salt remaining

(v) water pressure

Water ﬂow is mostly due to human activities (e.g., ﬂush-

ing toilets, showers). The corresponding data can be

disaggregated to identify individual tapping points and

thus to conclude on a concrete human-induced action

(Froehlich et al., 2011).

window (i) position Status changes are related to human activity.

ACKNOWLEDGEMENTS

This work was funded by the Bavarian State Ministry

of Family Affairs, Labour and Social Affairs.

REFERENCES

Alpaydin, E. (2020). Introduction to machine learning.

MIT press.

Bakar, U. A. B. U. A., Ghayvat, H., Hasanm, S. F., and

HEALTHINF 2021 - 14th International Conference on Health Informatics

270

Mukhopadhyay, S. C.(2015). Activity and anomaly

detection in smart home: A survey. In Smart Sen-

sors, Measurement and Instrumentation, pages 191–

220. Springer International Publishing.

Calvaresi, D., Cesarini, D., Sernani, P., Marinoni, M., Drag-

oni, A. F., and Sturm, A. (2016). Exploring the ambi-

ent assisted living domain: a systematic review. Jour-

nal of Ambient Intelligence and Humanized Comput-

ing, 8(2):239–257.

Chen, L., Hoey, J., Nugent, C. D., Cook, D. J., and Yu,

Z. (2012). Sensor-based activity recognition. IEEE

Transactions on Systems, Man, and Cybernetics, Part

C (Applications and Reviews), 42(6):790–808.

Clement, J., Ploennigs, J., and Kabitzsch, K. (2012). Smart

meter: Detect and individualize ADLs. In Ambient

Assisted Living, pages 107–122. Springer Berlin Hei-

delberg.

Clement, J., Ploennigs, J., and Kabitzsch, K. (2013). De-

tecting activities of daily living with smart meters.

In Ambient Assisted Living, pages 143–160. Springer

Berlin Heidelberg.

Eldib, M., Deboeverie, F., Philips, W., and Aghajan, H.

(2016). Behavior analysis for elderly care using a

network of low-resolution visual sensors. Journal of

Electronic Imaging, 25(4):041003.

Fischer, F. and Kr

amer, A., editors (2016). eHealth in

Deutschland. Springer Berlin Heidelberg.

Fitzner, K. and Finke, U. (2012). L

uftungsregeln f

freie L

uftung. Bundesanstalt f

ur Arbeitsschutz und Ar-

beitsmedizin.

Floeck, M. and Litz, L. (2009). Inactivity patterns and alarm

generation in senior citizens’ houses. In 2009 Euro-

pean Control Conference (ECC), IEEE.

Froehlich, J., Larson, E., Saba, E., Campbell, T., Atlas,

L., Fogarty, J., and Patel, S. (2011). A longitudinal

study of pressure sensing to infer real-world water us-

age events in the home. In Lecture Notes in Computer

Science, pages 50–69. Springer Berlin Heidelberg.

Ghasemi, V. and Pouyan, A. A. (2015). Activity recogni-

tion in smart homes using absolute temporal informa-

tion in dynamic graphical models. In 2015 10th Asian

Control Conference (ASCC), IEEE.

Gu, Y., Ren, F., and Li, J. (2016). PAWS: Passive human

activity recognition based on WiFi ambient signals.

IEEE Internet of Things Journal, 3(5):796–805.

Hamper, A. (2020). Digitale service innovation f

ur die

gesundheitsversorgung im vernetzten zuhause.

Hassan, M. M., Uddin, M. Z., Mohamed, A., and Almo-

gren, A. (2018). A robust human activity recognition

system using smartphone sensors and deep learning.

Future Generation Computer Systems, 81:307–313.

Hoffmann, S. (2016). Technik die unser leben vereinfacht.

Date last accessed 23-August-2019.

Kessel, L., Johnson, L., Arvidsson, H., and Larsen, M.

(2010). The relationship between body and ambient

temperature and corneal temperature. Investigative

Opthalmology & Visual Science, 51(12):6593.

Kim, E., Helal, S., and Cook, D. (2010). Human activ-

ity recognition and pattern discovery.IEEE Pervasive

Computing, 9(1):48–53.

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learn-

ing. nature, 521(7553):436–444.

Mlinac, M. E. and Feng, M. C. (2016). Assessment of ac-

tivities of daily living, self-care, and independence.

Archives of Clinical Neuropsychology, 31(6):506–

516.

Munstermann, M. (2015). Technisch unterst

utzte Pﬂege von

morgen. Springer Fachmedien Wiesbaden.

Parra, L., Sendra, S., Jim

enez, J. M., and Lloret, J. (2015).

Multimedia sensors embedded in smartphones for am-

bient assisted living and e-health. Multimedia Tools

and Applications, 75(21):13271–13297.

Paulus, W. (2015). Selbst

andig zuhause leben im alter: Auf

dem weg zu einer integrierten versorgung. Technical

report, Institut Arbeit und Technik (IAT), Westf

alische

Hochschule, Gelsenkirchen.

Perkowitz, M., Philipose, M., Fishkin, K., and Patterson,

D. J. (2004). Mining models of human activities from

the web. In Proceedings of the 13th conference on

World Wide Web - WWW’04. ACM Press.

Philipose, M., Fishkin, K. P., Perkowitz, M., Patterson,

D. J., Fox, D., Kautz, H., and Hahnel, D. (2004). In-

ferring activities from interactions with objects. IEEE

Pervasive Computing, 3(4):50–57.

Pu, Q., Gupta, S., Gollakota, S., and Patel, S. (2013).

Whole-home gesture recognition using wireless sig-

nals. In Proceedings of the 19th annual International

Conference on Mobile computing & networking - Mo-

biCom’13. ACM Press.

Rashidi, P. and Mihailidis, A. (2013). A survey on ambient-

assisted living tools for older adults. IEEE Journal of

Biomedical and Health Informatics, 17(3):579–590.

Reyes-Ortiz, J.-L., Oneto, L., Sam

a, A., Parra, X., and

Anguita, D.(2016). Transition-aware human activ-

ity recognition using smartphones. Neurocomputing,

171:754–767.

Saint-Erne, N. (2017). Water quality in the freshwater

aquarium.

Tan, X., Chen, S., Zhou, Z.-H., and Zhang, F. (2006). Face

recognition from a single image per person: A survey.

Pattern Recognition, 39(9):1725–1745.

Uddin, M., Khaksar, W., and Torresen, J. (2018). Ambi-

ent sensors for elderly care and independent living: A

survey.Sensors, 18(7):2027.

Wang, W., Liu, A. X., Shahzad, M., Ling, K., and Lu, S.

(2015). Understanding and modeling of WiFi sig-

nal based human activity recognition. In Proceedings

of the 21st Annual International Conference on Mo-

bile Computing and Networking - MobiCom’15. ACM

Press.

Wilhelm, S., Jakob, D., and Ahrens, D. (2020a). Human

presence detection by monitoring the indoor co2 con-

centration. In Mensch und Computer 2020 (MuC’20).

ACM, New York, NY, USA.

Wilhelm, S., Jakob, D., Kasbauer, J., and Ahrens, D.

(2021). Gelap: German labeled dataset for power

consumption. In Proceedings of the International

Congress on Information and Communication Tech-

nology. Springer, Springer Singapore. (to appear).

Activity-monitoring in Private Households for Emergency Detection: A Survey of Common Methods and Existing Disaggregable Data

Sources

271

Wilhelm, S., Jakob, D., Kasbauer, J., Dietmeier, M., Gerl,

A., Elser, B., and Ahrens, D. (2020b). Organizational,

technical, ethical and legal requirements of capturing

household electricity data for use as an AAL system.

In Proceedings of the International Congress on Infor-

mation and Communication Technology, pages 374–

392. Springer, Springer Singapore.

Wyatt, D., Philipose, M., and Choudhury, T. (2005). Un-

supervised activity recognition using automatically

mined common sense. In AAAI, volume 5, pages 21–

27.

Wyczarska-Kokot, J. (2015). Comparison of chloramine

concentration in swimming pool water depending on

swimming pool intended use. Ecological Chemistry

and Engineering A.

Xie, X., Xu, H., Yang, G., Mao, Z.-H., Jia, W., and Sun, M.

(2016). Reuse of WiFi information for indoor mon-

itoring of the elderly. In 2016 IEEE 17th Interna-

tional Conference on Information Reuse and Integra-

tion (IRI).

HEALTHINF 2021 - 14th International Conference on Health Informatics

272