UNOBTRUSIVE DATA RETRIEVAL FOR PROVIDING INDIVIDUAL

ASSISTANCE IN AAL ENVIRONMENTS

Carsten Rachuy

, Sandra Budde

and Kerstin Schill

Kognitive Neuroinformatik, Universit

at Bremen, Bremen, Germany

Cognitive Systems, Universit

at Bremen, Bremen, Germany

Keywords:

Ambient assisted living, Wearable computing.

Abstract:

We present a prototype for a wearable device that measures physiological data in an unobtrusive way. The

aim is to utilize changes in these physiological patterns to infer about the user’s affective state which is used

as a evidential source of contextual information for recognizing activities of daily living (ADL) in an ambient

assisted living (AAL) environment. We describe the device, compare it to a commercial stationary solution

and give an outlook on possible scenarios for its application.

1 INTRODUCTION

Current statistics on changes in age groups in Ger-

many show a major increase in the ratio between el-

derly people and the following younger generation.

While in 2008 the number of people below 20 ap-

proximately equals the number of people over the age

of 65, forecasts show that in the year 2060 the share

of people over the age of 65 will be approximately

twice as high as the share of people under 20 (see

(Pl

otzsch, 2009), p. 16.). As aging is always a pro-

cess that affects the capabilities of the human body

and impairs the general performance regarding phys-

ical as well as mental capabilities, the need of age-

adequate and context-dependent assistance increases.

Assistance in this context is focused to AAL environ-

ments where AAL denotes the principle of providing

assistance not through deﬁned tools like wheelchairs,

walkers or other devices but by incorporating the as-

sisting devices into the environment: the environment

itself both recognizes the need for assistance and pro-

vides it in an adequate way. As the main goal of such

environments is to support a life independent from the

support of caregivers as long as possible, such envi-

ronments mostly provide assistance in order to per-

form ADLs on one’s own. Assistance on these can

either be provided in a static way by e.g. structural

changes in the environment namely wheelchair rams,

rails, panic buttons and stair lift or in a more ﬂexible

way by adapting it to the varying condition over the

course of the day. In the latter case, assistance is pro-

vided on the basis of dynamic contextual data which

is situation-dependent and varies during the course of

the day. For instance physiological measurements can

be used to infer a person’s actual affective state (Calvo

and D’Mello, 2010). In this paper, we present a de-

vice that is able to measure physiological signals in

an unobtrusive way and show that its data is reliable

enough to infer the user’s affective state.

2 PREVIOUS AND RELATED

WORK

There exist various approaches to measure physio-

logical data and to use these measurements to infer

about the state of the user. Regarding the hardware

there are numerous devices which are mostly devel-

oped for clinical use and are therefore most often sta-

tionary or semi-mobile but also some wearable de-

vices are available. In the line of stationary devices

Thought Technology

offers sensor units which are

mainly used for monitoring, biofeedback or rehabil-

itation purposes. An example for a mobile device is

the Telcomed MiniClinic

which resembles a wrist-

watch and is eligible for measuring heart rate, electro-

cardiogram, heart rhythm regularity, respiratory rate

and body temperature. Advantages of stationary de-

vices are - being situated in the medical domain -

their high reliability and accuracy. The downside is

http://www.thoughttechnology.com

http://www.telcomed.ie/wristwatch.html (veriﬁed Mai

4, 2010)

517

Rachuy C., Budde S. and Schill K..

UNOBTRUSIVE DATA RETRIEVAL FOR PROVIDING INDIVIDUAL ASSISTANCE IN AAL ENVIRONMENTS.

DOI: 10.5220/0003137705170520

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2011), pages 517-520

ISBN: 978-989-8425-34-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

- in the stationary case - that they are not suitable

for monitoring over a longer duration of time when

performing activities of daily living. In the wearable

case problems arise from proprietary software which

makes working with the raw data complicated.

There also exist some approaches on how to use

physiological data in order to adapt the behavior of

an assistance system. In (Fischer et al., 2008) the

, a multi-sensoric hardware platform was devel-

oped using a fuzzy-logic-based control unit designed

for providing assistance for people suffering from in-

continence. Based on the collected data the fuzzy-

controller computes an estimation on the remaining

time until the next urination and provides appropriate

feedback and/or warnings to the user.

In the work performed by (Poh et al., 2010) a

wearable device was developed which was used to

measure electrodermal activity (EDA) and evaluated

certain patterns during common activities as cycling,

studying and watching a movie.

Another approach was the work performed in the

SHARE-it project

where aim was to infer affect on

the basis of physiological measurements and to model

the impact of affective states on the user’s cogni-

tive performance. These ﬁndings were used to both

adapt the level of driving-assistance of an autonomous

wheelchair to the current user’s needs and to modify

the complexity of displayed visual information with

the aim to reduce the amount of needed cognitive in-

formation processing in situations where the user’s at-

tention had to be focused on the navigation tasks.

3 HARDWARE DEVELOPMENT

The hardware consists of two components. These are

a commercial heart rate detection belt which is nor-

mally used during exercise and which is distributed

by Suunto

and a device which was manufactured by

LIGHTRONIC

based on speciﬁcation by the Univer-

sity of Bremen (see Figure 1).

It has the dimensions of 3.5 cm x 4.0 cm x 1.5

cm and is worn using a wristband. It incorporates

an integrated bluetooth module for wireless commu-

nication and a Li-Ion-Accumulator serving as power

source which has a lifetime of approximately 4 to 5

hours for continuous monitoring and sending. The

device provides a number of sensors to measure dif-

ferent types of data namely skin conductance (SC),

ambient temperature, skin temperature and both ori-

entation and acceleration information along the x-, y-

http://www.ist-shareit.eu/shareit

http://www.suunto.com

http://www.budelmann-elektronik.com

Figure 1: Top left: sensor module. Top right: skin conduc-

tance sensors. Bottom: combined device when worn.

and z-axes. Additional focus has been set to keep the

design as close to normal clothing therefore being as

obtrusive as a watch in terms of pressure and move-

ment constraints.

4 EVALUATION

The device was evaluated during experiments in order

to test the reliability and stability of the measurements

by comparing it to the measurements of a commer-

cial medical device and to investigate the correlations

between stimuli and induced affective states. The ex-

periment consists of two phases: during the ﬁrst we

focused on inducing affective responses, during the

second on inducing stress.

The ﬁrst phase of the experiment follows a method

which is developed by (Bradley et al., 2001) and uses

72 stimuli from the international affective picture sys-

tem (IAPS) (Lang et al., 2008) which is an interna-

tional valid and trusted method for inducing emotions.

The second phase of the experiment focuses on induc-

ing stress by presenting 25 mathematical tasks which

are a subset from the arithmetic tasks presented by

(Kellogg et al., 1999) and increasing time pressure

during solving these tasks. The participants for this

experiment were healthy volunteers, mostly students

in the age from 20 to 40.

In both phases, heart rate (HR), skin conduc-

tance (SC), and skin temperature (Temp) were mea-

sured using the developed device and the commer-

cial Thought Technology system. In addition, dur-

ing the ﬁrst experiment, participants were asked to

do Self-Assessment Manikin (SAM) ratings (Bradley

and Lang, 1994) with respect to the induced emotion.

In the ﬁrst phase, each stimuli presentation be-

gins with the display of a preparation slide and at

the same time the recording of the physiological data

is started. After three seconds, the affective stimu-

lus is presented to the subject for six seconds. Sub-

HEALTHINF 2011 - International Conference on Health Informatics

518

sequently, the physiological data is collected for two

more seconds. After this, the rating procedure follows

in which the subject rates the pleasure dimension and

the arousal dimension of the induced emotion.

In the second phase, the timing behavior of the

task presentation is triggered by the participants per-

formance during the very ﬁrst task of each task block.

These tasks are presented as long as the participant

needs to solve them. The answer is entered into the

system by the advisor. After this, each task of a cer-

tain difﬁculty level is presented for a slightly shorter

time as the participant needed or was given respec-

tively for the preceding task of the same time. To de-

pict this time limit, a progress bar is displayed which

illustrates the time spent for this task as well as the

time left. After the time has run out, the task is no

longer displayed and the participant has to give his

solution to the task.

5 RESULTS

To analyze the physiological data which is recorded

during the experiments, we extract and compute at-

tributes as arithmetic mean, standard deviation, ﬁrst-

forward-difference, pearson correlation coefﬁcient

and spearman’s rank correlation coefﬁcient.

5.1 Comparison with Medical Device

The correlation between the signals recorded from

the sensor device (heart rate and skin conductance)

and the values collected using the Though Technol-

ogy medical system was checked by normalizing both

data and calculating the mean square difference. The

mean square percental difference between the sen-

sors for the heart rate measurements is 6%, the mean

square percental difference for the skin conductance

and skin resistance is 13%.

5.2 Reaction on IAPS Stimulus Material

One major point is inferring from the physiological

signals to the emotional state and the stress level. For

that reason we analyzed the sensor output in conjunc-

tion with the ratings the subjects gave according to the

IAPS pictures. The pearson and spearman’s correla-

tion coefﬁcient for the preprocessed values are dis-

played in Figure 2. The four blocks correspond to

the different correlation coefﬁcients (pearson / spear-

man’s) and the different ratings ( valence / arousal).

Depicted are the correlation coefﬁcients for each sub-

ject. We have a sample size of N = 72 (72 images).

The tables for pearson and spearman’s are capped at

Figure 2: Correlation coefﬁcients for IAPS test.

N = 30 or N = 60. Due to the fact that for increas-

ing N the thresholds are decreasing too we can safely

work with a threshold for a N

table

< N

experiment

. For

α = 0.05 we get t

pearson

= 0.195 and t

spearman

0.214. If a value exceeds these thresholds, the zero

hypothesis (which states ”there is no correlation”) is

proven wrong and the correlation is considered be-

ing signiﬁcant. Applied to the physiological data it

means that if we ﬁnd a correlation coefﬁcient which

exceeds these thresholds, a signiﬁcant correlation ex-

ists between the valence / arousal and the appropriate

physiological measurement. All cases in which this is

the case have been marked with a red square. As we

can see we have a number of signiﬁcant correlations,

both from the pearson as well as from the spearman’s

test. This indicates that a signiﬁcant correlation exists

between the rated arousal and valence values and the

physiological measurements.

5.3 Reaction on Mathematical Stimulus

Material

As the difﬁculty of the mathematical tasks was in-

creasing while the time for each task was decreasing,

the stress level for the subjects gradually increased

during the experiment. The pearson and spearman’s

correlation coefﬁcient for the preprocessed values are

displayed in Figure 3. The four blocks correspond to

the different correlation coefﬁcients (pearson / spear-

man’s) and the task difﬁculty. Depicted are the corre-

lation coefﬁcients for each subject. We have a sample

size of N = 25 (25 mathematical tasks). For α = 0.05

we get t

pearson

= 0.323 and t

spearman

= 0.337. Just

like in the IAPS case, if a value exceeds these thresh-

olds, the zero hypothesis (which states ”there is no

correlation”) is proven wrong and the correlation is

considered being signiﬁcant. Applied to the physio-

logical data it means that if we ﬁnd a correlation co-

efﬁcient which exceeds these thresholds, a signiﬁcant

UNOBTRUSIVE DATA RETRIEVAL FOR PROVIDING INDIVIDUAL ASSISTANCE IN AAL ENVIRONMENTS

519

Figure 3: Correlation coefﬁcients for mathematical tasks.

correlation exists between the difﬁculty of the task

and the appropriate physiological measurement. All

situations in which this is the case have been marked

with a red square. As we can see we have a number of

signiﬁcant correlations, both from the pearson as well

as from the spearman’s test. This indicates that a sig-

niﬁcant correlation exists between the task difﬁculty

the experienced physiological reaction of the subject.

6 CONCLUSIONS

We presented a prototypical development of a wear-

able device which can be used for measuring phys-

iological data and showed that inferring from these

measurements to the affective state of the user is

possible for scenarios utilizing standardized affect-

inducing techniques. Analysis of the data and com-

parison between the subjects showed that measured

physiological responses are highly individual for dif-

ferent subjects, even in standardized experimental set-

tings. An even higher impact of these inter-individual

response characteristics to affective stimuli on the col-

lected data can be expected when these measurements

are taken in non-standardized scenarios - e.g. while

performing activities of daily living in a home envi-

ronment.

7 OUTLOOK

Further development and evaluation has to be per-

formed regarding two aspects. The ﬁrst is to evaluate

to which degree physiological signals - as responses

to standardized stimulus material - are comparable

with those recorded in real world scenarios. It has to

be investigated whether inference mechanisms based

on the former setting can be transferred and applied to

the latter. The second is to perform a thorough anal-

ysis on the recorded physiological patterns and eval-

uate whether channel-dependent, non-ambiguous sig-

nal characteristics can be found which could then be

used as discriminating features for improving the in-

ference algorithm.

ACKNOWLEDGEMENTS

The presented work is funded by the project I3-

[SharC], R1-[ImageSpace], and A5-[ActionSpace]

in the Transregional Collaborative Research Center

SFB/TR 8 Spatial Cognition. The support by the

German Research Foundation (DFG) is gratefully ac-

knowledged.

REFERENCES

Bradley, M. M., Codispoti, M., Cuthbert, B. N., and Lang,

P. J. (2001). Emotion and motivation I: Defensive and

appetitive reactions in picture processing. Emotion,

1(3):276–298.

Bradley, M. M. and Lang, P. J. (1994). Measuring emo-

tion: The self-assessment manikin and the semantic

differential. Journal of Behavioral Therapy and Ex-

perimental Psychiatry, 25:49–49.

Calvo, R. A. and D’Mello, S. (2010). Affect detection: An

interdisciplinary review of models, methods and their

applications. IEEE Transactions on Affective Comput-

ing, 1(1):18–37.

Fischer, C., Sancho-Pradel, D., and Schill, K. (2008).

Towards a Multisensory-based Automation of Uri-

nary Incontinence Behavioural Treatment. In AI IN

ELDERCARE: NEW SOLUTIONS TO OLD PROB-

LEMS: PAPERS FROM THE AAAI FALL SYMPO-

SIUM, Arlington, Virginia.

Kellogg, J. S., Hopko, D. R., and Ashcraft, M. H. (1999).

The effects of time pressure on arithmetic perfor-

mance. Journal of Anxiety Disorders, 13(6):591–600.

Lang, P. J., Bradley, M. M., and Cuthbert, B. N. (2008). In-

ternational affective picture system (IAPS): Affective

ratings of pictures and instruction manual. Technical

Report A-8, University of Florida, Gainesville, FL.

otzsch, O. (2009). Bev

olkerung Deutschlands bis 2060

- 12. koordinierte Bev

olkerungsvorausberechnung.

Statistisches Bundesamt, Wiesbaden.

Poh, M.-Z., Swenson, N. C., and Picard, R. W. (2010).

A wearable sensor for unobtrusive, long-term assess-

ment of electrodermal activity. IEEE Trans Biomed

Eng, 57(5):1243–52.

HEALTHINF 2011 - International Conference on Health Informatics

520