IN-VEHICLE MONITORING OF AFFECTIVE SYMPTOMS FOR

DIABETIC DRIVERS

In-vehicle Hypoglycemia Alerting System in EU Project METABO

Jonghwa Kim

Institute of Computer Science, University of Augsburg, Universit

atsstr. 6a, D-86159 Augsburg, Germany

Alessandro Ragnoni

FERRARI Spa, via Abetone Inferiore 4, 41053 Maranello (Mo), Italy

Jacopo Biancat

S.A.T.E. S.r.l., Santa Croce 664/a, 30135 Venezia, Italy

Keywords:

Diabetes mellitus, Hypoglycemia, Emotion recognition, Biosensors, Pattern recognition.

Abstract:

Can self-management of emotion help on safety driving of diabetic patients? Fluctuant emotions in driving

can lead to very critical situations. In particular for diabetic drivers experiencing hypoglycemic events it is

inevitable to provide an intelligent alerting/recommendation system that assesses continuously driver’s affec-

tive and metabolic states and predicts sudden hypoglycemic events, in order for avoiding dangerous situations

during driving. In this paper we introduce an innovative approach to in-vehicle emotion monitoring system

conceived in the EU project METABO. The system aims for providing the drivers a self-management oppor-

tunity to monitor/control their emotional states and apt recommendations according to detected hypoglycemic

symptoms.

1 INTRODUCTION

Emotions affect perception, action and internal pro-

cesses of which the person having the emotion may

not be aware. This unawareness is then very danger-

ous for drivers, since safe driving activity demands

various types of driver’s abilities simultaneously such

as psychomotor skills, visuospatial functions, vigi-

lance and rapid information processing and judge-

ment. Actually, it is reported that the inability to

manage one’s emotions during driving is identiﬁed as

one of major causes of road trafﬁc accidents (James,

2000). This is even more critical due to the fact that

drivers often lack the ability to calm themselves their

negative emotions, for instance, when they are an-

gry or frustrated. There are a number of research re-

sults in literature that support the importance of the

emotional state of drivers for trafﬁc safety. Lajunen

and Parker (Lajunen and Parker, 2001) established

the links between anger, aggression and reported acci-

dents. Anxious may impair driver’s capability to deal

with a complex situation that occurs unexpectedly be-

cause anxiety narrows attentional focus, leading to

misinterpretation (

Ohman, 2000). Furthermore, in the

work (Armitage et al., 1999) it is claimed that posi-

tive moods promote risky decision making and more

heuristic strategies, whereas negative moods instigate

a more problem-focussed approach.

The role of ﬂuctuant emotions in driving is more

critical for diabetic drivers who suffer from hyper-

/hypoglycemia accompanying extremely unsteady af-

fective symptoms. Acute hypoglycemia, the most

common side effect of insulin therapy, may com-

promise driving skills. Functions that are most af-

fected by hypoglycemia include crucial abilities for

safe driving such as rapid judgement, attention, anal-

ysis of complex visual stimuli, memory and process-

ing of information and hand-eye coordination. Such

dysfunctions cause problem with contrast sensitiv-

ity and increased irritability and promote anger and

mood changes. Particularly, it is important to note

that the patients who lose their ability to recognize

the early signs of hypoglycemia, called hypoglycemia

unawareness, suffer from at least ten times higher risk

367

Kim J., Ragnoni A. and Biancat J. (2010).

IN-VEHICLE MONITORING OF AFFECTIVE SYMPTOMS FOR DIABETIC DRIVERS - In-vehicle Hypoglycemia Alerting System in EU Project

METABO.

In Proceedings of the Third International Conference on Health Informatics, pages 367-372

DOI: 10.5220/0002759703670372

 SciTePress

for severe hypoglycemia than that of patients with-

out this hypoglycemia unawareness. The impaired

awareness of hypoglycemia is associated with more

profound cognitive dysfunction, which takes longer to

recover after acute hypoglycemia than is experienced

by individuals with normal awareness (Deary, 1999;

Gold et al., 1995).

All these ﬁndings call to mind the need for in-

vehicle emotion monitoring system that observes pa-

tient’s emotional state during driving, reminds patient

to manage his affective state himself, predicts/alerts

forthcoming hypoglycemia events and recommends

needed activity. Recently many works on auto-

matic emotion recognition using physiological mea-

surements have been reported especially in advanced

human-computer interaction (HCI) (Kim and Andr

2008). However, as emotion is a function of time,

context, space, culture, and person, it’s intensity and

effect may also widely differ from user to user and

from situation to situation. For diabetes mellitus, it

needs to pay a special attention to the fact that apply-

ing of emotion recognition systems developed based

on healthy people might result in a risky situation.

This is because of the fact that origin stimuli caus-

ing actual emotional state can hardly be traced for

diabetes patient, due to mutual interaction between

emotional change and the glucose level, which can be

described in the form of a vicious circle (i.e. cause-

result-cause).

In this paper, we present a design of in-

vehicle emotion monitoring system using multichan-

nel biosensors. First we brieﬂy introduce the EU

project METABO where we develop the monitoring

system as a part of the project. Then we move to sum-

marize the concept of in-vehicle hyperglycemia alert-

ing system and describe our systematical approach to

the in-vehicle emotion monitoring system.

2 PROJECT METABO

The METABO

(Controlling Chronic Diseases re-

lated to Metabolic Disorders) is an european collab-

orative project funded by European Commission and

started in January of 2008 by 22 partners from 9 EU

member states.

2.1 Objective of METABO

The aim of METABO is to set up a comprehen-

sive platform, running both in clinical settings and

in every-day life environments, for continuous and

EU FP7-ICT-2007-1-216270, www.metabo-eu.org

Figure 1: The concept of the METABO (after METABO

Annex I ”Description of Work”).

multi-parametric monitoring of the metabolic status

in patients with, or at risk of, diabetes and associ-

ated metabolic disorders. The type of parameters that

will be monitored, in addition to ”traditional” clinical

and biomedical parameters, will also include subcuta-

neous glucose concentration, dietary habits, physical

activity and energy expenditure, effects of ongoing

treatments, and autonomic reactions. The data pro-

duced by METABO will be integrated with the clini-

cal data and the history of the patient and will be used

in two major interrelated contexts of care:

1. Setting up a dynamic model of the metabolic be-

havior of the individual to predict the inﬂuence

and relative impact of speciﬁc treatments and of

single parameters on glucose level.

2. Building personalized care plans integrated in the

current clinical processes linking the different ac-

tors in primary and secondary care and improving

the active role of the Patient.

3. The combined use of tools for predictive mod-

elling and for the personalisation of the individ-

ual process of care will close the loop between

the Patients, the Professionals involved and the

Health Organisation. Mining the data produced

by METABO will allow the identiﬁcation of pat-

terns and trends that will allow the ﬁne tuning of

the model and the prompt adjustment of the pro-

cess of care.

METABO consists of a global platform that col-

lects and processes data coming from the patient and

the physicians’ tools (a mobile device for the patients

to acquire data from user and sensors and a web ap-

plication for the physicians to present them all data

collected and analyzed) and works as an information

exchange bridge between physicians and patients. On

HEALTHINF 2010 - International Conference on Health Informatics

368

top of this, the system provides both groups of users

with decisions support systems to give them recom-

mendations in a personalized short loop and an inte-

grated long loop Figure 2.

Figure 2: Diagram of METABO Platform (after METABO

Annex I ”Description of Work”).

2.2 In-vehicle Hypoglycemia Alerting

System

In-vehicle hypoglycemia alerting system (IHAS) is a

special case-study in the METABO, which aims at de-

signing and building an in-vehicle version of the sys-

tem to provide metabolic monitoring and preventive

support to drivers, especially to those suffering from

sudden and/or unaware hypoglycemia. This research

is innovative in that it aims to measure and predict

hypoglycemia events indirectly by using physiologi-

cal sensors and by analyzing driver’s behavior behind

the wheel and the change of emotional states. The hy-

pothesis is that hypoglycemia will affect the both the

bio-proﬁle of the patient and his/her driving behavior.

The IHAS consists of four subsystems, i.e. behavioral

monitor, emotive monitor, healthcare state and physi-

ological state:

Behavioral Monitor. This subsystem measures

driving-relevant signals such as steering wheel an-

gle, vehicle speed, lateral and longitudinal accelera-

tion, brake usage, etc. These will then be used to

develop a module able to evaluate driver’s behav-

ior. The ﬁnal scope will be to identify pattern and

trends in these signals which can be correlated to hy-

poglycemic events in order to alert the driver to the

forthcoming hypoglycemia.

Input: car signals available on the CAN networks.

Output: indexes quantifying driver behavior focusing

on those behaviors.

Emotive Monitor. This subsystem is responsible

for recognizing driver’s emotional states by using

multichannel physiological signals and visual infor-

mation. This task includes to verify predictability of

glucose level changes based on actual emotional state

under the condition of a short-term observation and

to provide the driver a possibility of emotion manage-

ment training through biofeedback. Main challenge

of the system is to ﬁnd interferential correlation be-

tween driver’s emotional state and the change of glu-

cose level.

Input: multichannel physiological signals and visual

information.

Output: driver’s emotional states and prediction of

hypoglycemic events.

Healthcare Monitor. This subsystem analyzes di-

etry, physical activity, treatment, and glycemic data

and predicts the metabolic status of the diabetic driver

in the short- and medium run. The data will be col-

lected by using patient’s mobile device (PMD) and

continuous glucose monitoring system (CGMS).

Input: treatment history, blood glucose values, insulin

intake, food intake, physical activity

Output: predicted blood glucose level and recommen-

dations.

Physiological Monitor. This module analyzes the

same physiological signals used for emotive monitor

in order for assessing the physiological state of the

driver. Particularly, it mainly focuses on extracting

cardiac features from electrocardiogram and blood

pressure.

Input: multichannel physiological signals

Output: drivers physiological states focusing on hy-

poglycemia detection.

3 IN-VEHICLE EMOTION

MONITORING

Figure 3 shows the frame work of in-vehicle emotion

monitoring (IEM) system.

3.1 Biosensors

We collect the physiological signals by using the

ME6000

with four biosensors, electromyogram

(EMG), skin conductivity (SC), electrocardiogram

(ECG), and respiration (RSP). The typical waveforms

and sensor positions are illustrated in Figure 4.

This is an 16 channel multi-modal Biofeedback sys-

tem with 14 bit resolution in sampling rate of 2000 Hz.

www.megaemg.com

IN-VEHICLE MONITORING OF AFFECTIVE SYMPTOMS FOR DIABETIC DRIVERS - In-vehicle Hypoglycemia

Alerting System in EU Project METABO

369

ACQUIRE

PREPROCESS

ANALYSE

FUSION

MAPPING

Rule-based

Mapping

Statistical

Classifier

ECG

GSR

EMG

RSP

Knowledge Bases:

sensor priority, calibration profile, context data, hypoglycemia symptoms

wireless

Behavioral/physilogical

hypoglycemia symptoms

General

affective states

Meta

information

Noise Reduction

Lowpass and Bandpass Filtering

Spectral Analyse, Discrimination

Feature, Pattern Extraction

Figure 3: Framework of physiological emotion recognition.

!""#

$"""#

!%""#

!"""#

##%""#

#%$&'#

#%$&(#

#%$&)#

#%$&*#

#%$&%#

####!+#

$*""# #

$,""# #

$$""# #

$"""# #

####!$#

####!!#

####!"#

######'#

######(#

######"#

#-%""#

"# $""# +""#

+""#

$%""#

*"""#%"""#,"""#+"""#$"""#!"""#"#

$"""#!%""#!"""#%""#"#

$%"#$""#!%"#!""#%"#"#

,""# %""# *""#

./0#

.10#

.20#

.30#

Position Typical Waveform

Interbeat Interval

Breath Amplitude

Figure 4: Position and typical waveforms of the biosensors:

(a) ECG, (b) RSP, (c) SC, (d) EMG.

3.2 Feature Extraction

In ofﬂine condition, we have a variety of choices

for applying signal analysis techniques to obtain rele-

vant features. In the previous work (Kim and Andr

2008), we proposed a wide range of physiological fea-

tures from various analysis domains including time,

frequency, entropy, geometric analysis, subband spec-

tra, multiscale entropy, and HRV/BRV in order to

search for the best emotionrelevant features. In the

work, we achieved an average recognition accuracy

of 95% from a naturalistic dataset obtained from a

reliable experiment using a musical induction, which

was not based on any lab setting or any deliberate in-

structions for evoking certain emotions. The recog-

nition accuracy of 95% for four emotions (joy, anger,

sadness, pleasure) connotes more than a prima facie

evidence that there are some ANS differences among

emotions.

For online systems, the choice of features is re-

stricted to those that can be calculated possibly in re-

altime or near realtime at least. Therefore the effec-

tive use of feature selection and realtime signal pro-

cessing techniques plays an important role. Based on

the features in the previous work, we select diabetes-

speciﬁc features for the IEM system.

3.3 Mapping Features with Emotions

In addition to the typical driving-relevant emotions

such as anger, stress, anxiety, exciting etc., we

classify physiological and behavioral symptoms of

hypoglycemic events. Driving-relevant emotional

states and hypoglycemic symptoms can be summa-

rized as follows:

Driving-relevant emotions:

- Anger: leads to horn-honking, rapid steering and

accelerating

- Stress (in multiple levels): related to trafﬁc situa-

tion (e.g. rush hour) and other road user.

- Anxiety, calm, excitation etc.

Diabetes-speciﬁc emotions:

- Stress: blood sugar release is symptomatic of

stress

- Fear of hypo-/hyperglycemia

- Depression: possibly because of imperfect rela-

tionship between self-care & health, or combina-

tion of acute & chronic stressors.

- Anger, nervous, anxiety

Signs of hypoglycemia:

- Mild: tremor, sweating, tachycardia, nervousness,

heart palpitations, hunger

- Moderate: shaking, dizzy, headache, confusion,

numbness of lips or extremities, cold, clammy

skin, slurred speech, hyperventilation or shortness

of breath

- Severe: disorientation, seizure, coma, uncon-

scious

Based on these factors, we consider a novel emotion

model combined with hypoglycemic symptoms (Fig-

ure 5) for the IEM system.

In addition to the statistical classiﬁers such as sup-

port vector machines, k-nearest neighbor, neural net-

works, etc. that are commonly used in pattern recog-

nition, we develop a rule-based mapping method

in which we generate rules for mapping particular

features linearly to the change of certain emotional

states. We employ this method especially for recog-

nition of hypoglycemic symptoms, such as shaking,

sweating, cold, tremor, rapid breathing and tachycar-

dia, which can directly be detected by analyzing linear

variation of related biosignals, for example, sweating

by GSR, rapid breathing by RSP, tremor by EMG,

etc. Therefore, the rule-based mapping algorithm can

easily be implemented for realtime system without

HEALTHINF 2010 - International Conference on Health Informatics

370

!"#$%&'($)*+,-./'-

.#,+&')'0&1$)*+,-./'-

#&0#*$('2+$)

)'3*$('2+$)

.'+&/&%"*%$)"41"

4"0$/&%"*%$)"41"

!"#$#%!&

!'&()%*+

#($%,-.#&()/%*+

(*+&#

)(0/10(#,%(

023,

02*)&*)4&*)

)%#&,

!3&&$%*&!!

0(34

521

!/(6%*+

)#&42#

7&(#

,&$#&!!%2*

(*8%&)1

!)#&!!

980%)&4&*)

Figure 5: Discrete emotion model combined with hypo-

glycemia symptoms.

training a classiﬁer. For the other emotions such as

stress, joy, anger, and neutral, the recognition system

needs to be learned from cross-correlated behavior of

features from multichannel biosignals and requires a

training dataset for supervised machine learning. As

a result, we will need to develop a combined classiﬁ-

cation scheme of both rule-based mapping and super-

vised classiﬁcation in order for recognizing multiple

symptoms and emotional states. An ensemble method

can also be considered for improved recognition ac-

curacy, e.g. classiﬁer ensemble method in which we

classify by using different classiﬁers and combine the

results from each classiﬁer by using decision-level fu-

sion algorithms such as majority voting, Borda count,

and boosting.

Main challenge in the IEM system is overcoming

possibly overlapping physiological reactions between

hypoglycemic events and common emotional states

and calibrating the biosensors in driving situation.

4 DISCUSSION: IMPACT OF

EMOTIONS ON DIABETES (&

VICE VERSA)

Patient’s emotional needs and problems are an impor-

tant component of treatment and integral component

of diabetes management. The role of emotions in dia-

betes management was observed as early as the seven-

teenth century, when British physician Thomas Willis

noted that diabetes ﬁrst appeared among patients who

had experienced signiﬁcant life stresses.

Recently it is widely recognized that negative

emotions such as stress, anxiety, fear, depression,

and sorrow affect the blood glucose level (Surwit and

Schneider, 1993). Patients experiencing such negative

emotional states may have greater difﬁculty in con-

trolling blood glucose compared to those not suffer-

ing emotional problems. Depression, for example, is

not generally listed as complication of diabetes. How-

ever, it can be one of the most common and dangerous

complications. More importantly, depression under-

mines the motivation of patient to maintain diabetic

management. Diabetics with major depression have a

very high rate of recurrent depressive episodes within

the following ﬁve years (Lustman et al., 1997b). In

the study of (Lustman et al., 1997a) it is proven that

effective treatment of depression can improve diabetic

control. Stress can also readily elevate blood glucose

and affects the autonomic nervous system, which in

turn affects the secretory rate of insulin and glucagon

and ﬁnally disrupts metabolic control. Stabler and

colleagues (Stabler et al., 1987) found that children

judged to have a ”Type A” personality structure had

an increased blood glucose elevation in response to

stress and children with a calmer disposition had a

smaller glucose rise when stressed. Several studies

have demonstrated a relationship of stress to glycemic

control in samples of patients with Type 1 diabetes

(Inui et al., 1998; Viner et al., 1996). Stress can

be managed, for example, by using behavioral stress

management program such as progressive muscle re-

laxation (PMR), or the administration of anxiolytic

medications. Recently, the study in (Surwit et al.,

2002) supported the efﬁcacy of outpatient stress man-

agement training for the improvement of glycemic

control in patients with Type 2 diabetes. For stress

management training, they used the PMR and medi-

cation instruction methods complementarily. The re-

sult of their experiment showed that at the end of a

1-yea follow-up period, patients who received train-

ing in stress management skills demonstrated approx-

imately a 0.5% reduction in HbA1c relative to control

patients. While positive emotions such as laughter

have been reported to modify the levels of neuroen-

docrine factors involved in negative emotions and to

modulate immune function (Berk et al., 1989; Taka-

hashi et al., 2001), less attention has been so far paid

to impact of positive emotions on diabetes. Through

the observation in (Hayashi et al., 2003) it is ﬁrstly

elucidated that laughter can suppress the elevation of

blood glucose level. From their 2-day experiment, it

turned out that the patients with Type 2 diabetes had

a smaller rise in post-meal blood glucose when they

watched a comedy show than when they listened to a

humorless lecture.

IN-VEHICLE MONITORING OF AFFECTIVE SYMPTOMS FOR DIABETIC DRIVERS - In-vehicle Hypoglycemia

Alerting System in EU Project METABO

371

5 CONCLUSIONS

Research on the role of emotions in diabetes is still

challenging work, because literature so far offers

ideas rather than well-deﬁned solutions. In this paper

we presented conceptual scheme of in-vehicle emo-

tion monitoring system for diabetic drivers. Many

works remain to be done in the project METABO.

To achieve our goals in this new area we conceived,

it requires not only a methodological, technical in-

novation but also conceptual changes with workable

thoughts focusing on speciﬁc contexts of medical ap-

plications.

ACKNOWLEDGEMENTS

The work described in this paper is funded by the EU

under research grant ICT-216270-METABO.

REFERENCES

Armitage, C. J., Conner, M., and Norman, P. (1999). Dif-

ferential effects of mood on information processing:

evidence from the theories of reasoned action and

planned behaviour. European Journal of Social Psy-

chology, 29:419–433.

Berk, L., Tan, S., Fry, W., Napier, B., Lee, J., Hubbard, R.,

Lewis, J., and Eby, W. (1989). Neuroendocrine and

stress hormone changes during mirthful laughter. Am.

J. Med. Sci., 298(390–396).

Deary, I. (1999). Symptoms of hypoglycaemia and ef-

fects on mental performance and emotions. In Fisher,

B., editor, Hypoglycaemia in Clinical Diabetes, pages

29–54. Chichester, U.K., Wiley.

Gold, A., MacLeod, K., Deary, I., and Frier, B. (1995).

Hypoglycemia-induced cognitive dysfunction in dia-

betes mellitus: effect of hypoglycemia unawareness.

Physiol. Behav., 58:501–511.

Hayashi, K., Hayashi, T., Iwanaga, S., Kawai, K., Ishii, H.,

Shoji, S., and Murakami, K. (2003). Laughter lowered

the increase in postprandial blood glucose. Diabetes

Care, 26:1651–1652.

Inui, A., Kitaoka, H., Majima, M., and et al. (1998). Effect

of the kobe earthquake on stress and glycemic control

in patients with diabetes mellitus. Arch. Intern. Med.,

158:274–288.

James, L. (2000). Road Rage and Aggressive Driving.

Amherst, NY: Prometheus Books.

Kim, J. and Andr

e, E. (2008). Emotion recognition based on

physiological changes in music listening. IEEE Trans.

Pattern Anal. and Machine Intell., 30(12):2067–2083.

Lajunen, T. and Parker, D. (2001). Are aggressive peo-

ple aggressive drivers? a study of the relationship

between self-reported general aggressiveness, driver

anger and aggressive driving. Accident Analysis and

Prevention, 33:243–255.

Lustman, P., Grifﬁth, L., Clouse, R., and et al. (1997a). Ef-

fects of nortryptiline on depression and glycemic con-

trol in diabetes: Results of a double-blind, placebo-

controlled trial. Psychosomatic Medicine, 59(3):241–

250.

Lustman, P., Grifﬁth, L., Freedland, K., and Clouse, R.

(1997b). The course of major depression in diabetics.

Gen. Hosp. Psychiatry, 19(2):138–143.

Ohman, A. (2000). Fear and anxiety: Evolutionary, cog-

nitive and clinical perspectives. In Lewis, M. and

Haviland-Jones, J. M., editors, Handbook of Emo-

tions. The Guilford Press, New York.

Stabler, B., Surwit, R., Lane, J., and et al. (1987). Type a

behavior pattern and blood glucose control in diabetic

children. Psychosomatic Medicine, 49:313–316.

Surwit, R. and Schneider, M. (1993). Role of stress in the

etiology and treatment of diabetes mellitus. Psycho-

som. Med., 55:380 –393.

Surwit, R., Tilburg, M. V., Zucker, N., and et al. (2002).

Stress management improves long-term glycemic

control in type 2 diabetes. DIABETES CARE, 25(1).

Takahashi, K., Iwase, M., Yamashita, K., Tatsumoto, Y.,

Ue, H., Kuratsune, H., Shimizu, A., and Takeda, M.

(2001). The elevation of natural killer cell activity in-

duced by laughter in a crossover designed study. Int.

J. Mol. Med., 8:645–650.

Viner, R., McGrath, M., and Trudinger, P. (1996). Family

stress and metabolic control in diabetes. Arch. Dis.

Child., 74:418–421.

HEALTHINF 2010 - International Conference on Health Informatics

372