Data-mining Approaches for the Study of

Emotional Responses in Healthy Controls

and Traumatic Brain Injured Patients:

Comparative Analysis and Validation

F. Riganello

, L.Pignolo

, V. Lagani

and A. Candelieri

S. Anna Institute, RAN – Research on Advanced Neuro-rehabilitation, Crotone, Italy

Laboratory for Decision Engineering and Health Care Delivery

Department of Electronic Informatics and Systemistics

University of Calabria, Cosenza, Italy

Abstract. Relationship between Heart Rate Variability (HRV) and emotions

subjectively reported by 26 healthy subjects during symphonic music listening

have been investigated through Data Mining approaches. Most reliable decision

models have been successively adopted to forecast an emotional assessment on

a group of 16 Traumatic Brain Injured patients during the same type of

stimulation, without algorithms retraining. The most performing decisional

models have been a Rule Learner (ONE-R) and a Multi Layer Perceptron

(MLP) but, comparing them, the first one was the best in terms of reliability

both on validation and independent test phases. Furthermore, ONE-R provides a

simple “human-understandable” rule useful to evaluate emotional status of a

subjects depending only on one HRV parameter: the normalized unit of Low

Frequancy BandPower (nu_LF). Specifically, the classification by HRV nu_LF

matched that on reported emotions, with 76.0% of correct classification; tenfold

cross-validation: 70.2%; leave-one-out validation: 71.1%. On the other hand,

MLP approache has provided an accuracy of 82.69% on healthy controls, but it

has decreased to 47.11% and 46.15% on 10folds-cross and leave-one-out

validation respectively. Finally, the accuracy has resulted in 51.56% when the

MLP model has been applied to the posttraumatic subjects, while the ONE-R

accuracy has resulted in 70.31%. Data mining proved applicable in

psychophysiological human research.

1 Introduction

Data-mining or hybrid techniques are used in medicine to sort significant information

out of large databases in mutagenicity studies, predictive toxicology, disease

classification, selective integration of multiple biological databases, etc. Applications

in neurology have focused on prognostic studies [1-2] or in the classification of

emotional responses [3] . The Heart Rate Variability (HRV) is an emerging objective

measure of the continuous interplay between the sympathetic and parasympathetic

autonomic nervous sub-systems. It is thought to provide information also on complex

Riganello F., Lagani V., Pignolo L. and Candelieri A. (2009).

Data-mining Approaches for the Study of Emotional Responses in Healthy Controls and Traumatic Brain Injured Patients: Comparative Analysis and

Validation.

In Proceedings of the 5th International Workshop on Artiﬁcial Neural Networks and Intelligent Information Processing, pages 125-133

DOI: 10.5220/0002263901250133

 SciTePress

patterns of brain activation, including emotional responses [4-11]. For instance, HRV

abnormalities are reportedly common in psychiatric or brain damaged patients [12-

14]; HRV proved a useful predictor of outcome in brain injured patients [15-16].

HRV studies require quantitative approaches and large numbers of parameters are

generated when the parametric and non-parameric HRV spectra are computed.

In our study we measured several HRV parameters from 26 healthy subjects

during the listening of music samples, suitably selected to induce specific emotional

status. At the end of each listening we asked to report felt emotions, without any

reference to pre-selected categories and to the feelings that the subjects thought the

music was intended to induce. On the basis of these interviews, we identified to

macro-categories of emotions (positive and negative emotions) and defined a two-

classes classification problem. The same procedure has been adopted for acquiring

data from a group of 16 posttraumatic subjects. More details about the two datasets

construction are reported in another our study [3]. Since the great amount of variables

(35 HRV parameters) and the relatively weak consolidated knowledge about the

problem, data mining techniques have represented an effective solution for identifying

a relationship between HRV parameters and emotions, without any preliminary

assumption about data distribution.

We compared different classification learning strategies through suitable

validation techniques, such as 10folds-cross and leave-one-out validation and test on

the independent test set.

2 Material and Methods

2.1 Patients and Controls

Two groups of subjects were studied:

1. twenty six healthy volunteers (14 women; mean age: 31.7±7.1 yrs., age range:

21-45 yrs.; high-school to university education);

2. sixteen patients without severe disabilities completing rehabilitation after severe

traumatic brain injury (5 women; mean age: 21.6±3.0 yrs., age range: 17 to 33

yrs., grammar to high school education); and

Subjects were informed in full detail about the study purpose and experimental

procedures and the ethical principles of the Declaration of Helsinki (1964) by the

World Medical Association concerning human experimentation were followed. The

procedures for data collection and the experimental setting caused no physical or

emotional discomfort and were discontinued whenever the subject felt tired or tense

or at the subject’s request. Controls and posttraumatic subjects had no musical

training (see [3] for methodological detail).

2.2 Stimulus Conditions

Four music samples (Table 1) were selected following characterization by intrinsic

structure and expected emotional response as indicated by the available formal

126

complexity and dynamics descriptors. These descriptors reportedly relate music

structure to self-assessed emotions and allow to characterize the emotional status

along a continuum from euphoria and well-being to melancholy, severe anxiety and

perceived aggressive tendencies [17-18].

Table 1. Selected music samples.

1) Luigi Boccherini: Quintet op. 11 n. 5, Minuetto (duration: 3’ and 50”);

2) Piotr Ilitch Tchaïkovski: sixth symphony, op. 74, first movement (duration: more than 10”);

3) Modest Petrovich Mussorgsky: St. John's Night on the Bald Mountain

(duration: more than 10”);

4) Edvard Grieg: Peer Gynt, op 23, The morning (duration: 4’ and 20”)

Experiments on patients and control took place at the same time of the day in a

familiar environment and did not interfere with the posttraumatic subjects’

medical/rehabilitative schedules. Subjects were comfortably lying on an armchair,

with constant 24º C ambient temperature and in absence of transient noises. They

were exposed binaurally (earplugs) to the four selected music samples balanced for

loudness and played in random sequence to minimize carry-on effects. There was a

20-min rest between consecutive samples to avoid overstimulation and excessive

fatigue.

The heart beat was recorded from the beginning of the music sample and for a

total of 300 beats (3',36"±24", with 83.7± 9.5 beats/min and a resulting total recording

time between 3',12" and 3':55") by means of the Virtual Energy Tester (Elamaya

Instruments, Milano, Italy). The photopletismographic sensors were positioned on the

third phalange of the left hand middle finger in order to minimize the subjects’

discomfort consistent with the guidelines of the Task Force of European Society of

Cardiology and the North American Society of Pacing and Electrophysiology[19].

The photopletismographic signal was sampled at 100 samples/sec; the series of

consecutive intervals between heart beats was analyzed in the time and frequency

domains by the HRV advanced analysis software developed at the Department of

Applied Physics, University of Kuopio, Finland [20]. The non-parametric (Fast

Fourier Transform, Welch spectrum) and parametric (autoregressive) spectra were

computed (Table 2). The power spectral density from 0.01 Hz to 0.5 Hz was

computed with 0.001 Hz resolution and three frequency ranges (very low frequency

[VLF]: 0.01-0.04 Hz; low frequency [LF]: 0.04-0.15 Hz; and high frequency [HF]:

0.15 Hz – 0.5 Hz) were considered (Table 2).

At the end of each music sample, controls and posttraumatic subjects were

requested to classify their emotions, without reference to any pre-selected categories

and irrespective of the emotional feeling they thought the music was intended to

induce. The distribution of the emotions expressed for each music sample was

determined [17-18].

127

Table 2. Spectral and Statistical Parameters.

Statistical Parameters Spectral Parameters

Mean RR interval (Mean RR) and SD

(STD RR);

Mean Heart Rate (Mean HR) and SD

(STD HR);

Root mean square of SD (RMSSD);

number (NN50) and percentage

(pNN50) of NN intervals longer than

50 ms

Very Low Frequency (VLF), Low Frequency (LF), High

Frequency (HF) and normalized unit (nu) in FFT and

autoregressive spectra

VLF Peak frequency in FFT and autoregressive spectra

LF Peak frequency in FFT and autoregressive spectra

HF Peak frequency in FFT and autoregressive spectra

Power Spectrum of VLF, LF, HF and Total in FFT and

autoregressive spectra

% of VLF, LF, HF in FFT and autoregressive spectra

Ratio LF/HF, nu LF, nu HF, nu LF/HF in FFT and autoregressive

spectra

2.3 Data Mining Techniques

We adopted several classification approaches for identifying an association between

HRV parameters and the self-assessed emotions. To this aim, two different classes or

categories were defined for the reported emotional status: positive (happiness, joy,

serenity, calm,…) and negative (fear, anxiety, tension, scare,…). The control group

was selected as training set for several data-mining classification techniques, such as

Decision Trees, Support Vector Machines, Artificial Neural Networks and Rules

Learner, provided by the WEKA open source software (Waikato Environment for

Knowledge Analisys) [21, 22].

Training set included 104 cases (26 healthy subjects x 4 music samples) and 35

variables (Table 2). In a first step, two internal validation techniques based on training

data (namely the 10 folds-cross and leave-one-out validation) were used to evaluate

how much extracted models would fit the new data.

Most reliable decision models have been selected and adopted to forecast an

“emotional status assessment” on the independent test set (16 posttraumatic subjects).

The models used to obtain this assessment on the test set has been not “re-learned”

from it. The independent test set included 64 cases (16 posttraumatic subjects x 4

music samples) and the aforementioned 35 HRV parameters.

In this way, we have been able to estimate “real” reliabilities for the selected

models. Figure 1 shows how we have designed our study.

Since every classification technique adopts a different structure for explaining the

relationship between input variables and output (HRV parameters and subjectively

reported emotions, respectively in our study) it may be really difficult to select a

priori a strategy for learning the relationship from the available data. For such reason

we decided to adopt several classification learning procedures applicable to our

problem, such as Decision Trees, Support Vector Machines, Rule Learners and

Artificial Neural Networks, all available in WEKA.

128

Fig. 1. Phases of the Study.

Respect to 10fold-cross and leave-one-out validation, we have obtained that the

most reliable classification techniques were ONE-R [23] and Multi Layer Perceptron

[24]. The first one is a Classification Rule Learner and works in a very easy way: it

accepts a training set as input and searches for a “1-rule” classifying instances on the

basis of a single variable (attribute). Initially, the algorithm ranks attributes according

to error rate on the training set then, if the attribute is numerical, the algorithm divides

the range of possible values into several disjoint intervals. In order to avoid

overfitting (that is when each interval contains only a value belonging to one class)

ONE-R permits to set a parameter named “bucket-size” that is the minimum number

of instances in an interval. We obtained the best performing ONE-R configuration

setting 7 as minimum number of instances in one interval. In figure 2 we report the

extracted rule (note that nu_LF can belong to only one interval, so only one prediction

is possible).

Fig. 2. Decision model for emotional status assessment learned through ONE-R algorithm on

healthy controls.

If nu_LF is in

[0; 29.35)

[29.45; 46.37)

[46.37; 64.25)

[64.25; 72.45)

[72.45; 100]

negative emotion predicted

positive emotion predicted

negative emotion predicted

positive emotion predicted

negative emotion predicted

[ . ; . ) inferior extreme included – superior extreme excluded

129

Multi Layer Perceptron (MLP) is a particular Artificial Neural Network (ANN)

architecture also known as “feed forward architecture”, and it is composed by, at the

least, three different layers: input layer, hidden layer and output layer. Generally, the

hidden layer can be more then one and also the number of neurons into hidden layers

can also vary, where a neuron is the “simple” processing unit of the net.

As each ANN, MLP is a mathematical/computational model based on biological

neuronal networks and is commonly applied to model complex input/output

relationships or to identify data patterns of distribution/correlation. Structurally, an

ANN is composed by a set of neurons, linked each other by a large number of

(usually nonlinear) weighted connections. Each neuron is able to calculate a specific

function, given the inputs and the weights on the connections are adjusted in order to

minimize some criteria as, usually, the errors number.

Using MLP we have obtained the best results setting up the following net

parameters: only one the hidden layer with 5 neurons, Learning Rate=0.2,

Momentum=0.1 and 1000 training epochs. Moreover we have previously ranked the

variables by preprocessing the dataset by ONE-R WEKA Filter, which evaluates the

worth of an attribute by using the ONE-R strategy. Finally, the best results for MLP

were obtained selecting the first 8 ranked attributes (nu_LF, powerHF, STDRR,

gender, powerVLF, peakVLF, peakLF and peakHF). The MLP architecture is showed

in figure 2.

Fig. 3. Multi Layer Perceptron architecture.

3 Results

ONE-R procedure sorted out the nu_LF descriptor as the more significant spectral

parameter in the selected experimental conditions and for the study purpose. The

ONE-R allowed a good trade-off between the portion of correct sample/label

association on the entire training set (recognition) and the portion of correct

sample/label association on validation and independent test phases (generalization).

In table 3 we summarize our results both for ONE-R and MLP. For the first one,

an overall correct classification on training set (healthy controls) has been obtained on

130

about 76.0% of the instances, 81.0% and 69.0% of negative and positive emotions,

respectively; (tenfold cross-validation: 70.2%; leave-one-out validation: 71.1%).

When applied to the independent test set (posttraumatic subjects), the

classification accuracy performed by the ONE-R decision model was comparable:

70.3% of correct classification on the entire test set, 65% and 74% of positive and

negative emotions, respectively.

Despite the greater accuracy of the MLP on the entire training set (82.7% vs.

76.0%, for MLP and ONE-R respectively), MLP accuracy decreased to 47.11% and

46.11% on 10folds-cross and leave-one-out validation phases respectively. Moreover,

also the correct attribution for each class of emotion has decreased: 33.33% and

57.63% for positive and negative emotions, respectively, on 10folds-cross validation,

and 26.67% and 61.02% for positive and negative emotions, respectively, on leave-

one-out validation.

Furthermore, also on independent test phase MLP has provided lower reliability

than ONE-R approaches: 70.31% of accuracy for ONE-R versus 51.56% for MLP

(32% and 64.10% for positive and negative emotions respectively). All the results are

summarized in Table 3.

Table 3. Results One-R vs MLP.

Analysis of data: ONE-R vs MLP

On training data 10 Folds-cross Leave-one-out Independent Test Set

OneR

75.96% 70.19% 71.15% 70.31%

MLP

82.69% 47.11% 46.15% 51.56%

Attribution of positive emotions

On training data 10 Folds-cross Leave-one-out Independent Test Set

OneR

68.89% 57.78% 64.44% 68.00%

MLP

68.89% 33.33% 26.67% 32.00%

Attribution of negative emotions

On training data 10 Folds-cross Leave-one-out Independent Test Set

OneR

81.36% 79.66% 76.27% 71.79%

MLP

93.22% 57.63% 61.02% 64.10%

4 Comment

Although the HRV is an emerging objective measure also in neurophysiology, there is

still a lack of knowledge about a possible and tangible relationship between heart

activity and the emotional status assessment. On the other hand, Data Mining

techniques may offer practical advantages for analyzing data, also medical [25, 26],

and they have proved useful analysis tools in our study, searching for the most

reliable and frequent relationships between HRV parameters measured during

131

symphonic music listening and emotions subjectively reported by a group of 26

healthy subjects at the end of each listening. Several data mining methods have been

investigated and evaluated through the most suitable validation techniques. Reliability

of resulting relationships has been then tested on an independent test group of 16

posttraumatic patients, without algorithm retraining.

ONE-R algorithm (a classification rule learner) has provided the best

performances and reliability, identifying a single HRV parameter (notably the nu_LF

measure) as the most relevant for assessing the emotional status, both for healthy

controls and posttraumatic subjects.

In this study ONE-R proved more effective then the best MLP configuration and

provided a simple “if…then” rule. Furthermore, this rule can be easily applied, in

combination with the non-invasive technique for HRV data acquisition (by a

photopletismographic sensor), to evaluate the emotional conditions of unconscious

subjects (such as subjects in vegetative state) in order to establish, in a more objective

way, when is better to continue or interrupt any contact or stimulation.

References

1. Herskovits, H. E., Joan, P. G.: Application of a data-mining method based on Bayesian

networks to lesion-deficit analysis. 19(4) (2003) 1664-1673.

2. Chen, S., Haskins, E. W., Ottens, K. A., Hayes, L. R., Denslow, N., Wang., K. W. K.,:

Bioinformatics for Traumatic Brain Injury: Proteomic Data Mining.: Data Mining in

Biomedicine. 7 (2008) 363-387.

3. Riganello, F., Quintieri, M., Candelieri A., Conforti, D., Dolce G.: Heart Rate Variability:

an artificial intelligence study on healthy and traumatic brain injured subjects: Journal of

Psychophysiology. 22(3) (2008) 166-174.

4. Lafranchi, P. A., & Somer, V.K.: Arterial baroflex function and cardiovascular variability:

Interactions and implications. American Journal of Physiology – Regulatory Integrative and

Comparative Physiology. 283 (2003) R815–R826.

5. Malpas, S. C.: Neural influences on cardiovascular variability: Possibilities and pitfalls.

American Journal of Physiology – Regulatory Integrative and Comparative Physiology. 282

(2002) H6–H20.

6. Nikki, S. R.: Intense emotional response to music: A test of the physiological arousal

hypothesis. Psychology of Music. 32 (2004) 371–388.

7. Urakawa, K., & Yokoyama, K.: Music can enhance exercise- induced sympathetic

dominancy assessed by HRV. Tohoku Journal of Experimental Medicine. 205 (2005) 213–

218.

8. Mashin, V. A., & Mashina, M. N.: Analysis of the HRV in negative functional states in the

course of the psychological relaxation session. Human Physiology. 26 (2000) 420–425.

9. Critchley, H.D., Rotshtein, P., Nagayi, Y., O’Doherty, J., Mathias, C.J., & Dolan, R.:

Activity in the human brain predicting heart rate response to emotional facial expression.

Neuro- Image. 24 (2005) 751–762.

10. Fraizer, T. W., Strauss, M.E., & Steinhauer, S. Respiratory sinus arrhythmia as an index of

emotional response in young adults. Psychophysiology. 41 (2004) 75–83.

11. Appelhans, B. M., & Luecken, L.J.: Heart rate variability as an index of regulated

emotional responding. Review of General Psychology. 10 (2006) 229–240.

12. Cohen, H., Benjamin, J., Geva, A.B., Matar, M.A., Kaplan, Z., & Kotler, M.: Autonomic

dysregulation in panic disorder and in posttraumatic stress disorder: Application of power

132

spectrum analysis of heart rate variability at rest and in response to recollection of trauma

or panic attacks. Psychiatry Research. 96(1) (2000) 1–13.

13. Draper, K., Ponsford, J., & Schönberger, M.: Psychosocial and emotional outcomes 10

years following traumatic brain injury. Journal of Head Trauma Rehabilitation. 22 (2007)

278–287.

14. Scholten,M. R., van Honk, J., Aleman,A.,&Kahn, R.S.: Behavioral inhibition system (BIS),

behavioral activation system (BAS), and schizophrenia: Relationship with psychopathology

and physiology. Journal of Psychiatric Research. 40 (2006) 638–645.

15. Briswas, A. K., Scott, W.A., Sommerauer, J.F., & Luckett, P.M.: Heart rate variability after

acute traumatic injury in children. Critical Care Medicine. 28 (2000) 3907–3912.

16. Keren, O., Yapatov, S., Radai, M.M., Elad-Yarum, R., Faraggi, D., Abboud, S.: Heart rate

variability of patients with traumatic brain injury during postinsult subacute period. Brain

Injury. 19 (2005) 605–611.

17. Imberty, M.: Epistemic subject, historical subject, psychological subject: Regarding

Lerdhal and Jackendoff ’s generative theory of music. In I. Deliege & J.A. Sloboda (Eds.),

Perception and cognition of music Hove, UK: Psychology Press. (1997) 429–432.

18. Tarasti, E.: A theory of musical semiotics. Bloomington, IN: Indiana University Press.

(1994).

19. Task Force of the European Society of Cardiology and the North American Society of

Pacing and Electrophysiology of Circulation.: Heart rate variability: Standard of

measurement, physiological interpretation, and clinical use. Circulation. 93 (1996) 1043-

1065.

20. Niskanen, P.J., Tarvainen , M.P., Ranta-aho, P.O., & Karjalainen P.A.: Software for

advanced HRV analysis. University of Kuopio Department of Applied Physics. Computer

Methods and Programs Biomedicine. 76(1) (2004) 73-81.

21. Witten, H.W., & Eibe, F.: Data mining – Practical machine learning tools and techniques

with Java implementations. San Francisco, CA: Morgan Kaufman. (2005).

22. Eibe, F.: Machine learning with WEKA. Department of Computer Science, University of

Waikato, New Zealand. (2004). http://puzzle.dl.sourceforge.net/sourceforge/

weka/weka.ppt

23. Holte, R.C.: Very simple classification rules perform well on most commonly used

datasets. Machine Learning. 11 (1993) 63–90.

24. van Bemmel, J.H.,&Munsen, M.A.: Handbook of medical informatics. Berlin: Springer-

Verlag. (1997).

25. C. Robert, C.D. Arreto, J. Azerad, J.F. Gaudy. Bibliometric overview of the utilization of

artificial neural networks in medicine and biology. Scientometrics, (2004) Vol. 59, No. 1

(117.130).

26. ChungK Lee, SK Yoo, YoonJ Park, NamHyun Kim, KeeSam Jeong, ByungChae Lee

(2005). Using Neural Network to Recognize Human Emotionsfrom Heart Rate Variability

and Skin Resistance, Proceedings of the 2005 IEEE Engineering in Medicine and Biology

Annual Conference Shangai, China, September 1-4, 2005.

133