Stress Dichotomy using Heart Rate and Tweet Sentiment
Jaromír Salamon, Kateřina Černá and Roman Mouček
Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia,
Universitni 8, Pilsen, Czech Republic
Keywords: Sentiment Extraction, Heart Rate, Time Series, Data Analysis, Stress Dichotomy.
Abstract: Automated detection of human stress from markers is very beneficial for the development of assistive
technologies. Blood pressure, skin temperature, galvanic skin response or heart rate are typical physiological
markers that help identify human stress. However, not only the human body itself but also the human mood
expressed in short text messages can be a useful source of such information about stress. This paper focuses
on detection of human stress using two different but synchronized sources of information, human heart rate
and sentiment extracted from tweets. During the preliminary experiment lasting for two fifty-day periods, we
obtained simultaneously 481 708 heart rate data samples from two wearables and sentiment from 2049 tweets.
The tweet data contain a subjective sentiment evaluation that was recorded using positive and negative
hashtags. A few states of stress were identified as the result of the data processing. The final discussion
provides conclusions and recommendations for future research.
1 INTRODUCTION
Detection of stress by using various techniques and
methods is a complex problem that many research
teams currently focus on. In our work, we build on
their results and move it further.
We have designed an experiment and measured
two following features: heart rate that gives us
information on when stress occurs and human mood
extracted as the sentiment from tweets.
These two information sources provide us with
another view on stress dichotomy. When we know
what kind of stress and when it occurs many
applications with a medical base might be designed.
Our motivation and original idea which we want
to expand and evaluate is coming from two cases of
medical research which have used SMS as the
treatment method in both cases and questionnaire
(Montes J. M., 2012), or assessment and survey as the
output (Agyapong V. I. O., 2015).
We think such methods can be enhanced by
measured values or stress dichotomy to determine
whether to continue with therapy when the subject is
in a relax state or stop it when the subject is stressed
out.
2 STATE OF THE ART
Stress is a mental state stemming from tension or
demanding circumstances. These circumstances or
tension is happening for different reasons (Mitra,
2008).
The paper published already in 1975 (Selye, 1975)
provides a model dividing stress into eustress and
distress. When the stress has positive effects such as
tough training or challenging work it might be
considered as eustress. On the other hand, the stress
leading to anxiety or depression is considered as
negative stress named distress.
The most commonly used physiological markers
of stress are as follows (Vassanyi I., 2016):
▪ Galvanic skin response (GSR): using changes
in skin conductivity. During stress, the
resistance of skin drops increases due to
secretion of sweating glands (Shi Yu D., 2007).
▪ Electromyogram (EMG): measuring the
electrical activity of the muscles. Stress causes
differences in the contraction of the muscles
which can be used to identify stress (Melin B.,
1994), (Wijsman J., 2010).
▪ Skin temperature: changes in temperature of
the skin are related to the stress level (Tanaka
S., 2008).
Salamon, J.,
ˇ
Cerná, K. and Mou
ˇ
cek, R.
Stress Dichotomy using Heart Rate and Tweet Sentiment.
DOI: 10.5220/0006650105270532
In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 5: HEALTHINF, pages 527-532
ISBN: 978-989-758-281-3
Copyright © 2018 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
527
▪ Electrical activity of the heart: the most
commonly used stress marker parameters are
derived from the electrocardiogram (ECG),
heart rate (HR) and heart rate variability (HRV)
(Spaepen A., 2009), (Schubert C., 2009).
▪ Respiration: acute stress causes changes in the
breath rate (Choi J., 2009)
▪ Blood pressure: stressors induce an increase in
the blood pressure compared to the baseline
(Blangsted A. K., 2004).
Stress can be identified from HR or HRV using a
variety of techniques and methods, for instance:
▪ Wearable monitors measure HR transformed to
HRV that is processed by principal dynamic
modes (PDM). By using machine learning
classification, the stressful events were
discriminated with a success rate of 83 %
within subjects and 69 % of subjects (Choi J.,
2009).
▪ A camera captures facial landmarks which are
turned into physiological parameters (HR and
HRV). Then the restful and stressful states,
based on training and testing of a dataset used
by a machine learning classifier, were predicted
(Mcdu D., 2014).
In our ongoing research, we have been working
with the HR captured from wearable monitors. This
can be later transformed to HRV, but at the moment
the HR itself is used. A piece of implicit information
about the participant’s mood recorded as a text
(tweet) is available at the same time together with the
HR data. It is enhanced by an explicit evaluation of
the participant who marks his/her mood as positive or
negative at the moment the text (tweet) is recorded.
The use of the machine learning method for text
classification is considered for further application.
From two datasets mentioned above and with the
assumption that HR increases when healthy subjects
are acutely stressed (Schubert C., 2009) the
combination of a few states that includes
increasing/decreasing HR and positive/negative
sentiment can be identified. It could be used together
with the eustress and distress definition (see above)
for further research and application.
3 EXPERIMENT AND DATA
3.1 Experiment Description
Two parts of the experiment were held in two fifty-
day periods using two wearables for the heart rate
(HR) measurement:
▪ Fitbit Charge HR
▪ Basis Peak
During this time, not only the HR was recorded 24
hours a day, but also the text representing mood and
recorded through the Twitter was obtained in two-
time windows:
▪ 7:30 AM CET to 0:00 AM CET during
working days,
▪ 9:00 AM CET to 0:00 AM CET during
weekdays.
The text data contain the text itself and own
subjective evaluation of the sentiment recorded using
the hashtags #p for positive and #n for negative
sentiment.
3.2 Data Description
The output of the experiment are two pairs of datasets
(the HR data and tweets representing the sentiment
for each wearable):
▪ 9:00 AM CET to 0:00 AM CET during
weekdays.
▪ Experiment #1 using Fitbit Charge HR
o 1029 tweets with the average of 20.56
tweets per day,
o 411 799 HR records with the frequency of
6 - 7 records per minute
▪ Experiment #2 using Basis Peak
o 1017 tweets with the average of 20.32
tweets per day,
o 69 909 HR records with the frequency of
1 record per minute
3.3 Sentiment Extraction
In a real application, it is assumed to use an
unsupervised sentiment classification method (for
instance the classification method described in
(Joulin A., 2016)). Since the work is primarily
focused on stress dichotomy analysis and not on
sentiment extraction, the subjective evaluation of the
sentiment is used.
4 ANALYSYS
Since the data obtained from the experiment are two
time-series with different granularity, this has to be
solved by using the following procedures:
▪ to aggregate or reduce the HR data to the same
granularity as the sentiment data has,
▪ to interpolate the sentiment data to achieve the
same granularity as the HR data has.
HEALTHINF 2018 - 11th International Conference on Health Informatics
528
Since the heart rate data is supposed to carry
original information about stress, the sentiment data
needs to be interpolated.
4.1 Sentiment Interpolation
During the sentiment interpolation process, it is
necessary to de ne when the sentiment is supposed to
be valid. Based on that several interpolation
approaches can be used.
4.1.1 Interpolation by Splitting the Interval
The first option is to split the interval between two
neighbourhood sentiment values and perform their
interpolation (see Figure 1).
Figure 1: Sentiment interpolation by splitting the interval,
the original sentiment is represented by the black points,
each splitting of the interval between two black points is
represented by the corresponding red points.
The original sentiment is represented by the black
points in the graph. The interpolated sentiment is
represented by the corresponding red points A and B
calculated as
(1)
(2)
where is a timestamp, represents information
about the sentiment and
.
4.1.2 Interpolation by Moving Window
The second option is to de ne a window around each
sentiment occurrence and perform the interpolation
process inside this window (see Figure 2).
Figure 2: Sentiment interpolation using the moving window
around the sentiment occurrence, the original sentiment is
represented by the black points, the start and end points
obtained by the interpolation are represented by
corresponding red points.
The original sentiment is represented by the black
points in the graph. The interpolated sentiment is
represented by the corresponding red points A and B
calculated as
(3)
(4)
where is a timestamp, represents information
about the sentiment and is the length of the
interpolation window (that is set to 30 minutes in our
experiment).
4.1.3 Interpolation Notes
The first interpolation method results in an extended
time window whenever the sentiment does not
change. The second interpolation method is
dependent on the width of the time window set around
the sentiment occurrence. It is necessary to use the
split interval approach (1) and (2) to overlap
interpolation windows (3) and (4).
4.2 HR Linear Regression
When both data sources are properly combined the
heart rate data can be simplified to express a
decreasing or increasing trend. Simple linear
regression (5) can be used to extract the trend from
the HR data. The application of such statistical
method assumes that
(5)
which describes a line with the slope and y-
intercept . From this equation, we can take slope,
respective its signature as the HR trend (decreasing
Stress Dichotomy using Heart Rate and Tweet Sentiment
529
for the negative slope and increasing for the positive
slope).
The results depicting the application of the simple
linear regression to the HR data together with the
interpolation of the sentiment by splitting the interval
and interpolation of the sentiment by using the
moving window are available in Figure 3 and Figure
4 respectively.
Figure 3: Splitting the interval sentiment interpolation
combined with the HR data linear regression.
Figure 4: Moving window sentiment interpolation
combined with the HR data linear regression.
4.3 Stress Dichotomy
Some approaches to stress identification have been
already described in Section 2. Whenever HR
increases, the subject could be in acute stress.
The primary goal of this work is to determine
whether this stress is positive or negative. The
combination of the interpolated sentiment with the
trend of the HR data could help us to determine stress
dichotomy. Using this approach stress and relax states
can be identified from the heart rate trend and on top
of that stress dichotomy can be identified from the
sentiment (see Table 1).
Table 1: Stress and relax states identified by the heart rate
trend and stress dichotomy identified from the sentiment.
HR Trend
Decreasing
Increasing
Negative Sentiment
Relax
Distress
Positive Sentiment
Relax
Eustress
4.4 Results
The interpolation methods and simple linear
regression were performed on the data from both
experiments. Table 2 presents the detailed results, the
numbers of distress, eustress and relax states are
provided for each experiment and interpolation
method.
Table 2: Stress and relax states for each experiment and
interpolation method.
Experiment
Distress
Eustress
Relax
Total
#1 split
126
(12.2 %)
297
(28.9%)
606
(58.9%)
1029
#1 window
129
(12.6 %)
355
(34.6%)
541
(52.8%)
1025
#2 split
144
(14.2 %)
297
(29.2%)
576
(56.6%)
1017
#2 window
137
(13.5 %)
332
(32.6%)
548
(53.9%)
1017
From which we can calculate mean and standard
deviation for distress, eustress and relax over all
experiments and interpolation methods or per each of
them as presents Table 3.
Table 3: Mean and standard deviation overall and per
interpolation method and experiment for stress and relax
states.
Mean SD
Distress
Eustress
Relax
Split
interpolation
135 12.7
297 0
591 21.2
Window
interpolation
133 5.7
343.5 16.2
544.5 5
Overall
134 8.1
13 0.9 %
320 28.4
31 2.8 %
568 29.6
55.5 2.8 %
5 CONCLUSIONS
5.1 Discussion
The relax state was identified on average in 55.5 %
cases, distress in 13 % and eustress in 31 % cases.
When we compare the average results with the results
of individual experiments in which both interpolation
methods were used we can see that the resulting
values are similar. This could imply that the choice of
the interpolation method does not significantly
HEALTHINF 2018 - 11th International Conference on Health Informatics
530
influence the determination of stress and relax states
in this case.
On the other hand, the use of the interpolation by
moving window gives us more flexibility. Compared
to the interpolation by splitting the interval, we can
experiment with the delta parameter and achieve a
better fit of the linear regression method to improve
the identification of stress dichotomy. We are fully
aware of this matter, but the confirmation of such
hypothesis can be achieved only with a bigger data
sample in the following research.
5.2 Further Research
The presented approach and the preliminary results
themselves are good starting points for further
research.
It is evident that the method itself has several
problems. For example, it describes the situation
related to acute stress within the time window that is
given by the sentiment interpolation window in which
the linear regression applied to the HR data was
performed. Since the stress state is supposed to be
present in a time window around its occurrence, we
can experiment with the length of the interpolation
window.
Moreover, when using the simple linear
regression, the regression fit was not calculated. This
needs to be considered in the next work. Besides
using the simple linear regression, we can also
process the HR data by using the method of simple
moving average (SMA).
Furthermore, the current study focuses on fairly
simple signal pre-processing steps rather than using
arguably more appropriate methods such as dynamic
Bayesian networks (DBN) (Dagum P., 1992; Dagum
P., 1995) and Hidden Markov Models (HMM)
(Wahlström J., 2017).
About the related work on the topic of fusing
social media and sensor data, we would also take into
account existing research in similar fields such as
(Farseev A., 2017) and (Choudhury M. De, 2017).
5.3 Further Data Collection
With a bigger and more rigorous experimental setup
(multiple subjects of both genders and various age
categories) would be possible to validate the
presented results and make more generalizable
conclusions.
The current approach of data collection and pre-
processing is described in (Salamon J., 2017)
including the description of heart rate devices
accuracy. The accuracy of the used measurement
devices and uncertainty is sufficient to determine the
slope of the heart rate used in this paper.
Nevertheless, other experiments validated one of
the devices (Fitbit Charge HR) with various results
(Montoye A. H. K., 2017), (Dooley E. E., 2017) and
(Boudreaux B. D., 2017). It leads to the conclusion
the uncertainty modeling for more complex methods
(such as DBN or HMM) needs to be considered for
further research.
ACKNOWLEDGEMENTS
The work is supported by the UWB grant SGS-2016-
018 Data and Software Engineering for Advanced
Applications.
REFERENCES
Choi J., Gutierrez-Osuna R., 2009. Using Heart Rate
Monitors to Detect Mental Stress. In Proceedings of the
2009 Sixth International Workshop on Wearable and
Implantable Body Sensor Networks (BSN ’09). IEEE
Computer Society, Washington, DC, USA, 219–223.
https://doi.org/10.1109/BSN.2009.13.
Selye H.,1975. Confusion and Controversy in the Stress
Field. Journal of Human Stress 1, 2 (1975), 37–44.
https://doi.org/10.1080/0097840X.1975.9940406.
Blangsted A. K. et al., Hjortskov N., Rissen D., 2004. The
effect of mental stress on heart rate variability and
blood pressure during computer work. Eur J Appl
Physiol (2004). https://doi.org/10.1007/s00421-004-1055-z.
Spaepen A., Taelman J., Vandeput S. and Hu S. V., 2009.
Influence of mental stress on heart rate and heart rate
variability. In Proceedings 4th European Conference of
the International Federation for Medical and
Biological Engineering (IFMBE ’09), 1366–1369.
Joulin A., Grave E., Bojanowski P., and Mikolov T., 2016.
Bag of Tricks for Efficient Text Classification. (2016).
http://arxiv.org/abs/1607.01759.
Vassanyi I., Salai M., and Kosa I., 2016. Stress Detection
Using Low Cost Heart Rate Sensors. Journal of
Healthcare Engineering 2016 (2016). https://
doi.org/10.1155/2016/5136705.
Mcduff D., Gontarek S., and Picard R.. 2014. Remote
Measurement of Cognitive Stress via Heart Rate
Variability. Annual International Conference of the
IEEE Engineering in Medicine and Biology Society.
Mitra A., 2008. Diabetes and stress: a review.
Ethnomedicine 2 (2008), 131–135.
Schubert C., Lambertz M., Nelesen R.A., Bardwell W, Choi
J. B., and Dimsdale J.E., 2009. Effects of stress on heart
rate complexity — A comparison between short-term
and chronic stress. Biological psychology 80, 3 (03
2009),325–332.
https://doi.org/10.1016/j.biopsycho.2008.11.005.
Stress Dichotomy using Heart Rate and Tweet Sentiment
531
Shi Yu D., Ruiz N., Taib R., Choi E. H.C., and Chen F.,
2007. Galvanic skin response (GSR) as an index of
cognitive load. In CHI Extended Abstracts (2010-04-
15), Mary Beth Rosson and David J. Gilmore (Eds.).
ACM, 2651–2656.
http://dblp.unitrier.de/db/conf/chi/chi2007a.html#ShiR
TCC07.
Tanaka S., Yamakoshi T., Yamakoshi K., 2008. Feasibility
study on driver’s stress detection from differential skin
temperature measurement. August 2008, 1076– 1079.
Melin B., Lundberg U., Kadefors R., 1994.
Psychophysiological stress and emg activity of the
trapezius muscle. International Journal of Behavioral
Medicine 1 (1994), 354–370.
Wijsman J., Grundlehner B., Penders J., and Hermens H.,
2010. Trapezius muscle EMG as predictor of mental
stress. In Wireless Health, Irwin Mark Jacobs, Patrick
Soon-Shiong, Eric Topol, and Christofer Toumazou
(Eds.). ACM, 155–163.
http://dblp.uni-trier.de/db/conf/wh/wh2010.html.
Agyapong V. I. O., Mrklas K., Suen V. Y. M., Rose M. S.,
Jahn M., Gladue I., Greenshaw, A., 2015. Supportive
Text Messages to Reduce Mood Symptoms and
Problem Drinking in Patients with Primary Depression
or Alcohol Use Disorder: Protocol for an
Implementation Research Study. JMIR Research
Protocols, 4(2), e55.
http://doi.org/10.2196/resprot.4371.
Montes J. M., et al., 2012. A short message service (SMS)-
based strategy for enhancing adherence to antipsychotic
medication in schizophrenia. Psychiatry Research,
Volume 200, Issue 2, 89 - 95.
http://dx.doi.org/10.1016/j.psychres.2012.07.034.
Salamon J., Mouček M., 2017 Heart rate and sentiment
experimental data with common timeline, In Data in
Brief, Volume 15, 2017, Pages 851-861, ISSN 2352-
3409. https://doi.org/10.1016/j.dib.2017.10.037.
Dagum P., Galper A., Horvitz E., 1992. Dynamic Network
Models for Forecasting. Proceedings of the Eighth
Conference on Uncertainty in Artificial Intelligence.
AUAI Press: 41–48.
Dagum P., Galper A., Horvitz E., Seiver A., 1995.
Uncertain Reasoning and Forecasting. International
Journal of Forecasting. 11 (1): 73–87.
https://doi.org/10.1016/0169-2070(94)02009-e.
Wahlström J., et al., 2017. A Hidden Markov Model for
Seismocardiography, in IEEE Transactions on
Biomedical Engineering, vol. 64, no. 10, pp. 2361-
2372.
https://doi.org/10.1109/TBME.2017.2648741.
Farseev A., Chua T., 2017. TweetFit: Fusing Multiple
Social Media and Sensor Data for Wellness Profile
Learning. Thirty-First AAAI Conference on Artificial
Intelligence, AAAI Publications, 95-101.
Choudhury M. De, Kumar M, Weber I., 2017.
Computational Approaches Toward Integrating
Quantified Self Sensing and Social Media. In
Proceedings of the 2017 ACM Conference on Computer
Supported Cooperative Work and Social Computing
(CSCW '17). ACM, New York, NY, USA, 1334-1349.
https://doi.org/10.1145/2998181.2998219.
Montoye A. H. K., Mitrzyk J. R., Molesky M. J., 2017.
Comparative Accuracy of a Wrist-Worn Activity
Tracker and a Smart Shirt for Physical Activity
Assessment, Measurement in Physical Education and
Exercise Science, 1091-367.
http://dx.doi.org/10.1080/1091367X.2017.1331166.
Dooley E.E., Golaszewski N.M., Bartholomew J.B., 2017.
Estimating Accuracy at Exercise Intensities: A
Comparative Study of Self-Monitoring Heart Rate and
Physical Activity Wearable Devices. JMIR Mhealth
Uhealth 2017; 5(3):e34.
http://doi.org/10.2196/mhealth.7043.
Boudreaux B. D., Cormier C. L., Williams B. M., Hebert E.
P., Kraemer R. R., 2017. Accuracy of Wearable
Devices for Determining Physiological Measures
during Different Physical Activities, Medicine &
Science in Sports & Exercise: May 2017 - Volume 49 -
Issue 5S - p 762.
http://doi.org/10.1249/01.mss.0000519027.48408.29.
HEALTHINF 2018 - 11th International Conference on Health Informatics
532
