Quantitative Analysis of Mental Effort Investment using
Photoplethysmogram
Yongqiang Lyu
1,2
, Tianshu Yang
1
, Xiaomin Luo
3
, Chun Yu
1
, Lei Wang
3
and Yuanchun Shi
1
1
RTsinghua National Laboratory for Information Science and Technology, Beijing, China
2
Research Institute of Information Technology, Tsinghua University, Beijing, China
3
BGI (Wuhan) Translational Research Center, Wuhan, china
Keywords: Photoplethysmography, Mental Effort, Psychophysiology Approach, Sensors-based Applications.
Abstract: Recent studies have shown a close relationship between the mental effort and the photoplethysmograms
(PPG), but lack an index which can analyze the invested mental effort quantitatively during the cognitive
tasks, which is notable for evaluating user experience in ubiquitous scenarios. In this study, we propose the
stress-induced vascular response index (sVRI) to quantify the mental effort invested in cognitive tasks, and
verify it via the experiments on aritehmatic tasks with different difficulty levels. The results show an
outstanding correlation between the sVRI and the task difficulties, which suggests a dose-response of the
sVRI when users perform different cognitive tasks with different difficulties. This is useful for evaluating
the user experience in performing cognitive tasks to guide the task or the interaction designs.
1 INTRODUCTION
Peripheral vasoconstriction has a methodological
advantage over other general arousal measures: it
reflects uniquely the activity of the sympathetic
nervous system (Iani et al., 2004). Changes in pulse
amplitude reflect predominantly sympathetic
influences: a decrease in pulse amplitude is caused
almost exclusively by the activation of this system
(Iani et al., 2004; Harris, 2001; Collins et al., 2005).
It has also been accepted that sympathetic tone is
the dominant influence of systolic amplitude. The
waveform amplitude of Photoplethysmography
(PPG) signals changes in association with
sympathetic tone (Award, 2001) (Shelley and
Shelley, 2001). However, the absolute amplitude of
the PPG signal is not comparable because there are
no calibrating procedures currently available to
standardize the PPG amplitudes for comparing the
waveform of one subject to another. The main
difficulty to realize that lies in the uncertainty due to
the hardware and subject factors; even the variations
in sensor placement may also influence the analysis
much (Nilsson et al., 2010). The relative measure
without a unit designation is necessary, particularly
serving repeated measurements in ubiquitous
scenarios.
Photoplethysmogram (PPG) is an optical
measurement technique that can be used to detect
blood volume changes in the microvascular bed of
tissue (Allen, 2007; Alneab et al., 2007). The pulse
wave analysis (PWA) of photoplethysmogram
provides lots of circulatory information including
but not limited to the pulse amplitude (Schmitz et al.,
2006). Goor et al. (Goor et al., 2004) has
demonstrated that the mental-stress induced
peripheral arterial vasoconstriction predicts mental-
stress-induced myocardial ischemia. Luo et al (Luo
et al., 2011) also assessed the changes of
photoplethysmogram appearance and investigated
the morphological components of normalized PPG
waveform associated with sympathetic activities in
past years (Xiao et al., 2011; Luo et al., 2012).
In the quantitative studies of stress-related
changes in pulse shape characteristics extracted from
the PPG waveform reported previously, a modular
system named Detecting Blood Flow Parameters
Based on Pulse Wave Measurement and Analysis
was employed in the post processing and data
mining of PPG signals to search indices
representatively fit the stress induced vascular
responses (Xiao et al., 2011; Luo et al., 2012).
However, it was proven in association with the
magnitude of stress tested by the traidtional standard
psychological materials, rather than cognitive tasks.
166
Lyu Y., Yang T., Luo X., Yu C., Wang L. and Shi Y..
Quantitative Analysis of Mental Effort Investment using Photoplethysmogram.
DOI: 10.5220/0004732001660171
In Proceedings of the International Conference on Health Informatics (HEALTHINF-2014), pages 166-171
ISBN: 978-989-758-010-9
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
This study proposes the stress-induced vascular
response index (sVRI) with an online statistical
noise-filting technique embeded in the PPG sensor.
The index is also tested over the arithematic
calculation based cognitive tasks to verifiy its dose-
response between different difficulties of tasks. In
the experimental design, we hypothesized that the
association between photoplethysmogram
appearance and mental effort investment could be
quantitatively measured. We aim to explore the
impact of increasing mental effort on its value and
quantify its effects on trend of varying mental effort
invested throughout experiment. As an extended
study of stress-induced vascular response
represented by PPG derived features, this study
firstly proposes the index of it and verify its dose-
response during cognitive tasks.
2 MATERIALS AND METHODS
2.1 Subjects
Twenty-three undergraduate students (12 females,
11 males) were paid to participate in this
experiment. Two participants were excluded from
the analysis because of failing to comply with the
instructions, therefore, the final sample consisted of
21 participants (age range of 21–26 years; M=23.43,
SD=2.56 years). All participants were healthy and
did not use any medications. None of them was
familiar with the experimental task.
2.2 Methods
2.2.1 Setting
The experiments were done in a quiet, temperature-
controlled room at 24±2°C. Participants one by one
sat in a comfortable chair throughout the experiment.
All participants were asked to rest for no less than
20 minutes before each task performance.
2.2.2 The Tasks
The arithmetic calculation is a validated and widely
used test able to induce a considerable degree of
perceived stress (Dedovic et al., 2005; Rimmele et
al., 2007). In the study, a 3-minute mental arithmetic
task was performed continuously during which the
participants were asked to calculate the arithmetic
operations of two 2-digit numbers as quickly and
accurately as possible. Their final scores were
recorded and correlated to their pays. The tests with
bad scores may be reevaluated to be withdrawn from
the testing results to be analyzed. The easy level of
the task difficulty is defined as that there are only
additions and subtractions; while the hard level is
defined as that there appears multiplications
frequently. None of the operands were greater than
20. Answers spending more than 15 seconds were
automatically judged wrong. Only a tablet PC is
used in calculation and its position lies always in the
horizontal plane throughout the experiment.
Participants were all right-handed and asked to place
the tips of their index finger and thumb of their right
hand only on touch screen during performance of
tasks, which we believed helpful to lower down
some uncontrolled noise. The software interface as
shown in the following Figure 1 was employed for
the participants to complete the arithematic
calculation tasks.
Figure 1: The arithmetic calculating software. There is a
timing-count prompt as the progress bar, the calculation
inputs and the results panels on the graphic user interface
of the software.
2.2.3 Measuring sVRI, Blood Pressure and
Heart Rate during the Taks
Blood pressure and heart rate were ever believed
correlated to user experience in processing tasks in
the acedemic field. So they were also measured
together with the sVRI right before, during and after
the tasks performed.
The blood pressure and the
heart rate were all measured with the
digital
sphygmomanometer, OMRON HEM-7112
(OMRON China Co., Ltd.). The sVRIs were
recorded at one fingerflip of the left hand of the
participant in sitting position with the left hand
placed natually on the table surface. The sVRI was
noninvasively measured by a prototype finger-clip
sensor developed by us at a sampling rate of 500Hz.
Figure 2 illustrates the differences in pulse pattern
and quantitative features against different task
difficulties.
QuantitativeAnalysisofMentalEffortInvestmentusingPhotoplethysmogram
167
(a) Pulse pattern associated with task of lower difficulty.
(b) Pulse pattern associated with task of higher difficulty.
Figure 2: The waveform morphology comparison.
This study proposes a time window-based
evaluation algorithm to evaluate the sVRI based on
the PPG patterns. For example, the time window
from t0 to t1 in Figure 3, there may be couple of
patterns found in the window, and each pattern can
be quantified to a normalized number. However,
some of the patterns are without sanity or subject to
noise. So, a statistical data filtering is needed to
normalize the index in order to resolve the effects of
the noise and outlying data. As Figure 3 shows, the
sVRI model in this study realizes the window based
extraction and indexing together with the statistical
filtering to give the stable and good-quality sVRI.
Figure 3: The sVRI calculation model: the original PPG
waveforms are extracted as stress patterns, based on which
a time window-based index evaluation algorithm is
employed to give the sVRI in the target window.
In previous works, the stress-induced vascular
response was evaluated from the normalized PPG
signals and had been repeatedly proven to be stress-
dependent (Luo et al., 2013; Luo et al., 2012). In this
study, it is further studied with the statistical filtering
technique that helps it become more stable.
Furthermore, it becomes a quantitative index which
is verfied to capture the dose-response of the user
when performing cognitive tasks.
2.3 Experimental Design
In the experiment, all participants were asked to
perform the arithmetic test of leveled difficulties to
induce a perceived mental effort. Two difficulties
were tested in different days with randomized
sequence of the participants. The sVRI, blood
pressure and heart rate were collected right before,
during and after the tasks. At the same time, all
participants were asked to finish a survey to verify
the task difficulties literally in order to check the
experiment design had guaranteed the consistency of
the mental efforts and the task diffculties. The
results of the sVRI, blood pressure and heart rate
measurements of all the participants were analyzed
against the task difficulties to examine the dose-
response.
2.4 Statistical Analysis
The statistical analysis was performed with the SPSS
16.0. The measured values presented as mean ±
standard deviation. The General Lineal Model
Repeated Measures and the general paired-samples t
tests were used to compare subjects’ states before,
during and after the tasks. The statistical
significance was defined as P<0.05.
3 RESULTS
Table 1 lists all the testing results of the sVRI, blood
pressure (systolic: SBP and diastolic: DBP) and
heart rate (HR). The data were organized according
to different task levels (the upper part is the easy and
the lower part is the hard). The corresponding
measured values were recorded as mean ± standard
deviation in the testing periods of pre-task (before
the task), in-task (in performing the task) and post-
task (after the task) respectively. The DBP and SBP
stand for the diastolic blood pressure and systolic
blood pressure respectively.
The first analysis was to test the effect of the
difficulty manipulation. According to our survey
after the experiments, all subjects thought the hard
tasks were really harder no matter which level of the
task the subjects did first. This is also consistent
with the fact that all the participants finished fewer
calculations in the hard task than in the easy task.
The second analysis was to check the baselines
of the signal levels of each vital sign (sVRI, SBP,
DBP and HR) in order to guarantee there were no
accidental outstanding noises introduced before and
in the tasks. Table 2 listed the analysis results. The
HEALTHINF2014-InternationalConferenceonHealthInformatics
168
Table 1: The values of stress-induced vascular response index (sVRI), heart rate (HR), blood pressure (BP) measured in the
two levels of tasks.
Task Difficulty
Time Period
Pre-task In-task Post-task
Easy
sVRI 0.8270.0551 0.9060.0479 0.8250.0539
HR 79.510.8 81.38.87
DBP 64.38.33 63.97.84
SBP 109.215.6 109.516.7
Hard
sVRI 0.8320.0440 0.9310.0528 0.8440.0464
HR 79.311.5 79.611.3
DBP 64.510.6 65.78.10
SBP 109.616.3 110.414.6
Table 2: The paired-sample t tests on the vital signs in the same task stage with different difficulty levels.
Easy Hard T P
Pre-task
sVRI 0.8270.0551 0.8320.0440 0.116 0.735
HR 79.510.8 79.311.5 0.002 0.967
DBP 64.38.33 64.510.6 0.007 0.936
SBP 109.215.6 109.616.3 0.006 0.939
In-task
sVRI 0.9060.0479 0.9310.0528 5.468 0.000
HR 81.38.87 79.611.3 0.301 0.587
DBP 63.97.84 65.78.10 0.513 0.478
SBP 109.516.7 110.414.6 0.031 0.860
Table 3: ANOVA tests of sVRI, HR, BP over the different stages of the tasks (before and in task).
Pre-task In-task F P
Easy
sVRI 0.8270.0551 0.9060.0479 9.427 0.001
HR 79.510.8 81.38.87 0.353 0.556
DBP 64.38.33 63.97.84 0.023 0.879
SBP 109.215.6 109.516.7 0.003 0.955
Hard
sVRI 0.8320.0440 0.9310.0528 16.518 0.000
HR 79.311.5 79.611.3 0.005 0.946
DBP 64.510.6 65.78.10 0.154 0.697
SBP 109.616.3 110.414.6 0.025 0.874
Table 4: The results of Repeated Measurements ANOVA.
Pre-task In-task Post-task F P
Easy 0.8270.0551 0.9060.0479 0.8250.0539 8153 0.000
Hard 0.8320.0440 0.9310.0528 0.8440.0464 14934.13 0.000
pre-task values of those three vital signs were firstly
tested via the paired-sample t tests in order to verify
the normal signal bases of the participants before the
tasks. We can see that there are no significant
differences found on the three vital signs between
the easy and hard tasks (P > 0.05, also with little F
values). That is, the base signal levels of the three
vital signs were all kept normal and the participants
were all in good status to enter the tasks. Secondly,
the similar t-tests were done for the data of in-task to
check the signal bases of the participants when
performing the tasks. It can be seen that there were
no significant difference found on the blood pressure
and heart rate between the two levels of tasks. It
should be noticed that there was a significant
difference found on sVRI between the easy task and
hard task (P=0 and T=5.468). Figure 4 also gives the
bar charts of the sVRI values of the three stages
(pre-task, in-task and post-task) in the easy and hard
tasks.
QuantitativeAnalysisofMentalEffortInvestmentusingPhotoplethysmogram
169
Figure 4: The sVRI values of the three stages.
So, we can differentiate the task levels (hard v.s.
easy) just via the sVRI values; it captures the dose-
response of the mental effort invested in the
cognitive tasks with tiny different difficulties as set
in this study.
The third analysis was done to observe the
sVRI’s changes with the different stages of the tasks:
before and during the task. We employed ANOVA
tests as Table 3 listed. For the easy and hard task
respectively, the ANOVA results tested over the two
testing stages: before (pre-task) and during (in-task)
the task. From Table 3, we can see that the upper
part shows the results of those of the easy task: only
the sVRI had outstanding F and P values in this test,
which means that the sVRI got significantly
different when the participants performed from the
pre-task stage to the in-task stage; others (BP and
HR) failed to pass the test. We can also see the
lower part of the table shows the similar results of
the in-task: only the sVRI passed the test with
F=16.518 and P=0. Therefore, it can be concluded
that there was a clear dose-response on the
participants when the cognitive loads of them were
increased by an arithematic calculation task. The
dose-response is even much clearer in the hard task
than in the easy task (higher F and lower P).
The sVRI values were also recorded after the
tasks, so the repeated measurements of ANOVA was
also used to test the continuous dose-response down
through the entire tasks, i.e. from before the task to
after the task. Table 4 gives the testing results, from
which we can see that there were clear significance
found in the sVRI values of three stages in both easy
and hard tasks. It means that the sVRI can
distinguish the situation how the mental effort
invested in processing the cognitive tasks with the
small task difficulties set in the experiment; it shows
the continuous dose-response in processing the
cognitive tasks.
We also tested the correlation between the task
difficulties and the sVRIs; we classified three
difficulties: no task, easy task and hard task. We got
the correlation coefficient r=0.566 with P=0 by
Sperman testing, which means there is a moderate
correlation between sVRI and task difficulties.
4 DISCUSSION
This study examined the relationship between the
mental effort invested in the cognitive tasks and the
changes of sVRI. The testing results on the different
difficulties of cognitive tasks showed that the PPG-
based sVRI can be easily obtained from the real time
PPG signals and can be used to distinguish the
different extents to which the mental effort is
invested during a cognitive task. This can help it
adopted in more diverse ubiquitous applications of
sVRI, particularly in ubiquitous healthcare service
(Farooq et al., 2010; Shin, et al., 2010).
As illustrated in the results, the significant
differences of sVRI between different task
difficulties indicated that the dose-response against
easy task could be represented by a relatively lower
increase of sVRI value above the baseline.
Conversely, the dose-response against hard task
could be represented by a relatively higher increase
of sVRI value above the baseline, indicating more
mental effort invested. This finding provides strong
evidence for the sVRI to be potentially used as a
linear classifier of mental effort investment.
In comparison with sVRI, other vital signs such
as the heart rate and blood pressure did not reach
any convincing significance in task difficulties in
this experiment. The similar conclusion has also
been reported previously that the effect of mental
stress on central pressures is more prominent
compared with the effect on peripheral pressures
(Vlachopoulos et al., 2006; Madelon et al., 1998),
and can partly explain the results.
The major limitation of this study is the
relatively small sample size and the age range of
participants involved in the experiment. A database
of large sample size can be proposed in the future
work to setup the baseline of rest state and impact of
task difficulty on sVRI, which can evaluate
quantitatively an individual against the age-matched
normative data as a reference.
5 CONCLUSIONS
As an optical technique for the noninvasive
measurement of blood volume changes, PPG is low
cost and easy to use, especially suitable for rapid and
HEALTHINF2014-InternationalConferenceonHealthInformatics
170
repeated measurements particularly in ubiquitous
environments. In this experiment, PPG-based sVRI
is found to be able to reflect the dose-response of the
mental effort invested in cognitive tasks: the mental
effort invested changes with the task difficulties can
be shown efficiently via the sVRIs. This could be
used to evaluate the user experience in an interactive
task or to evaluate the usability of a cognitive design.
ACKNOWLEDGEMENTS
This work is funded by NSFC 61201357 and FIT
foundation of Tsinghua University.
REFERENCES
Allen, J., 2007. Photoplethysmography and its application
in clinical physiological measurement. Physiological
measurement, 28(3), p. R1.
Alneab, M. E., Alobaid, N. & Seifalian, A. M., 2007.
Optical techniques in the assessment of peripheral
arterial disease. Current Vascular Pharmacology, 5(1),
pp. 53-59.
Award, A., 2001. "How Does the Plethysmogram Derived
from the Pulse Oximeter Relate to Arterial Blood
Pressure in Coronary Artery Bypass Graft Patients?".
Anesth Anal, Volume 93, pp. 1466-1471.
Collins, S. M., Karasek, R. A. & Costas, K., 2005. Job
strain and autonomic indices of cardiovascular disease
risk. American journal of industrial medicine, 48(3),
pp. 182-193.
Dedovic, K., Renwick, R. & Mahani, N. K., 2005. The
Montreal Imaging Stress Task: using functional
imaging to investigate the effects of perceiving and
processing psychosocial stress in the human brain.
Journal of Psychiatry and Neuroscience, 30(5), p. 319.
Farooq, U., Jang, D. G. & Park, J. H., 2010. PPG
delineator for real-time ubiquitous applications. In:
Engineering in Medicine and Biology Society (EMBC),
2010 Annual International Conference of the IEEE.
s.l.:IEEE, pp. 4582-4585.
Goor, D. et al., 2004. Sheps DS. Peripheral arterial
tonometry: a diagnostic method for detection of
myocardial ischemia induced during mental stress
tests: a pilot study. Clin Cardiol, Volume 27, pp. 137-
41.
Harris, C. R., 2001. Cardiovascular responses of
embarrassment and effects of emotional suppression in
a social setting. Journal of personality and social
psychology, 81(5), p. 886.
Iani, C., Gopher, D. & Lvie, P., 2004. Effects of task
difficulty and invested mental effort on peripheral
vasoconstriction. Psychophysiology, 41(5), pp. 789-
798.
Luo, L. et al., 2013. On psychological measurement and
application of photoplethysmography. China Medical
Equipment, 10(105), pp. 44-47.
Luo, L., Xiao, L., Miao, D. & Luo, X., 2012. The
relationship between mental stress induced changes in
cortisol levels and vascular responses quantified by
waveform analysis: investigating stress-dependent
indices of vascular changes.. Proceedings of the 2012
International Conference on Biomedical Engineering
and Biotechnology, Volume 2, pp. 929-933.
Luo, X., Zhang, B., Hao, X. & Wang, X., 2011.
Identification of Photoplethysmogram Appearance for
Clinical Interpretation. proceedings of The World
Congress on Engineering and Technology, Volume 7,
pp. 442-445.
Madelon, L. P. et al., 1998. Cardiovascular and endocrine
responses to experimental stress: Effects of mental
effort and controllability. Psychoneuroendocrinology,
January, 23(1), pp. 1-17.
Nilsson, L., Goscinski, T. & Lindenberger, M., 2010.
Respiratory variations in the photoplethysmographic
waveform: acute hypovolaemia during spontaneous
breathing is not detected. Physiological mearurement,
31(7), p. 953.
Rimmele, U., Zellweger, B. C. & Marti, B., 2007. Trained
men show lower cortisol, heart rate and psychological
responses to psychosocial stress compared with
untrained men. Psychoneuroendocrinology, 32(6), pp.
627-635.
Schmitz, C. H., Graber, H. L. & Barbour, R. L., 2006.
Peripheral vascular noninvasive measurements [J].
Encyclopedia of Medical Devices and Instrumentation.
Shelley, K. H. & Shelley, S., 2001. "Pulse Oximeter
Waveform: Photoelectric Plethysmography.". In:
Clinical Monitoring: Practical applications for
anesthesia and critical care.. s.l.:s.n., pp. 420-428.
Shin, W., Cha, Y. D. & Yoon, G., 2010. ECG/PPG integer
signal processing for a ubiquitous health monitoring
system. Journal of Medical systems, 34(5), pp. 891-
898.
Vlachopoulos, C., Kosmopoulou, F. & Alexopoulos, N.,
2006. Acute mental stress has a prolonged unfavorable
effect on arterial stiffness and wave reflection.
Psychosomatic medicine, 68(2), pp. 231-237.
Xiao, L. et al., 2011. Assessment of Stress Reactions of
Recruits Based on Quantitative analysis of the
characteristics of fingertip Photoplethysmographic.
Med J Chin, 1 Sep, 36(9), pp. 991-994.
QuantitativeAnalysisofMentalEffortInvestmentusingPhotoplethysmogram
171