Quantitative Analysis of Mental Effort Investment using

Photoplethysmogram

Yongqiang Lyu

1,2

, Tianshu Yang

, Xiaomin Luo

, Chun Yu

, Lei Wang

and Yuanchun Shi

RTsinghua National Laboratory for Information Science and Technology, Beijing, China

Research Institute of Information Technology, Tsinghua University, Beijing, China

BGI (Wuhan) Translational Research Center, Wuhan, china

Keywords: Photoplethysmography, Mental Effort, Psychophysiology Approach, Sensors-based Applications.

Abstract: Recent studies have shown a close relationship between the mental effort and the photoplethysmograms

(PPG), but lack an index which can analyze the invested mental effort quantitatively during the cognitive

tasks, which is notable for evaluating user experience in ubiquitous scenarios. In this study, we propose the

stress-induced vascular response index (sVRI) to quantify the mental effort invested in cognitive tasks, and

verify it via the experiments on aritehmatic tasks with different difficulty levels. The results show an

outstanding correlation between the sVRI and the task difficulties, which suggests a dose-response of the

sVRI when users perform different cognitive tasks with different difficulties. This is useful for evaluating

the user experience in performing cognitive tasks to guide the task or the interaction designs.

1 INTRODUCTION

Peripheral vasoconstriction has a methodological

advantage over other general arousal measures: it

reflects uniquely the activity of the sympathetic

nervous system (Iani et al., 2004). Changes in pulse

amplitude reflect predominantly sympathetic

influences: a decrease in pulse amplitude is caused

almost exclusively by the activation of this system

(Iani et al., 2004; Harris, 2001; Collins et al., 2005).

It has also been accepted that sympathetic tone is

the dominant influence of systolic amplitude. The

waveform amplitude of Photoplethysmography

(PPG) signals changes in association with

sympathetic tone (Award, 2001) (Shelley and

Shelley, 2001). However, the absolute amplitude of

the PPG signal is not comparable because there are

no calibrating procedures currently available to

standardize the PPG amplitudes for comparing the

waveform of one subject to another. The main

difficulty to realize that lies in the uncertainty due to

the hardware and subject factors; even the variations

in sensor placement may also influence the analysis

much (Nilsson et al., 2010). The relative measure

without a unit designation is necessary, particularly

serving repeated measurements in ubiquitous

scenarios.

Photoplethysmogram (PPG) is an optical

measurement technique that can be used to detect

blood volume changes in the microvascular bed of

tissue (Allen, 2007; Alneab et al., 2007). The pulse

wave analysis (PWA) of photoplethysmogram

provides lots of circulatory information including

but not limited to the pulse amplitude (Schmitz et al.,

2006). Goor et al. (Goor et al., 2004) has

demonstrated that the mental-stress induced

peripheral arterial vasoconstriction predicts mental-

stress-induced myocardial ischemia. Luo et al (Luo

et al., 2011) also assessed the changes of

photoplethysmogram appearance and investigated

the morphological components of normalized PPG

waveform associated with sympathetic activities in

past years (Xiao et al., 2011; Luo et al., 2012).

In the quantitative studies of stress-related

changes in pulse shape characteristics extracted from

the PPG waveform reported previously, a modular

system named Detecting Blood Flow Parameters

Based on Pulse Wave Measurement and Analysis

was employed in the post processing and data

mining of PPG signals to search indices

representatively fit the stress induced vascular

responses (Xiao et al., 2011; Luo et al., 2012).

However, it was proven in association with the

magnitude of stress tested by the traidtional standard

psychological materials, rather than cognitive tasks.

166

Lyu Y., Yang T., Luo X., Yu C., Wang L. and Shi Y..

Quantitative Analysis of Mental Effort Investment using Photoplethysmogram.

DOI: 10.5220/0004732001660171

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2014), pages 166-171

ISBN: 978-989-758-010-9

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

This study proposes the stress-induced vascular

response index (sVRI) with an online statistical

noise-filting technique embeded in the PPG sensor.

The index is also tested over the arithematic

calculation based cognitive tasks to verifiy its dose-

response between different difficulties of tasks. In

the experimental design, we hypothesized that the

association between photoplethysmogram

appearance and mental effort investment could be

quantitatively measured. We aim to explore the

impact of increasing mental effort on its value and

quantify its effects on trend of varying mental effort

invested throughout experiment. As an extended

study of stress-induced vascular response

represented by PPG derived features, this study

firstly proposes the index of it and verify its dose-

response during cognitive tasks.

2 MATERIALS AND METHODS

2.1 Subjects

Twenty-three undergraduate students (12 females,

11 males) were paid to participate in this

experiment. Two participants were excluded from

the analysis because of failing to comply with the

instructions, therefore, the final sample consisted of

21 participants (age range of 21–26 years; M=23.43,

SD=2.56 years). All participants were healthy and

did not use any medications. None of them was

familiar with the experimental task.

2.2 Methods

2.2.1 Setting

The experiments were done in a quiet, temperature-

controlled room at 24±2°C. Participants one by one

sat in a comfortable chair throughout the experiment.

All participants were asked to rest for no less than

20 minutes before each task performance.

2.2.2 The Tasks

The arithmetic calculation is a validated and widely

used test able to induce a considerable degree of

perceived stress (Dedovic et al., 2005; Rimmele et

al., 2007). In the study, a 3-minute mental arithmetic

task was performed continuously during which the

participants were asked to calculate the arithmetic

operations of two 2-digit numbers as quickly and

accurately as possible. Their final scores were

recorded and correlated to their pays. The tests with

bad scores may be reevaluated to be withdrawn from

the testing results to be analyzed. The easy level of

the task difficulty is defined as that there are only

additions and subtractions; while the hard level is

defined as that there appears multiplications

frequently. None of the operands were greater than

20. Answers spending more than 15 seconds were

automatically judged wrong. Only a tablet PC is

used in calculation and its position lies always in the

horizontal plane throughout the experiment.

Participants were all right-handed and asked to place

the tips of their index finger and thumb of their right

hand only on touch screen during performance of

tasks, which we believed helpful to lower down

some uncontrolled noise. The software interface as

shown in the following Figure 1 was employed for

the participants to complete the arithematic

calculation tasks.

Figure 1: The arithmetic calculating software. There is a

timing-count prompt as the progress bar, the calculation

inputs and the results panels on the graphic user interface

of the software.

2.2.3 Measuring sVRI, Blood Pressure and

Heart Rate during the Taks

Blood pressure and heart rate were ever believed

correlated to user experience in processing tasks in

the acedemic field. So they were also measured

together with the sVRI right before, during and after

the tasks performed.

The blood pressure and the

heart rate were all measured with the

digital

sphygmomanometer, OMRON HEM-7112

(OMRON China Co., Ltd.). The sVRIs were

recorded at one fingerflip of the left hand of the

participant in sitting position with the left hand

placed natually on the table surface. The sVRI was

noninvasively measured by a prototype finger-clip

sensor developed by us at a sampling rate of 500Hz.

Figure 2 illustrates the differences in pulse pattern

and quantitative features against different task

difficulties.

QuantitativeAnalysisofMentalEffortInvestmentusingPhotoplethysmogram

167

(a) Pulse pattern associated with task of lower difficulty.

(b) Pulse pattern associated with task of higher difficulty.

Figure 2: The waveform morphology comparison.

This study proposes a time window-based

evaluation algorithm to evaluate the sVRI based on

the PPG patterns. For example, the time window

from t0 to t1 in Figure 3, there may be couple of

patterns found in the window, and each pattern can

be quantified to a normalized number. However,

some of the patterns are without sanity or subject to

noise. So, a statistical data filtering is needed to

normalize the index in order to resolve the effects of

the noise and outlying data. As Figure 3 shows, the

sVRI model in this study realizes the window based

extraction and indexing together with the statistical

filtering to give the stable and good-quality sVRI.

Figure 3: The sVRI calculation model: the original PPG

waveforms are extracted as stress patterns, based on which

a time window-based index evaluation algorithm is

employed to give the sVRI in the target window.

In previous works, the stress-induced vascular

response was evaluated from the normalized PPG

signals and had been repeatedly proven to be stress-

dependent (Luo et al., 2013; Luo et al., 2012). In this

study, it is further studied with the statistical filtering

technique that helps it become more stable.

Furthermore, it becomes a quantitative index which

is verfied to capture the dose-response of the user

when performing cognitive tasks.

2.3 Experimental Design

In the experiment, all participants were asked to

perform the arithmetic test of leveled difficulties to

induce a perceived mental effort. Two difficulties

were tested in different days with randomized

sequence of the participants. The sVRI, blood

pressure and heart rate were collected right before,

during and after the tasks. At the same time, all

participants were asked to finish a survey to verify

the task difficulties literally in order to check the

experiment design had guaranteed the consistency of

the mental efforts and the task diffculties. The

results of the sVRI, blood pressure and heart rate

measurements of all the participants were analyzed

against the task difficulties to examine the dose-

response.

2.4 Statistical Analysis

The statistical analysis was performed with the SPSS

16.0. The measured values presented as mean ±

standard deviation. The General Lineal Model

Repeated Measures and the general paired-samples t

tests were used to compare subjects’ states before,

during and after the tasks. The statistical

significance was defined as P<0.05.

3 RESULTS

Table 1 lists all the testing results of the sVRI, blood

pressure (systolic: SBP and diastolic: DBP) and

heart rate (HR). The data were organized according

to different task levels (the upper part is the easy and

the lower part is the hard). The corresponding

measured values were recorded as mean ± standard

deviation in the testing periods of pre-task (before

the task), in-task (in performing the task) and post-

task (after the task) respectively. The DBP and SBP

stand for the diastolic blood pressure and systolic

blood pressure respectively.

The first analysis was to test the effect of the

difficulty manipulation. According to our survey

after the experiments, all subjects thought the hard

tasks were really harder no matter which level of the

task the subjects did first. This is also consistent

with the fact that all the participants finished fewer

calculations in the hard task than in the easy task.

The second analysis was to check the baselines

of the signal levels of each vital sign (sVRI, SBP,

DBP and HR) in order to guarantee there were no

accidental outstanding noises introduced before and

in the tasks. Table 2 listed the analysis results. The

HEALTHINF2014-InternationalConferenceonHealthInformatics

168

Table 1: The values of stress-induced vascular response index (sVRI), heart rate (HR), blood pressure (BP) measured in the

two levels of tasks.

Task Difficulty

Time Period

Pre-task In-task Post-task

Easy

sVRI 0.827േ0.0551 0.906േ0.0479 0.825േ0.0539

HR 79.5േ10.8 81.3േ8.87

DBP 64.3േ8.33 63.9േ7.84

SBP 109.2േ15.6 109.5േ16.7

Hard

sVRI 0.832േ0.0440 0.931േ0.0528 0.844േ0.0464

HR 79.3േ11.5 79.6േ11.3

DBP 64.5േ10.6 65.7േ8.10

SBP 109.6േ16.3 110.4േ14.6

Table 2: The paired-sample t tests on the vital signs in the same task stage with different difficulty levels.

Easy Hard T P

Pre-task

sVRI 0.827േ0.0551 0.832േ0.0440 0.116 0.735

HR 79.5േ10.8 79.3േ11.5 0.002 0.967

DBP 64.3േ8.33 64.5േ10.6 0.007 0.936

SBP 109.2േ15.6 109.6േ16.3 0.006 0.939

In-task

sVRI 0.906േ0.0479 0.931േ0.0528 5.468 0.000

HR 81.3േ8.87 79.6േ11.3 0.301 0.587

DBP 63.9േ7.84 65.7േ8.10 0.513 0.478

SBP 109.5േ16.7 110.4േ14.6 0.031 0.860

Table 3: ANOVA tests of sVRI, HR, BP over the different stages of the tasks (before and in task).

Pre-task In-task F P

Easy

sVRI 0.827േ0.0551 0.906േ0.0479 9.427 0.001

HR 79.5േ10.8 81.3േ8.87 0.353 0.556

DBP 64.3േ8.33 63.9േ7.84 0.023 0.879

SBP 109.2േ15.6 109.5േ16.7 0.003 0.955

Hard

sVRI 0.832േ0.0440 0.931േ0.0528 16.518 0.000

HR 79.3േ11.5 79.6േ11.3 0.005 0.946

DBP 64.5േ10.6 65.7േ8.10 0.154 0.697

SBP 109.6േ16.3 110.4േ14.6 0.025 0.874

Table 4: The results of Repeated Measurements ANOVA.

Pre-task In-task Post-task F P

Easy 0.827േ0.0551 0.906േ0.0479 0.825േ0.0539 8153 0.000

Hard 0.832േ0.0440 0.931േ0.0528 0.844േ0.0464 14934.13 0.000

pre-task values of those three vital signs were firstly

tested via the paired-sample t tests in order to verify

the normal signal bases of the participants before the

tasks. We can see that there are no significant

differences found on the three vital signs between

the easy and hard tasks (P > 0.05, also with little F

values). That is, the base signal levels of the three

vital signs were all kept normal and the participants

were all in good status to enter the tasks. Secondly,

the similar t-tests were done for the data of in-task to

check the signal bases of the participants when

performing the tasks. It can be seen that there were

no significant difference found on the blood pressure

and heart rate between the two levels of tasks. It

should be noticed that there was a significant

difference found on sVRI between the easy task and

hard task (P=0 and T=5.468). Figure 4 also gives the

bar charts of the sVRI values of the three stages

(pre-task, in-task and post-task) in the easy and hard

tasks.

QuantitativeAnalysisofMentalEffortInvestmentusingPhotoplethysmogram

169

Figure 4: The sVRI values of the three stages.

So, we can differentiate the task levels (hard v.s.

easy) just via the sVRI values; it captures the dose-

response of the mental effort invested in the

cognitive tasks with tiny different difficulties as set

in this study.

The third analysis was done to observe the

sVRI’s changes with the different stages of the tasks:

before and during the task. We employed ANOVA

tests as Table 3 listed. For the easy and hard task

respectively, the ANOVA results tested over the two

testing stages: before (pre-task) and during (in-task)

the task. From Table 3, we can see that the upper

part shows the results of those of the easy task: only

the sVRI had outstanding F and P values in this test,

which means that the sVRI got significantly

different when the participants performed from the

pre-task stage to the in-task stage; others (BP and

HR) failed to pass the test. We can also see the

lower part of the table shows the similar results of

the in-task: only the sVRI passed the test with

F=16.518 and P=0. Therefore, it can be concluded

that there was a clear dose-response on the

participants when the cognitive loads of them were

increased by an arithematic calculation task. The

dose-response is even much clearer in the hard task

than in the easy task (higher F and lower P).

The sVRI values were also recorded after the

tasks, so the repeated measurements of ANOVA was

also used to test the continuous dose-response down

through the entire tasks, i.e. from before the task to

after the task. Table 4 gives the testing results, from

which we can see that there were clear significance

found in the sVRI values of three stages in both easy

and hard tasks. It means that the sVRI can

distinguish the situation how the mental effort

invested in processing the cognitive tasks with the

small task difficulties set in the experiment; it shows

the continuous dose-response in processing the

cognitive tasks.

We also tested the correlation between the task

difficulties and the sVRIs; we classified three

difficulties: no task, easy task and hard task. We got

the correlation coefficient r=0.566 with P=0 by

Sperman testing, which means there is a moderate

correlation between sVRI and task difficulties.

4 DISCUSSION

This study examined the relationship between the

mental effort invested in the cognitive tasks and the

changes of sVRI. The testing results on the different

difficulties of cognitive tasks showed that the PPG-

based sVRI can be easily obtained from the real time

PPG signals and can be used to distinguish the

different extents to which the mental effort is

invested during a cognitive task. This can help it

adopted in more diverse ubiquitous applications of

sVRI, particularly in ubiquitous healthcare service

(Farooq et al., 2010; Shin, et al., 2010).

As illustrated in the results, the significant

differences of sVRI between different task

difficulties indicated that the dose-response against

easy task could be represented by a relatively lower

increase of sVRI value above the baseline.

Conversely, the dose-response against hard task

could be represented by a relatively higher increase

of sVRI value above the baseline, indicating more

mental effort invested. This finding provides strong

evidence for the sVRI to be potentially used as a

linear classifier of mental effort investment.

In comparison with sVRI, other vital signs such

as the heart rate and blood pressure did not reach

any convincing significance in task difficulties in

this experiment. The similar conclusion has also

been reported previously that the effect of mental

stress on central pressures is more prominent

compared with the effect on peripheral pressures

(Vlachopoulos et al., 2006; Madelon et al., 1998),

and can partly explain the results.

The major limitation of this study is the

relatively small sample size and the age range of

participants involved in the experiment. A database

of large sample size can be proposed in the future

work to setup the baseline of rest state and impact of

task difficulty on sVRI, which can evaluate

quantitatively an individual against the age-matched

normative data as a reference.

5 CONCLUSIONS

As an optical technique for the noninvasive

measurement of blood volume changes, PPG is low

cost and easy to use, especially suitable for rapid and

HEALTHINF2014-InternationalConferenceonHealthInformatics

170

repeated measurements particularly in ubiquitous

environments. In this experiment, PPG-based sVRI

is found to be able to reflect the dose-response of the

mental effort invested in cognitive tasks: the mental

effort invested changes with the task difficulties can

be shown efficiently via the sVRIs. This could be

used to evaluate the user experience in an interactive

task or to evaluate the usability of a cognitive design.

ACKNOWLEDGEMENTS

This work is funded by NSFC 61201357 and FIT

foundation of Tsinghua University.

REFERENCES

Allen, J., 2007. Photoplethysmography and its application

in clinical physiological measurement. Physiological

measurement, 28(3), p. R1.

Alneab, M. E., Alobaid, N. & Seifalian, A. M., 2007.

Optical techniques in the assessment of peripheral

arterial disease. Current Vascular Pharmacology, 5(1),

pp. 53-59.

Award, A., 2001. "How Does the Plethysmogram Derived

from the Pulse Oximeter Relate to Arterial Blood

Pressure in Coronary Artery Bypass Graft Patients?".

Anesth Anal, Volume 93, pp. 1466-1471.

Collins, S. M., Karasek, R. A. & Costas, K., 2005. Job

strain and autonomic indices of cardiovascular disease

risk. American journal of industrial medicine, 48(3),

pp. 182-193.

Dedovic, K., Renwick, R. & Mahani, N. K., 2005. The

Montreal Imaging Stress Task: using functional

imaging to investigate the effects of perceiving and

processing psychosocial stress in the human brain.

Journal of Psychiatry and Neuroscience, 30(5), p. 319.

Farooq, U., Jang, D. G. & Park, J. H., 2010. PPG

delineator for real-time ubiquitous applications. In:

Engineering in Medicine and Biology Society (EMBC),

2010 Annual International Conference of the IEEE.

s.l.:IEEE, pp. 4582-4585.

Goor, D. et al., 2004. Sheps DS. Peripheral arterial

tonometry: a diagnostic method for detection of

myocardial ischemia induced during mental stress

tests: a pilot study. Clin Cardiol, Volume 27, pp. 137-

41.

Harris, C. R., 2001. Cardiovascular responses of

embarrassment and effects of emotional suppression in

a social setting. Journal of personality and social

psychology, 81(5), p. 886.

Iani, C., Gopher, D. & Lvie, P., 2004. Effects of task

difficulty and invested mental effort on peripheral

vasoconstriction. Psychophysiology, 41(5), pp. 789-

798.

Luo, L. et al., 2013. On psychological measurement and

application of photoplethysmography. China Medical

Equipment, 10(105), pp. 44-47.

Luo, L., Xiao, L., Miao, D. & Luo, X., 2012. The

relationship between mental stress induced changes in

cortisol levels and vascular responses quantified by

waveform analysis: investigating stress-dependent

indices of vascular changes.. Proceedings of the 2012

International Conference on Biomedical Engineering

and Biotechnology, Volume 2, pp. 929-933.

Luo, X., Zhang, B., Hao, X. & Wang, X., 2011.

Identification of Photoplethysmogram Appearance for

Clinical Interpretation. proceedings of The World

Congress on Engineering and Technology, Volume 7,

pp. 442-445.

Madelon, L. P. et al., 1998. Cardiovascular and endocrine

responses to experimental stress: Effects of mental

effort and controllability. Psychoneuroendocrinology,

January, 23(1), pp. 1-17.

Nilsson, L., Goscinski, T. & Lindenberger, M., 2010.

Respiratory variations in the photoplethysmographic

waveform: acute hypovolaemia during spontaneous

breathing is not detected. Physiological mearurement,

31(7), p. 953.

Rimmele, U., Zellweger, B. C. & Marti, B., 2007. Trained

men show lower cortisol, heart rate and psychological

responses to psychosocial stress compared with

untrained men. Psychoneuroendocrinology, 32(6), pp.

627-635.

Schmitz, C. H., Graber, H. L. & Barbour, R. L., 2006.

Peripheral vascular noninvasive measurements [J].

Encyclopedia of Medical Devices and Instrumentation.

Shelley, K. H. & Shelley, S., 2001. "Pulse Oximeter

Waveform: Photoelectric Plethysmography.". In:

Clinical Monitoring: Practical applications for

anesthesia and critical care.. s.l.:s.n., pp. 420-428.

Shin, W., Cha, Y. D. & Yoon, G., 2010. ECG/PPG integer

signal processing for a ubiquitous health monitoring

system. Journal of Medical systems, 34(5), pp. 891-

898.

Vlachopoulos, C., Kosmopoulou, F. & Alexopoulos, N.,

2006. Acute mental stress has a prolonged unfavorable

effect on arterial stiffness and wave reflection.

Psychosomatic medicine, 68(2), pp. 231-237.

Xiao, L. et al., 2011. Assessment of Stress Reactions of

Recruits Based on Quantitative analysis of the

characteristics of fingertip Photoplethysmographic.

Med J Chin, 1 Sep, 36(9), pp. 991-994.

QuantitativeAnalysisofMentalEffortInvestmentusingPhotoplethysmogram

171