Physiological Data Validation of the Hexoskin Smart Textile

N. H. Cherif

1,3

, N. Mezghani

2,3

, N. Gaudreault

, Y. Ouakrim

2,3

, I. Mouzoune

3,4

and P. Boulay

Faculty of Science and Technology, University of Lorraine, France

LICEF Research Center, TELUQ University, Montréal, Quebec, Canada

Laboratoire de Recherche en Imagerie et Orthopédie (LIO), Centre de Recherche du CHUM, Montréal, Quebec, Canada

Faculté de Médecine et des Sciences de la Santé, Université de Sherbrooke, Quebec, Canada

Faculté des Sciences de L’activité Physique, Université de Sherbrooke, Quebec, Canada

Keywords:

Concordance Correlation Coefﬁcient, Intraclass Correlation Coefﬁcient, Bland-Altman, Agreement Analysis,

Cardiorespiratory.

Abstract:

The aim of this study is to validate cardiorespiratory function measurement of a healthy population provided

by a wearable textile during a progressive maximal exercise test. The following measurements were collected

using embedded sensors to assess three variables: heart rate (HR), breathing rate (BR) and ventilation (Ve).

These variables were recorded simultaneously by the wearable textile and using as a reference system for

a comparison purpose. The validation was performed based on the two systems agreement estimation by

calculating the intraclass correlation coefﬁcient (ICC), the concordance correlation coefﬁcient (CCC) and the

Bland-Altman plot for each variable. Twenty-eight healthy volunteers participated in this study. Analysis

of each participant under exercise condition by the two measurement systems revealed high CCC values (ρ

between 0.91 and 0.99), no deviation from the 45

◦

line (C

between 0.96 and 0.99) and signiﬁcant ICC values

(ρ between 0.91 and 0.99, p < 0.05) for HR and BR. The Bland Altman plot for HR and BR indicated no

deviation of the mean difference from zero and a small variability with tight agreement limits. However, the

analysis of the estimated ventilation Ve of each participant revealed doubtful values for the CCC (ρ

between

0.2 and 0.99) and ICC (ρ between 0.11 and 0.99). In summary, the Hexoskin presented good agreement for

HR and BR. However, for ventilation, it is difﬁcult to conclude from the results due to variability.

1 INTRODUCTION

Technological advances have offered new solutions to

monitor and record physiological information such as

heart rate, breathing rate and physical activity levels,

from sensors embedded in wearable clothing.These

technological solutions are growing and evolving

rapidly to become new trends (Paradiso et al., 2005).

Hexoskin

(Carre Technlogies Inc, Quebec,

Canada) is an intelligent shirt with ﬂexible sensors

sewn directly into the fabric and it is perfectly ma-

chine washable. It is embedded with several sensors:

(1) Activity sensors (Figure 1 (a)), which record the

movements (in three dimensions) and the acceleration

of the user, (2) Respiration sensors (Figure 1 (b)), in

the form of two rings present in the chest and abdom-

inal. By analyzing the user’s thoracic and abdominal

breathing data, Hexoskin can calculate the number of

breaths per minute and the volume of air consumed

http://www.hexoskin.com/

(in L/min), and (3) Heart sensors (Figure 1 (c)), in the

form of electrodes, can act as an electrocardiogram

ECG and deliver the heart rhythm of the intelligent

textile wearer (in number of beats per minute) (Fig-

ure 1). The intelligent textile is able to measure levels

of physical activity and physiological signals in real

time. Using Bluetooth technology, these signals can

be monitored remotely from a smart device.

The Hexoskin has been developed initially for

high-level sports. However, there is a current grow-

ing interest for its use in clinical settings. Hence, the

physiological data provided by the Hexoskin need to

be validated. A few studies have considered such a

validation. To validate the Hexoskin wearable shirt

during lying, sitting, standing, and walking activi-

ties, a sample of 20 participants have been consid-

ered (Villar et al., 2015). The study revealed a low

coefﬁcient of variation and high intraclass correlation

values for heart rate, breathing rate, and hip motion

intensity while comparing the Hexoskin and labora-

tory standard devices. The magnitude of changes in

150

Cherif, N., Mezghani, N., Gaudreault, N., Ouakrim, Y., Mouzoune, I. and Boulay, P.

Physiological Data Validation of the Hexoskin Smart Textile.

DOI: 10.5220/0006588001500156

In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 1: BIODEVICES, pages 150-156

ISBN: 978-989-758-277-6

(a)

(b)

(c)

Figure 1: The Hexoskin acquisition system: (a) Activity

sensor, (b) Respiration sensor and (c) Heart sensor.

tidal volume and minute ventilation from the Hex-

oskin closely tracked those measured by the labora-

tory standard method. In another study, to measure

cardiorespiratory ﬁtness, the Hexoskin has been used

in a sample of 10 participants using a hiking situa-

tion during two ﬁeld activities at different intensities

(Montes et al., 2015). No signiﬁcant differences for

trail types were noted for maximal heart rate and res-

piratory rate. This is due to many missed data dur-

ing the data collection. In a recent study, to moni-

tor heart rate, twelve male volunteers wore the Hex-

oskin and Polar H7 heart rate sensor and they com-

pleted variable physical activities, using a stationary

training bicycle under two different climate condi-

tions (Al Sayed et al., 2017). The study revealed a

high correlation and an absence of signiﬁcant differ-

ences between the two systems in monitoring the sub-

jects’ heart rates. In another study, to investigate the

validity and reliability of the Hexoskin vest for mea-

suring respiration and heart rate, ten male elite cy-

clists conducted a maximal aerobic cycle ergometer

test using a ramped protocol (Elliot et al., 2017). Au-

thors conclude that the Hexoskin vest is sufﬁciently

valid and reliable for measurements of breathing fre-

quency and heart rate in elite athletes but the calcu-

lated minute ventilation value produces during such

exercise should be used with caution due to the lower

validity and reliability of this variable. Finally, the

maximum oxygen uptake (VO

2max

) has been evalu-

ated by (Bassett Jr and Howley, 2000; Hawkins et al.,

2007). This parameter, which can be deﬁned as the

maximum amount of oxygen that the body consumes

during an intense effort (Howley et al., 1995), is con-

sidered as the best indicator of cardiorespiratory func-

tion. Moreover, an important parameter in determin-

ing the quality of a medical instrument is the agree-

ment with a gold standard. These previous studies

have been performed on a small sample size which

can seriously limit the interpretation of the results and

the validation process. In addition, in some cases

(Al Sayed et al., 2017; Elliot et al., 2017), the data col-

lection has been performed on stationary bikes which

does not take into account the usual movements of the

upper body.

In this study, we compared cardiorespiratory pa-

rameters from the Hexoskin against standard labo-

ratory devices during a progressive maximal exer-

cise test. Therefore, the purpose was to validate car-

diorespiratory parameters measured simultaneously

by Hexoskin and a laboratory standard devices dur-

ing a progressive maximal exercise test according

to BSU/Bruce ramp (Kaminsky and Whaley, 1998)

protocol. More precisely, we will compare heart

rate (HR) from the Hexoskin and heart rate from a

reference laboratory electrocardiogram (ECG). The

breathing rate (BR) and the estimated ventilation (Ve)

will be compared between Hexoskin and a reference

laboratory metabolic cart.

2 METHODOLOGY

2.1 Participants

Thirty healthy volunteers (18 males and 12 females)

participated in the study. For 2 participants, the mea-

surements were removed from the database due to

technical issues that occurred when recording of the

sensors signals that were not in the correct format.

The problem was reported to Hexoskin for investi-

gation. This gave a total of twenty-eight volunteers

(17 males and 11 females, age 27.07± 7.25 years old,

weight 73.73 ± 13.41 kg, heigh 173.12 ± 8.15 cm,

Body Mass Index BMI 24.5 ± 3.44).

The study was approved by institutional ethics

committees, with all subjects providing written in-

formed consent before their participation. All partic-

ipants completed a sociodemographic questionnaire

and the Q-AAP + questionnaire to conﬁrm that they

were able to perform a vigorous exercise.

2.2 Data Collection

Data collection was performed at the University of

Sherbrooke exercise physiology laboratory during a

progressive maximal exercise test. Each participant

wore the smart textile Hexoskin while they were sub-

mitted to a progressive maximal exercise test using

the Cosmed Quark metabolic cart and 12 lead elec-

trocardiogram which are considered gold standard in

an exercise physiology lab.

The test was performed according to the

BSU/Bruce ramp protocol (Kaminsky and Whaley,

1998), a testing protocol commonly used in clinical

Physiological Data Validation of the Hexoskin Smart Textile

151

and research settings. The treadmill grade starts at

0% and the walking pace at 1.7 mph. Every 20 sec-

onds, the treadmill speed increases gradually (by 0 to

0.1 mph) and/or grade increases gradually (by 0 to

0.5%). Every 3 minutes during this ramp protocol,

the work rates were identical to those of the standard

Bruce protocol. Ratings of perceived exertion was as-

sessed throughout the maximum cardiorespiratory ef-

fort test using the Borg scale of 6-20 (Borg, 1982).

The recorded data (HR, BR and Ve) of the two

devices were averaged every 20 seconds leading to a

time series data of each variable. Figure 2 illustrates,

for one participant the measurements of the heart rate

(Figure 2 (a)), breathing rate (Figure 2 (b)) and venti-

lation (Figure 2 (c)) using the hexoskin and using the

reference equipment Cosmed.

100

150

200

20 60 100 140 180 220 260 300 340 380 420 460 500 540 580 620 660 700 740 780

(a)- Heart-rate- (bpm)

Cosmed

Hexoskin

Time%(seconds)

20 60 100 140 180 220 260 300 340 380 420 460 500 540 580 620 660 700 740 780

(b)-Breathi ng- rate-(/min)-

Cosmed

Hexoskin

Time%(seconds)

100

120

140

160

20 60 100 140 180 220 260 300 340 380 420 460 500 540 580 620 660 700 740 780

(c)-Ventilation-(L/min)

Cosmed

Hexoskin

Time%(seconds)

Figure 2: (a) Heart rate, (b) breathing rate and (c) ventila-

tion measurements using the Hexoskin (red color) and using

the reference equipment Cosmed (blue color) for one par-

ticipant. The x-axis represents the averaged values for each

variable every 20 seconds.

A preliminary analysis of the collected data shows

the presence of outlier data for one participant (par-

ticipant 12). Indeed the participant is muscular and

when he was to run towards the end of the test,

the pectorals make complex movements (bouncing)

and the Hexoskin’s cardiac sensors detached from the

body which leading to abnormal recording (as illus-

trated in Figure 3. In consequence, the participant

is considered as an outlier and was deleted from our

database.

100

120

140

160

180

200

Heart&rate&(Participant&12)&

Cosm ed

Hexoskin

Time&(seconds)

Figure 3: Heart rate curves for the Hexoskin and the labo-

ratory reference equipment for participant 12.

2.3 Statistical Analysis for Physiological

Data Validation

An important parameter in determining the quality

of the wearable textile Hexoskin is the agreement

with a gold standard. Agreement means the accu-

racy of this new measurement method (de Vet H.C.W.,

1998). Several statistical methods were used to test

the agreement of medical devices with quantitative re-

sults (Bland and Altman, 1983; Luiz RR, 2005).

In this study, the agreement (concordance) be-

tween the two methods of measuring Heart Rate

(HR), Breathing Rate (BR) and Ventilation (Ve) were

estimated by calculating the Intraclass Correlation

Coefﬁcient (ICC) and the Concordance Correlation

Coefﬁcient (CCC). The 95% conﬁdence interval was

estimated for ICC.

We also used the Bland Altman method to nuance,

invalidate, or conﬁrm the level of concordance quan-

tiﬁed by numerical methods. Bland-Altman (Bland

and Altman, 1986) is a method used to compare two

measurements of the same variable. It is based on the

quantiﬁcation of the agreement between two quanti-

tative measurements by studying the mean difference

and constructing limits of agreement.

2.3.1 Intra-class Correlation Coefﬁcient (ICC)

The intra-class correlation coefﬁcient (ICC) measures

the agreement and assess the reliability of medical in-

strument continuous outcomes. It was originally pro-

posed by Fisher (Fisher, 1925), who suggested to use

an analysis of variance with a separation of within-

subject and between-subject variability. There are dif-

ferent forms of ICCs. Shrout and Fleiss have pre-

BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices

152

sented six forms of ICCs (Shrout and Fleiss, 1979)

and McGraw and Wong have presented ten forms

(McGraw and Wong, 1996). The ICC is deﬁned as

the following ratio,

ρ =

+ σ

(1)

where, σ

denotes the variance between subjects, and

denotes the variance within subjects. The lower

the within subjects variability compared to the vari-

ability between subjects, the greater the intra-class

correlation coefﬁcient. The ICC varies between 0 and

1. When reliability is high, ICC is close to 1 and when

reliability is low, ICC is close to 0. It’s recommended

to calculate the conﬁdence intervals (CI) for the ICC

(Shrout and Fleiss, 1979). In addition, p-values of

ICC can be used to evaluate the accuracy and signiﬁ-

cance of the ICC. We consider as reliable an ICC with

a p-value ≤ 0.05.

2.3.2 Concordance Correlation Coefﬁcient

(CCC)

The concordance correlation coefﬁcient (CCC) quan-

tiﬁes the agreement between a new measurement

method and a gold standard measurement method

(Lin L, 1989; Lin and Kuei, 2000). The CCC com-

bines measures of both precision and accuracy. Pre-

cision is measured with the correlation coefﬁcient r.

Model accuracy is evaluated with the bias correction

factor C

. A C

equal to 1 indicates no deviation

from the 45

◦

line. The multiplication of both values

gives both precision and accuracy at the same time.

To be perfectly concordant, two measurement tech-

niques have to be perfectly correlated and also pro-

vided identical measurements, i.e., situated on the 45

◦

line through the origin.

Let x

= (x

, x

, ..., x

) denote the series of mea-

surement of the Hexoskin and y

= (y

, y

, ..., y

) de-

note the series of measurement of the reference labo-

ratory equipment for a speciﬁc participant. The CCC

is deﬁned as,

2σ

+ σ

+ (µ

− µ

)

= rC

(2)

where µ

and µ

denote the means of each series

of measurements, σ

and σ

are the corresponding

variances, and σ

is the covariance between the two

series of measurements. The CCC takes values be-

tween -1 and 1, where these values mean respectively

a perfect discordance and a perfect concordance.

McBride (McBride, 2005) suggests the following

descriptive scale for values of the concordance

correlation coefﬁcient:

Value of ρ

Strength of agreement

>0.99 Almost perfect

0.95 - 0.99 Substantial

0.90 - 0.95 Moderate

<0.90 Poor

3 RESULTS AND DISCUSSION

Experimental results were obtained using data from

28 healthy participants (as described in section 2.1).

The variables of interest are the heart rate (HR),

breathing rate (BR) and estimated ventilation (Ve),

which were measured simultaneously by Hexoskin

and a laboratory reference equipment. CCC and ICC

were calculated for the three variables of interest HR,

BR and Ve for each participant. Simulations were per-

formed using packages of the open source program-

ming language and data science environment R.

Figures 4, 5 and 6 show the frequency distribu-

tion of the obtained intra-class correlation coefﬁcient

(ICC) and concordance correlation coefﬁcient (CCC)

values from all participants.

The HR measured simultaneously for each par-

ticipant by the Hexoskin and the reference labora-

tory equipment revealed high CCC values (ρ

be-

tween 0.98 and 0.99), and signiﬁcant ICC values (ρ

between 0.98 and 0.99, p < 0.05) for almost all par-

ticipants (Figure 4). All ICC values are in the 95%

conﬁdence interval except for one participant (partic-

ipant 12) who has a low CCC (ρ

= 0.58) and a low

ICC (ρ = 0.59, p-value < 0.05).

Figure 4: Histogram of the CCC and ICC values for heart

rate, for all participants.

Physiological Data Validation of the Hexoskin Smart Textile

153

Figure 5: Histogram of the CCC and ICC values for breath-

ing rate, for all participants.

Figure 6: Histogram of the CCC and ICC values for Venti-

lation, for all participants.

The analysis of BR of each participant under ex-

ercise condition by the two measurement systems re-

vealed high CCC values (ρ

between 0.91 and 0.99)

and signiﬁcant ICC values (ρ between 0.91 and 0.99,

p < 0.05) (Figure 5). All ICC values are in the 95%

conﬁdence interval.

The estimated ventilation (Ve) measured simulta-

neously by the Hexoskin and the laboratory reference

equipment during a progressive maximal exercise test

revealed doubtful values for each of the CCC (ρ

be-

tween 0.2 and 0.99), ICC (ρ between 0.11 and 0.99)

and C

between 0.2 and 0.99 (Figure 6). It is noted

that there is a signiﬁcant difference between the two

groups male and female in relationship to the CCC

and ICC values. We observed that 8 out of 11 women

(73%) had CCC values greater than 0.9, this goes to

moderate to an almost perfect agreement according to

the adopted scale (McBride, 2005). This can be ex-

plained by the fact that the two intelligent textiles are

not made the same way for men and women.

Figure 7 shows the deviation from the 45

◦

line.

Recall that the model accuracy is evaluated with the

bias correction factor C

. A C

equal to 1 indicates

no deviation from the 45

◦

line. Figure 7 (example

1) shows no deviation from the 45

◦

line (C

than 0.99) for heart rate for one of the participants

(all participants have almost the same graphs). With

an average of ρ

of more than 0.99, this corresponds

to an almost perfect agreement with the scale pro-

posed by McBride (2005). The results obtained with

the numerical methods were conﬁrmed by the Bland-

Altman method. The Bland Altman results indicated

no deviation of the bias (mean difference) from zero

and relatively small variability with tight agreement

limits.

Figure 7 (example 2) shows no deviation from the

◦

line (C

more than 0.99) for breathing rate for

one of the participants (all participants have almost

the same graphs). With an average of ρ

of more

than 0.97, this corresponds to a substantial agree-

ment. The results obtained with the numerical meth-

ods were conﬁrmed by the Bland-Altman method, the

bias (mean difference) is close to 0 for all participants

and the agreement limits are fairly tight. Figure 7 (ex-

ample 3) shows no deviation from the 45

◦

line for

ventilation for one of the participants (female). How-

ever, example 4 (Figure 7) shows a large deviation of

the 45

◦

line for one of the participants (male).

4 CONCLUSIONS

In this study, we tested the sensor data extracted from

the wearable textile Hexoskin. Speciﬁcally, we val-

idated the cardiorespiratory variables measured with

Hexoskin and with laboratory standard metabolic cart

during a progressive maximal exercise test.

Using the Intraclass Correlation Coefﬁcient, the

Concordance Correlation Coefﬁcient and the Bland

Altman, we calculate the agreement between the two

methods which revealed good agreement and low

variability for heart rate and breathing rate. Hexoskin

provides ventilation values from estimates, which is

an important limitation of the Hexoskin.

The heart rate values are more accurate because

they are calculated directly from the ECG signal using

BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices

154

Example 1: Heart rate beats per min

Example 3: Ventilation in L/min (Female) Example 4: Ventilation in L/min (Male)

Cosmed

CosmedCosmed

Hexoskin

Example 2: Breathing rate per min

-- 45° line!

--- Regression line!

Figure 7: Graphical representation of the results of the two measurement systems for heart rate (example 1), breathing rate

(example 2) and ventilation for one of the women and one of the men (examples 3 and 4). The 45

◦

line represents the perfect

match with the reference method.

the integrated electrodes. Same for the breathing rate

which is calculated directly from the two integrated

respiration sensors in the thoracic and abdominal lev-

els. However, ventilation is an estimation provided

using the two integrated respiration sensors.

This study conﬁrm the good agreement for heart

rate and breathing rate. However, for the ventilation,

it remains difﬁcult to conclude on the validity of Hex-

oskin because the results are variable.

ACKNOWLEDGEMENTS

The authors would like to thank the Natural Sciences

and Engineering Research Council (NSERC), and the

Canada Research Chair.

REFERENCES

Al Sayed, C., Vinches, L., and Hallé, S. (2017). Validation

of a wearable biometric systemâ

Zs ability to moni-

tor heart rate in two different climate conditions under

Physiological Data Validation of the Hexoskin Smart Textile

155

variable physical activities. E-Health Telecommunica-

tion Systems and Networks, 6(02):19.

Bassett Jr, D. R. and Howley, E. T. (2000). Limiting factors

for maximum oxygen uptake and determinants of en-

durance performance. Medicine & Science in Sports

& Exercise, 32(1):70.

Bland, J. M. and Altman, D. (1983). Measurement in

medicine: the analysis of method comparison studies.

The Statistician, 32:307–317.

Bland, J. M. and Altman, D. (1986). Statistical methods for

assessing agreement between two methods of clinical

measurement. The lancet, 327(8476):307–310.

Borg, G. (1982). Psychophysical bases of perceived ex-

ertion. Medicine & Science in Sports & Exercise,

14(5):377–381.

de Vet H.C.W. (1998). Observer reliability and agreement.

John Wiley & Sons Ltd, pages 3123–3128.

Elliot, C. A., Hamlin, M. J., and Lizamore, C. A. (2017).

Validity and reliability of the hexoskin (r) wearable

biometric vest during maximal aerobic power testing

in elite cyclists. The Journal of Strength & Condition-

ing Research.

Fisher, R. A. (1925). Theory of statistical estimation.

Proceedings of the Cambridge Philosophical Society,

22(5):700–725.

Hawkins, M. N., Raven, P. B., Snell, P. G., Stray-

Gundersen, J., and Levine, B. D. (2007). Maximal

oxygen uptake as a parametric measure of cardiores-

piratory capacity. Medicine & Science in Sports &

Exercise, 39(1):103–107.

Howley, E. T., Bassett, D. R., and Welch, H. G. (1995).

Criteria for maximal oxygen uptake: review and com-

mentary. Medicine and science in sports and exercise,

27(9):1292–1301.

Kaminsky, L. A. and Whaley, M. H. (1998). Evaluation of a

new standardized ramp protocol: the bsu/bruce ramp

protocol. Journal of Cardiopulmonary Rehabilitation

and Prevention, 18(6):438–444.

Lin, L. and Kuei, I. (2000). A note on the concordance

correlation coefﬁcient. Biometrics, 56:324–325.

Lin L, K. I. (1989). A concordance correlation coefﬁcient to

evaluate reproducibility. Biometrics, pages 255–268.

Luiz RR, S. M. (2005). More than one statistical strategy to

assess agreement of quantitative measurements may

usefully be reported. Journal of Clinical Epidemiol-

ogy, 58(3):215–216.

McBride, G. (2005). A proposal for strength-of-agreement

criteria for lin’s concordance correlation coefﬁcient.

NIWA Client Report: HAM2005-062.

McGraw, K. O. and Wong, S. P. (1996). Forming inferences

about some intraclass correlation coefﬁcients. Psycho-

logical methods, 1(1):30.

Montes, J., Stone, T. M., Manning, J. W., McCune, D.,

Tacad, D. K., Young, J. C., DeBeliso, M., and Navalta,

J. W. (2015). Using hexoskin wearable technology to

obtain body metrics during trail hiking. International

journal of exercise science, 8(4):425.

Paradiso, R., Loriga, G., and Taccini, N. (2005). A wearable

health care system based on knitted integrated sen-

sors. IEEE transactions on Information Technology

in biomedicine, 9(3):337–344.

Shrout, P. E. and Fleiss, J. L. (1979). Intraclass correla-

tions: uses in assessing rater reliability. Psychological

bulletin, 86(2):420.

Villar, R., Beltrame, T., and Hughson, R. L. (2015). Vali-

dation of the hexoskin wearable vest during lying, sit-

ting, standing, and walking activities. Applied Physi-

ology, Nutrition, and Metabolism, 40(10):1019–1024.

BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices

156