Thumbs-Up

Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits

Asaph Azaria, Brian Mayton and Joseph Paradiso

Media Lab, Massachusetts Institute of Technology, 75 Amherst st, Cambridge, MA, U.S.A.

Keywords:

Biomedical Computing, Bioimpedance, Sensor Applications, Artiﬁcial Intelligence, Wearable Computers.

Abstract:

Thumbs-Up explores a novel sensing method for detection of hand-to-mouth compulsive habits. Using elec-

trical bio-impedance spectroscopy and inertial measurement units, a prototype system was implemented. The

system can easily be worn around the arm and may perhaps be integrated into future wearable devices. It

recognises occurrences of the habits in real-time, allowing monitoring and immediate interventions. These

have so far been very limited, impeding behavioural studies and the development of therapeutic treatments.

Throughout this paper the method’s feasibility is demonstrated and aspects of its performance are explored.

We present an approach to process the bio-impedance signals and associate them with possible body postures.

A positioning strategy optimises the device’s sensitivity and increases its efﬁcacy. Machine learning algo-

rithms are leveraged to infer the hand-to-mouth detection. We achieve 92% detection accuracy for recurrent

usage and 90% accuracy for users that have not been previously encountered.

1 INTRODUCTION

Hand-to-mouth compulsive habits, such as thumb

sucking and nail biting, are surprisingly common.

Studies found that 28-33% of children, 44% of ado-

lescents, and 19-29% of young adults engage in nail

biting alone (Tanaka et al., 2008). These compulsive

behaviours expose those who exhibit them to multi-

ple health risks. Threats range from expedited trans-

mission of diseases, to dental malocclusion and even

abnormal facial development (Baydas¸ et al., 2007).

Studies into the roots of these habits and their

consequences have so far been limited to either sub-

jective reports or lab settings. Treatment has been

constrained to retrospective correction, relying on

self awareness and manual monitoring (Azrin and

Nunn, 1973). A wearable device, detecting hand-to-

mouth habits in real-time, would mitigate these im-

pediments, advancing therapeutic interventions and

personal health monitoring. With aetiologies includ-

ing anxiety, loneliness, frustration, and more, such a

device could even serve as a diagnostic tool to reﬂect

on quality of life impairments (Pacan et al., 2014).

This work presents a wearable hand-to-mouth

classiﬁcation system, consisting of an inertial mea-

surement unit (IMU) and an electrical bio-impedance

spectrometer. It can easily be worn around the arm.

The device tracks the arm’s orientation and electri-

cally excites it, discovering changes in its impedance

properties. These are leveraged to infer hand-to-

mouth behaviours in real-time.

To the best of our knowledge, this is the ﬁrst sys-

tem to demonstrate automated detection of hand-to-

mouth habits. Furthermore, it is innovative in util-

ising bio-impedance spectroscopy for this goal. A

related detection scheme is described in US Patent

6762687 B2. Rather than sensing physical contact,

this scheme senses proximity between tagged body

parts. Bio-impedance has been widely studied in the

medical context for estimating biophysical quantities

such as body composition, metabolic functioning, and

cardiac activity. Recent research has demonstrated

integrating bio-impedance measurements into wear-

able devices – monitoring biophysical markers (Sepp

et al., 2007) and even estimating users’ biometrics

(Martinsen et al., 2007). None, however, leveraged

bio-impedance technology for behaviour recognition,

as our system uniquely does.

2 THEORY OF OPERATION

2.1 Electrical Properties of Biomaterials

Electrical bio-impedance (EBI) describes the electri-

cal properties biological materials exhibit as current

Azaria, A., Mayton, B. and Paradiso, J.

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits.

DOI: 10.5220/0005680900540065

In Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2016) - Volume 1: BIODEVICES, pages 54-65

ISBN: 978-989-758-170-0

ﬂows through them. It is commonly measured by

injecting a small sinusoidal alternating current (AC)

into the tissue under study. The injected current in-

duces an electrical ﬁeld within the tissue and results

in a measurable voltage drop across it. The AC ver-

sion of Ohm’s Law (Equation 1) relates the material’s

electrical bio-impedance Z to the ratio between the

measured voltage V and the injected current I.

Z =

(1)

It is a complex quantity, since biomaterials not only

oppose current ﬂow, but also store electrical charge,

phase-shifting the voltage with respect to the current

in the time-domain.

When current ﬂows through a tissue, it passes

through extracellular and intracellular ﬂuids. These

ﬂuids are highly conductive as they contain salt ions

that can easily be displaced by a potential difference.

Conversely, the cells’ lipid membranes are insulators.

They act like capacitive plates, which prevent electri-

cal charges from ﬂowing through. Accordingly, the

tissue’s impedance reﬂects its chemical composition,

membrane structures, and ﬂuids distribution. For sim-

ilar reasons, the speciﬁc cell types (blood, adipose,

muscle, bone, etc.), the anatomic conﬁguration (i.e.,

bone or muscle orientation and quantity), and the state

of the cells (normal or osteoporotic bone, oedematous

vs. normally hydrated tissue, etc.) greatly affect mea-

sured impedance quantities (Gabriel et al., 1996).

Most tissues display dispersive characteristics, i.e.

their impedance varies with the frequency of the ap-

plied current. A typical dispersion curve is illus-

trated in Figure 1. It is displayed as a Cole-Cole plot

which superimposes impedance measurements from

a range of frequencies on the complex plane. At low

frequencies, the cells’ membranes block the current.

Thus, the impedance corresponds only to the extracel-

lular resistance. As frequency increases, more current

passes through the intracellular capacitive path, and

the phase angle accumulates. At high frequencies,

the intracellular capacitance becomes negligible. The

impedance is once again purely resistive, dominated

by the intracellular and extracellular ﬂuid resistances

connected in parallel. The frequency at which the tis-

sue’s reactance reaches a peak is known as the centre

frequency ( f

2.2 The Cole-Cole Equivalent Circuit

Model

The simplest electrical circuit that can be used to

model EBI response is presented in Figure 2. R

and

represent the intracellular current branch and R

Figure 1: Typical Cole-Cole plot of biological tissue.

Figure 2: Simple circuit model for biological tissues.

represents the extra cellular one. This model results

in a perfect semicircle in a Cole-Cole plot, with the

centre of the circle on the resistance axis.

In real tissue, however, the cells’ membrane is an

imperfect capacitor. Moreover, the large variation in

cell type, structure, and size causes a distribution of

the cells’ capacitive time constants (Cornish et al.,

1993). Cole and Cole showed that when capacitive

time constant distribution is added to the aforemen-

tioned circuit model, the impedance is related to the

frequency by (Cole and Cole, 1941):

Z = R

∞

− R

∞

1 + ( jωτ)

1−α

(2)

where,

= R

(3)

∞

= R

||R

(4)

τ = (R

+ R

(5)

This model preserves the circular shape in a Cole-

Cole plot, but depresses the circle’s centre below the

resistance axis. α has a value between 0 and 1, and is

proportional to the angle to the depressed centre.

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits

2.3 Electrode Sensitivity Field and

Conﬁguration

Two types of electrode systems are commonly used

to obtain EBI measurements. A 2-electrode system

uses the same pair of electrodes to inject current (IC)

and pick up (PU) the tissue’s response. A 4-electrode

system uses different pairs for excitement and pick up.

The type of electrodes in use (needle or skin surface,

gel or dry, etc.), and their conﬁguration around the

tissue affect the sensed impedance almost as much as

the electrical properties of the tissue.

The electrode type determines how the electrical

conductor in the measurement leads interfaces with

the ionic conductor in the biological tissue. As current

ﬂows, substance concentration may change near the

electrodes’ interface, adding bias impedance called

electrode polarisation. The skin-electrode contact in-

troduces an additional resistive bias. The 4-electrode

system is a robust setup that reduces the inﬂuence of

these factors (Seoane et al., 2008). When voltage pick

up is implemented with high-impedance differential

ampliﬁers, such artefacts can be neglected.

The electrode conﬁguration sets boundary values

on the electrical ﬁelds that develop inside the tissue.

Thus, it governs the ﬁelds’ propagation and in effect,

the relative contribution of internal tissue regions to

the measured mutual impedance. Geselowitz formu-

lated this idea for a 4-electrode system (Geselowitz,

1971). He established that the measured impedance

Z resulting from the variable conductivity σ within a

volume conductor can be evaluated by:

Z =

Sdv (6)

Sensitivity S is a scalar ﬁeld, determining the con-

tribution of a local conductivity change (∆σ) to the

overall potential. It can be calculated from the dot

product of two current density ﬁelds:

S =

(7)

represents the current density ﬁeld generated by a

unit current applied through the IC electrodes.

the reciprocal current density ﬁeld that would have

been generated had the same current been injected

through the PU electrodes.

Depending on the angle of the two ﬁelds, there

can be regions where the sensitivity is zero, posi-

tive, or negative. Hence, the tissue regions, in which

impedance changes are measured, can effectively be

manipulated by the electrode conﬁguration. Note that

the measured mutual impedance is indifferent to in-

terchanges between the IC and PU electrodes. In this

context, this phenomenon is commonly referred to as

the Reciprocity Theorem.

Figure 3: Images of the prototype system (From top: IMU

wristband, EBI spectrometer, and PC Interface).

3 SYSTEM DESCRIPTION

Inspired by how pervasively electrical ﬁelds propa-

gate inside biological tissues, we set out to examine

if EBI spectroscopy can be applied to sense hand-to-

mouth behaviours. We hypothesised that placing the

hand inside the mouth will form different paths for

the current to traverse inside the body, resulting in

distinct impedance spectra. We presumed these can

be analysed, detecting the placement of the hand in-

side the mouth. In light of the multiple factors that

manifest in an EBI spectrum, we explored machine-

learning techniques for making such distinctions. A

similar technique, learning from capacitive signals,

has previously been used to classify simple gestures

for human-computer interfaces, suggesting promise

to this approach (Sato et al., 2012).

To test this hypothesis and assess its validity,

we built a prototype system. Images of the sys-

tem are presented in Figure 3. We present the sys-

tem’s design in detail throughout the following sec-

tion. When necessary, we discuss alternative imple-

mentation schemes and possible improvements.

3.1 Hardware

The hardware consists of an EBI spectrometer and

an IMU, wired to a nearby personal computer via

USB. The bio-impedance spectrometer was imple-

mented on a printed circuit board (PCB) and was

BIODEVICES 2016 - 9th International Conference on Biomedical Electronics and Devices

based on Analog Devices’ AD5933 on-chip network

analyser. The analyser contains a frequency generator

capable of outputting a sinusoidal excitation voltage.

The response signal is picked up by the analyser and

processed by an on-chip Discrete Fourier Transform.

Real and imaginary 16-bit data words, proportional to

the measured impedance, are returned for each output

frequency. A frequency range between 1 to 100 kHz

is supported, with a 0.1 Hz resolution.

We interfaced the AD5933 with the analogue

front-end presented in (Seoane et al., 2008). This

front-end, illustrated in Figure 4, converts the

AD5933 from a 2-electrode to a 4-electrode measure-

ment system. Hence, it cancels impedance contribu-

tions from electrode polarisation and skin-electrode

coupling. An instrumentation ampliﬁer, voltage fol-

lower, and a hardwired 20 kΩ resistor construct a volt-

age controlled current source (VCCS), which mod-

iﬁes the AD5933 excitation from being voltage to

current driven. An auxiliary high-pass-ﬁlter (HPF)

blocks DC voltage from ﬂowing into the tissues. This

way, current is controlled and regulated well be-

low hazardous levels. Frequencies are limited above

3.5 kHz to fully comply with IEC-60601 safety guide-

lines. AC current of 174 µA (RMS) was conﬁrmed by

an independent measurement in our system.

An ATmega328P micro-controller controls the

AD5933, via I

C, and obtains the impedance mea-

surements. It is used to execute parts of our algo-

rithms, assessing their performance on an embedded

platform. We followed the layout by (Blomqvist et al.,

2012) to power the board and communicate with the

micro-controller over USB.

This layout does not restrict power consumption

and requires 0.5W for its operation. It is directly

wired to a PC, and so requires inefﬁcient power and

communication isolation, that will not be necessary in

a battery powered wearable device. The basic compo-

nents in use, however, are well suited for low-power

applications. Typical power consumption of the AT-

Mega328P and the AD5933 are 33mW and 20mW,

respectively. Likewise, the analogue front-end can be

adjusted to require minimal power. Future versions

could easily revise power and communication to ﬁt

the requirements of a wearable device.

We used 1” adhesive gel electrodes (Covidien

H124SG), commonly used in ECG and EMG, for

our experiments. These surface electrodes allow non-

invasive and precise ﬁxture of the electrodes to the

arm. The contact electrolyte gel is of less impor-

tance to our application. It is designed to mitigate

the high impedance of the skin at lower frequencies,

typically under 1 kHz. By removing the DC compo-

nent from the excitation signal and alternating to a

Figure 4: Block diagram of the analogue front-end de-

scribed by Seoane et al.

4-electrode conﬁguration, the polarisation character-

istics of the particular electrodes in use can also be

neglected (Seoane et al., 2008).

We therefore expect that a future change to dry

electrodes – relying on sweat as the electrolytic so-

lution – will not signiﬁcantly affect our results. Dry

electrodes will lower our system’s cost and improve

its unobtrusiveness and reusability. It will, however,

have to assume the continuous natural perspiration of

the skin. Otherwise, the sweat electrolytes, carrying

our system’s current, can be depleted over time.

The IMU is attached to a wristband-like strap

and worn on the user’s forearm. It streams mea-

surements of the forearm’s orientation and displace-

ment. We used an MPU9150, which comprises a 3-

axis accelerometer, a gyroscope, and a magnetometer

in a single chip. The chip was placed on a breakout

board and connected to an off-the-shelf Arduino with

an identical ATMega328P micro-controller. In future

versions, the IMU can easily be connected to the spec-

trometer’s I

C bus, embedding all sensors on a single

board.

3.2 Software

3.2.1 Calibration

The impedance measurements – real and imaginary

16-bit data words for each exciting frequency – are

ﬁrst scaled and calibrated. The calibration accounts

for the complex impedance introduced by our sys-

tem’s various electronic components – between the

AD5933 and the studied tissues.

A system gain factor scales the magnitude of each

impedance measurement. Such scaling can be formu-

lated as:

tissue

(ω)| =

Magnitude

Gain Factor

R (ω)

+ I (ω)

G(ω)

(8)

We used a 2R-1C circuit with known impedance, like

in Figure 2, to derive the frequency-dependent gain

factor of our system. Similarly, the system phase off-

set is derived from a circuit with pure resistance. It

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits

factors in as a frequency dependent bias when cali-

brating the impedance measurements:

∠ Z

tissue

(ω) = tan

−1

(

I (ω)

R (ω)

) − ∠ system(ω) (9)

The IMU allows programmable full-scale ranges

to each of its components. Based on the arm’s char-

acteristic motion, we set these ranges to ±250 dps,

±2 g, and ±1200 µT for the gyroscopes, accelerom-

eter, and magnetometer respectively. We measured

offsets in the angular velocity and linear acceleration

with a simple gimbal. Each of the 9 measurements

were scaled and corrected accordingly.

3.2.2 Feature Extraction

Features for hand-to-mouth detection are extracted

from calibrated EBI and IMU measurements. These

features are listed in Table 1.

Table 1: List of extracted features.

EBI features IMU features

Circle centre (X

) Quaternion orient. (q

)

Circle centre (Y

) Quaternion orient. (q

)

Circle radius (R) Quaternion orient. (q

)

Inﬁnity impedance (R

∞

) Quaternion orient. (q

)

Static impedance (R

)

Depression angle (α)

Centre frequency ( f

)

Fit accuracy (Var{R

})

EBI Features

We chose the Cole-Cole model to compactly repre-

sent the EBI data, using its parameters as features. To

calculate them, the calibrated impedance samples are

mapped to the impedance plane in a Cole-Cole plot.

Such mapping was exempliﬁed in Figure 1. Each data

point on the plane corresponds to the resistance and

reactance measured at a single frequency. We seek to

ﬁt these points to the Cole equation – a perfect semi-

circle with the centre depressed below the resistance

axis.

We leverage the circular shape to estimate the pa-

rameters of the Cole-Cole model. Instead of solving

directly for the model parameters (R

∞

, R

, τ, α and

), we estimate the circular curve that best represents

our data. We do so by ﬁnding a point (X

) that

minimises the variance in the distances R

, measured

from it to the N impedance data points.

min Var{R

} = min Var{|(X

) − Z

} (10)

This point will be the Cole-Cole model circle cen-

tre. A closed form solution for such a minimisation

is obtained by differentiating Equation 10. The Cole-

Cole model parameters, R

∞

, R

and α, can then be

solved geometrically from the circle centre. A de-

tailed derivation is found in (Ayll

on et al., 2009).

The centre frequency f

is determined as pre-

sented in the work of (Cornish et al., 1993). Lengths

of cords u

and v

, which respectively join each

impedance data point i with R

∞

and R

, are calcu-

lated. The impedance points are then projected to a

log(u

) vs. log(ω

) plane, where they yield a line

with slope (1 − α). The x-axis intercept of this line

determines the centre frequency.

We calculate the variance in R

and afﬁx it as an

additional parameter to our EBI data representation.

It serves as an indicator of how accurately our data

ﬁts to a circular curve. Later, it will be utilised

when performing real-time classiﬁcation, to identify

movement artefacts and discard them.

IMU Features

We used an AHRS (Attitude and Heading Reference

System) sensor-fusion algorithm to handle the IMU

measurements. The algorithm produces a four-vector

quaternion representation of the device’s orientation

in 3D. Rather than using computationally expen-

sive Kalman-Filter implementations, we chose the

sensor fusion algorithm developed by Madgwick

(Madgwick et al., 2011). This algorithm derives

simple gradient descent steps to conduct fusion and

estimations iteratively, allowing it to operate on

computationally constrained platforms compatible

with wearable devices.

Execution

The extraction algorithms were designed to execute

in real time. The orientation state is updated con-

tinuously, processing IMU measurements at a 400 Hz

sampling rate. The EBI feature extraction algorithm

processes a frequency sweep at a 0.64 Hz rate. We

conﬁgured the frequency sweep to obtain 50 samples

equally spaced between 4 kHz and 100 kHz. The fre-

quency resolution can be modiﬁed to boost the sweep-

ing rate at the expense of the ﬁt accuracy. The sweep-

ing rate determines the rate at which the feature vec-

tors are generated. The EBI features are ﬁrst extracted

and then extended with the current four-vector quater-

nion orientation.

The feature extraction algorithms were imple-

mented both for a personal computer and for the AT-

mega328P micro-controllers. We experimented with

executing them on both. Running on the micro-

controller reduces dramatically (factor 18) the data to

be communicated to the next stage. It does so, how-

ever, at the cost of an increased computational load.

BIODEVICES 2016 - 9th International Conference on Biomedical Electronics and Devices

We experienced no performance issues executing the

algorithms on the ATmega328P, when the sampling

rates were set as previously mentioned. We leave the

decision between the execution alternatives open for

future applications.

3.2.3 Hand-to-Mouth Detection

The ﬁnal stage of our software allows three modes of

operation: Collection mode logs real-time collected

data and extracted features into a local ﬁle for later

analysis. A human observer labels the data points

with a binary label. Positive and negative labels cor-

respond to situations when the hand is inside and out-

side the mouth. Enrolment mode accepts previously

collected labeled data either from a speciﬁc user or a

group of users. It optimises a set of machine-learning

algorithms and chooses the one which is most likely

to perform best. Prediction mode makes real-time

predictions on the hand’s situation. It uses the opti-

mal classiﬁer from the previous mode to predict on

each feature vector as it is produced. The predictions,

as well as the arm’s orientation, are visualised to the

user as feedback.

Our system allows learning via any binary-

classiﬁcation model, as long as it implements a sim-

ple fit, score and predict interface. We embedded

the S cikit − learn implementation of ﬁve classiﬁca-

tion models in our system: Random Forests (RF),

Support Vector Machines (SVM), AdaBoost (AB), K-

Nearest Neighbours (KNN) and Logistic Regression

(LR) (Pedregosa et al., 2011). The models were cho-

sen to intentionally differ in their underlying statis-

tical assumptions and algorithmic implementations.

Hence, they differ not only in the accuracy they may

achieve, but also in the computational complexity

their training or prediction involve.

We optimise the unique parameters of the classi-

ﬁcation models, by searching a grid of possible val-

ues. A classiﬁer instance is trained for each parameter

combination. How well it generalises is assessed us-

ing 3-iteration random subsampling cross-validation

(Murphy, 2012). Once the optimal parameters per

classiﬁcation model are set, we choose among the

models by their mean validation accuracy.

Prediction mode introduces two additional mech-

anisms to facilitate the real-time prediction and feed-

back. First, it discards data points whose Cole-Cole

circular ﬁt accuracy is below a threshold (i.e. high

Var{R

}). These were observed during natural move-

ment, in singular cases – when by the time a fre-

quency sweep is over, the user has already changed

between postures with different EBI spectra. Clearly,

ﬁtting to the inconsistent measurements does not

truthfully represent the electrical path taken by the

current. Discarding them, therefore, prevents unnec-

essary prediction errors. Second, a moving average

window on the predicted labels was implemented,

smoothing the visual feedback presented to the user.

4 EVALUATION

In this section we demonstrate the system’s feasibil-

ity by evaluating various aspects of its performance.

Note that the prototype system leaves many degrees

of freedom in its application – from conﬁguring the

electrodes on the forearm, to when it is worn and

by whom. We therefore tackle this challengingly

broad space by breaking the evaluation into three

stages. First, we present an approach to process the

bio-impedance signals and verify they carry informa-

tion about hand-to-mouth behaviours. We associate

them with possible body postures, narrowing down

the scope of movement for our experiments. Second,

an electrode positioning strategy is proposed, optimis-

ing the system’s sensitivity to conductivity changes

around the ﬁngers. Finally, the system’s detection ac-

curacy is assessed, analysing how it performs in two

possible use cases with multiple subjects.

4.1 EBI of Hand-to-Mouth Behaviours

4.1.1 Method

The ﬁrst stage explored how a wearer’s hand-to-

mouth behaviours manifest in the measured EBI spec-

tra. To this end, we have conducted an experiment

with a single subject in an indoor working environ-

ment. The subject wore the device for 5 consecutive

hours, and data were recorded as he naturally moved

in this everyday setting. Periodically, the subject was

asked to place any of his 5 ﬁngers in his mouth. An

external observer annotated the recorded data with

the subject’s activity and body postures. Throughout

this experiment, the electrodes were placed in a single

conﬁguration on the subject’s forearm, marked (10,1)

as will later be detailed in Section 4.2.

We utilised the recorded data to narrow down the

scope of movement for the next stages of this work.

This satisﬁes experimental control and reproducibil-

ity. The subject’s imitation of natural movement cap-

tures most possible orientations of the forearm, as

well as the variance of bioelectrical measurements.

Special attention was paid to ensure that the dataset

also included situations where different body parts

come in contact with the subject’s ﬁngers. These are

important as they create different paths through which

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits

Figure 5: Sample EBI spectra generated by different body postures.

current can traverse, hence changing the characteris-

tics of the EBI spectrum.

4.1.2 Results and Discussion

The EBI recordings supported our detection hypoth-

esis. Noticeable differences were observed between

situations where the subject’s hand was inside or out-

side the mouth. These manifest not only for spe-

ciﬁc exciting frequencies, but indeed throughout the

entire spectrum. Fitting the spectra to a Cole-Cole

model, we observe satisfactory differences in many

of the model’s parameters. For example, the static

impedance R

demonstrated a mean decrease of 32%

when juxtaposing sitting with placing the thumb in-

side the mouth.

As mentioned, we were particularly interested in

postures where different parts of the body come in

contact with the ﬁngers. We experimented with an

assortment of these, in an attempt to identify ones

which create unique EBI spectra. Our experimen-

tation was limited to contact with body parts which

were exposed wearing full clothing. We exemplify

three such postures – crossing the arms, clasping the

hands, and leaning the head on the hand – in Figure

5. EBI spectra from sitting, standing and placing any

of the 5 ﬁngers inside the mouth are overlaid for com-

parison.

Note that the EBI spectrum generated by clasp-

ing the hands resembles the ones generated by plac-

ing any ﬁnger inside the mouth. In some instances,

we noticed a circular spectrum almost identical to one

from a hand-to-mouth posture. Our design trusts dif-

ferences in the orientation components to settle these

ambiguities.

Supporting this assumption, we analysed differ-

ences in the system’s detection accuracy, restricting

it to rely only on a subset of components from the ex-

tracted features. Classiﬁcation models were trained,

as indicated in Section 3.2.3, for three cases: includ-

ing only the orientation components, only the EBI

components, or all available components. We then

compared the models by their prediction accuracies,

classifying an independent test set. This set was col-

lected by repeating the aforementioned experiment

with the same subject on a different day. Detection

accuracies of 74.39%, 90.74% and 95.76% were ob-

tained for each of the cases respectively. They imply

the suitability of the different sensors for our detec-

tion task, and bolster our decision to combine them.

Finally, we studied the measured EBI spectra to

select 10 representative body postures. The ﬁrst ﬁve

were tied with the placement of each of the wearer’s

5 ﬁngers inside the mouth. The remaining postures

were selected to generate the highest diversity in their

corresponding EBI spectra. Crossing the arms, clasp-

ing the hands, and leaning the head on the hand were

chosen by this criterion. Sitting still and standing

were chosen due to their high rate of occurrence in the

dataset. The last two represent EBI spectra similar to

walking and typing which were the most frequently

observed postures.

BIODEVICES 2016 - 9th International Conference on Biomedical Electronics and Devices

Figure 6: Electrode Instrumentation on a subject’s forearm

used to evaluate an optimal positioning.

4.2 Positioning Optimisation

The second stage examined possible positioning of

the electrodes on the arm. We limited the electrodes’

arrangements to the forearm, as only such arrange-

ments would make practical sense for wearing or

wiring in future applications.

4.2.1 Instrumentation

Figure 6 illustrates how a single subject was instru-

mented with 20 electrodes. In accord with the muscu-

lar anatomy of the forearm, the electrodes were lined

in two bands. Electrodes following the posterior and

anterior fascial compartments were marked with L

and R respectively. Within each band, the electrodes

were indexed from 1 to 10 starting from the elbow.

The mean distance between the centre of consecutive

electrodes was 2.66 cm.

Though each pair of electrodes is interchangeable

up to a sign, and the PU and IC electrodes are in-

terchangeable by the Reciprocity Theorem, it is still

infeasible to exhaustively evaluate each of the 19,380

possible electrodes’ combinations. Therefore, we in-

troduced a constraint that the PU and the IC electrodes

are conﬁgured in a crossed alignment, in which the

current density ﬁelds are intersecting with each other.

Figure 7 illustrates such an arrangement. The IC (+)

electrode and the PU (-) electrode were always as-

signed to the R band. Correspondingly, the IC (-)

electrode and the PU (+) electrode were always as-

signed to the L band. Forming an intersection, the (+)

pair and the (-) pair were restricted each to maintain

the same index along the forearm. We use the elec-

Figure 7: Schematic illustration of a crossed alignment of

the electrodes. The propagation lines are a coarse approxi-

mation of the induced current density ﬁelds.

trodes’ indices to denote their conﬁguration. Conﬁg-

uration (10,1), for example, marks an electrode con-

ﬁguration where that the (+) and (-) electrodes are in

indices 10 and 1 respectively.

The assumption that the electrodes should be

crossed is not veriﬁed within the scope of this work.

However, it builds on our intuition of the current den-

sity ﬁelds (

and

) induced by the electrode sys-

tem. A coarse approximation of these is visualised

in Figure 7. Unlike most cases where conductiv-

ity changes are circumscribed by the electrodes, we

would like to sense changes further away – in regions

which are electrically coupled by the touch of the ﬁn-

gers. The crossed alignment, therefore, attempts to

nullify the sensitivity ﬁeld (dot product) inside the

forearm by creating antiparallel components between

the current density ﬁelds. On the other hand, it at-

tempts to increase the sensitivity as we go further

away – where the ﬁelds become more parallel.

4.2.2 Method

Our experiment varied the distance between the (+)

and (-) electrodes, as well as their position along the

forearm. Starting with conﬁguration (10,1), we de-

creased the distance in three different ways, evaluat-

ing each conﬁguration we encounter:

• Moving the (+) pair towards the elbow – from

conﬁguration (10,1) to (2,1).

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits

• Moving the (-) pair towards the wrist – from con-

ﬁguration (10,1) to (10,9).

• Symmetrically moving both pairs towards each

other – from conﬁguration (10,1) to (6,5).

Each electrode conﬁguration was evaluated by two

separate metrics.

First, we compared the electrode conﬁgurations

by their sensitivity to changes around the ﬁngers. We

calculated ﬂuctuations in the measured impedance, as

we alternated the body tissues that the ﬁngers touch.

By proxy, the more intense the ﬂuctuations were, the

more conductivity changes around the ﬁngers con-

tributed to the overall impedance. We used the 10

body postures to represent a ﬁxed set of body tissues

that come in contact with the ﬁngers. For each elec-

trode conﬁguration, an identical number of samples

were recorded in every posture. Fluctuations were

then calculated by determining the coefﬁcient of vari-

ation for the impedance magnitude in the recording:

Coefﬁcient of variation =

std{|Z|}

mean{|Z|}

(11)

Second, we assessed the electrode conﬁgurations

by their predictive power. We held a second record-

ing session with the same subject on a different day

and re-instrumented his arm with the electrodes. The

EBI spectra, generated by the various postures in this

session, served as an independent test set. In this

experiment, we only used the EBI features, discard-

ing the orientation components. This way, our results

only reﬂect the electrical dissimilarities between the

electrode conﬁgurations. A classiﬁcation model was

trained and its optimal parameters were selected, us-

ing data only from the ﬁrst recording session. The

electrode conﬁgurations were compared with their

prediction accuracy on the test set.

4.2.3 Results and Discussion

Figure 8 presents the coefﬁcient of variation in the

impedance magnitude obtained for each of the eval-

uated conﬁgurations. The presented results were de-

rived from the impedance sampled with a single ex-

citation frequency 52 kHz. Similar trends were ob-

served measuring with other excitation frequencies

within the system’s range.

It is evident that the coefﬁcient of variation gradu-

ally increases as the electrodes in the pairs go further

from each other. This ﬁnding is consistent with the

theoretical sensitivity ﬁeld model that was previously

reviewed. The further the electrodes are, the vaster the

current density ﬁelds spread inside the tissue, grant-

ing distant regions, such as around the ﬁngers, higher

relative contribution to the overall impedance. The

Figure 8: Coefﬁcient of variation of the electrode conﬁgu-

rations. The overlaid Indices pairs mark speciﬁc conﬁgura-

tions of interest.

Figure 9: Predictive power of the electrode conﬁgurations.

region circumscribed by the electrodes also appears

to have an effect on the coefﬁcient of variation. This

can be identiﬁed by studying the three graphs of Fig-

ure 8 against each other. Notice how the coefﬁcient is

higher when the circumscribed region is closer to the

wrist rather than the elbow.

A second assessment was based on the conﬁgura-

tions’ predictive power. Figure 9 plots the prediction

accuracies achieved by the optimal classiﬁcation al-

gorithms after selecting their parameters. Conﬁgura-

tion (10,1) yields powerful accuracy results as antici-

pated by its coefﬁcient of variation. Nonetheless, the

BIODEVICES 2016 - 9th International Conference on Biomedical Electronics and Devices

best accuracy results (up to 97.6%) are encouragingly

produced by conﬁguration (6,5). As it is geometri-

cally concentrated, such conﬁguration can greatly fa-

cilitate a wearable design for practical applications.

We chose to proceed to the next stage of the eval-

uation with two electrode conﬁgurations: (10,1) due

to its high coefﬁcient of variation, and (6,5) due to

its highest prediction accuracy. Detection accuracies

from both are presented in the next section. We have

also conducted experiments in which the IC and PU

electrodes were not constrained to move symmetri-

cally or together. These are not included here, since

they revealed neither an observable trend nor higher

sensitivity or accuracy.

4.3 Detection Accuracies

The ﬁnal stage analysed aspects of our system’s ro-

bustness, using data collected from multiple subjects

on multiple days. Within the scope of this work, we

explored two probable use cases – recurrent usage by

the same user and encountering previously unseen

users for the ﬁrst time.

Recurrent Usage

First, we investigated situations when the same user

wears the system on separate days. This captures

plausible imprecisions from recurrent usage of the

system, like inconsistent placement of the electrodes

or misalignment of the IMU. Additionally, it accounts

for transient changes in bioelectrical properties of the

wearer’s forearm. These may introduce perturbations

of over 8% in an EBI spectrum (Gabriel et al., 1996).

They are likely to stem from temporal variations in

ﬂuid volumes, blood pressure, body temperature,

nervous activity, hydration, electro-dermal activity,

etc. (Schwan, 1956)

Previously Unseen Users

Next, we examined how well the system generalises,

performing on new subjects that have not been pre-

viously seen. This tests the system’s tolerance to

person-to-person differences, including differences in

adipose, muscle and vasculature ratios, bone anatomy,

sub-dermal water content and skin thickness. Such

generalisation is of crucial importance for future de-

sign of practical applications. It involves implications

on the system’s capability to scale, deploying to a

broad user base without a need to ﬁrst collect their

data and customise per individual.

4.3.1 Method

We recruited 15 subjects, 8 females and 7 males, to

participate in a pilot user study. The study was ap-

proved by IRB protocol #1504007088. All subjects

were between the ages 21 and 33 (mean=26.5; σ =

3.6). Their forearm length was measured and ranged

between 22 cm and 27.5 cm (mean=24.8; σ = 1.7).

They were asked for their weight, which varied be-

tween 47 kg and 77 kg (mean=58.3; σ = 9.4), and

their height which varied between 155 cm and 185 cm

(mean=169; σ = 8.9).

The subjects were invited twice to identical ses-

sions on two separate days over the course of a sin-

gle month. At the beginning of each session, they

were instrumented with the electrode conﬁgurations

selected in Section 4.2, on their left forearm. The sub-

jects’ forearm length was measured, and the place-

ment of each electrode within the combination was

scaled accordingly. Data were then recorded as the

subjects were asked to pose in the postures from Sec-

tion 4.1. Positive and negative labels were respec-

tively assigned to postures when the subjects’ hand

was inside or outside their mouth.

An additional subtlety was introduced for postures

in which the hand was placed inside the mouth. For

each ﬁnger, measurements were taken both when the

ﬁnger was fully sucked (up to the middle phalanx)

and when its tip was only lightly touching the inner

lip. This extension intends to simulate dissimilarities

between thumb sucking and nail biting, and veriﬁes

the system’s ability to detect both.

We framed the evaluation as a binary-

classiﬁcation problem, and report the system’s

detection performance in terms of classiﬁcation

accuracy. According to the evaluated use case, the

dataset was partitioned for training, validation and

testing purposes. Prior to any analysis, all data points

were scaled using feature standardisation (Murphy,

2012). In total, we have collected 10,483 data points;

Each of which is a Cole-Cole representation of an

EBI spectrum, comprising 50 complex impedance

measurements, and a corresponding orientation.

6,348 of the data points had a positive label and 4,134

had a negative one.

4.3.2 Results and Discussion

Recurrent Usage

First, we established how the system generalises

when the same individual recurrently wears it. We did

so by tailoring a classiﬁer per user. The ﬁrst recording

session is randomly partitioned into training and val-

idation datasets; Data recorded in the second session

was held-out as a test set. We searched the param-

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits

eter space of each classiﬁcation model, as indicated

in Section 3.2.3. The combination of algorithm and

parameters, which optimised the performance on the

validation set, was selected as the subject’s personal

classiﬁer.

Table 2 reports the median, average and standard

deviation of the test accuracy scores obtained by the

personal classiﬁers. It determines that when recur-

rently worn by a user, our system is expected to ac-

complish a median accuracy of up to 92.95%, detect-

ing hand-to-mouth behaviours. Encouragingly, elec-

trode conﬁguration (6,5) performed equally well as

conﬁguration (10,1). We regard the median statistic

as the most relevant one for our evaluation, as it is the

most robust to subjects which are potentially outliers

in our user base.

We also examined a condition when the personal

classiﬁers were constrained to train only with a spe-

ciﬁc classiﬁcation model. In this case, only the

model’s parameters were optimised per user. For con-

ﬁguration (10,1), similar median accuracies were ob-

tained when using only KNN classiﬁers and when op-

timising with all possible models. The same emerges

for conﬁguration (6,5), constraining its training only

to RF classiﬁers. This ﬁnding implies, that a single

classiﬁcation model may be used to generalise for

recurrent usage. Relevant accuracy scores of these

constrained cases are also presented in Table 2. We

state, however, that a larger user base, which yields a

smaller distance between the median and mean statis-

tics, should be analysed to conﬁrm such observations.

We highlight that these accuracies were obtained

from training with a single usage. An alternative

approach may suggest training with multiple usages

to potentially increase detection accuracies. Despite

its rigour, we chose the former, trusting that future

applications would greatly beneﬁt from requiring not

more than a single usage for training. Otherwise, the

users’ enrolment procedure can become cumbersome

and impractical.

Previously Unseen Users

Next, we assessed the system’s performance when en-

countering subjects it has not previously seen. We

partitioned our dataset, holding-out data points from

3 subjects (20%) for testing purposes. The remaining

subjects were used for training and validation. Per

subject, we included data points from the two record-

ing sessions, diversifying the datasets with multiple

usages. As in Section 3.2.3, the optimal combination

of model and parameters was selected by its valida-

tion accuracy. Its test accuracy was derived from mak-

ing predictions on the previously unseen test subjects.

Due to the relatively small number of subjects, the

Table 2: Detection accuracies for recurrent usage by the

same user. Baseline accuracies were generated by a strat-

iﬁed dummy classiﬁer.

Conﬁg. Classiﬁer Test accuracies

Median Mean (σ)

(10,1) Optimal 92.95% 90.98% (6.3)

KNN 92.15% 88.31% (6.5)

Baseline 53.42% 53.4% (3.3)

(6,5) Optimal 91.6% 90.96% (7.1)

RF 91.6% 85.47% (9.2)

Baseline 53.83% 52.9% (3.2)

particular assignment of test subjects is likely to af-

fect the resulting accuracy. Therefore, instead of ran-

domly selecting an assignment, we chose to exhaus-

tively search through all possible ones. This approach

gains a more credible insight into the system’s capa-

bility to generalise person-to-person differences. It

protects from arbitrary bias that may mislead our eval-

uation. In Table 3 we report the median, average and

standard deviation of the test accuracies obtained by

the optimal classiﬁers as they predicted for their cor-

responding assignments.

Our results suggest that our system is likely to pro-

duce an accuracy of 87.5%, predicting hand-to-mouth

behaviours for subjects it has never before seen. Re-

grettably, these high accuracy results were only pro-

duced for electrode conﬁguration (10,1). Conﬁgu-

ration (6,5), performed signiﬁcantly worse, yielding

median accuracy of only 79.69%. For two thirds of

the subjects’ assignments, RF was selected as the op-

timal model regardless of the electrode conﬁguration.

This indicates that the RF model may be the ﬁttest one

for this kind of task.

We continued our analysis, exploring the possi-

bility of adjusting the classiﬁers by the users’ phys-

icality. We clustered our subjects into two groups,

based on their gender, age, weight, height and fore-

arm length. A 2-component Gaussian Mixture Model

clustering technique was used, utilising Expectation

Minimisation (EM) to ﬁt the subjects’ physicality

measurements. Except for a single subject, the re-

sulting clusters overlapped with the subjects’ genders.

We mark the clusters F and M respectively, according

to the majority of females and males in their popula-

tion.

We repeated the previous analysis for each of the

clusters separately. To preserve the ratio from the for-

mer analysis, a single subject was held-out for test-

ing from the M cluster, and a pair were held-out from

cluster F. The median, mean and standard deviations

of the test accuracies are also presented in Table 3 for

comparison.

Successful improvements of up to 3% in the me-

BIODEVICES 2016 - 9th International Conference on Biomedical Electronics and Devices

Table 3: Detection accuracies predicting for users that have

not been previously seen.

Conﬁg. Dataset Test accuracies

Median Mean (σ)

(10,1) All 87.5% 87.21% (5.1)

Cluster F 90.52% 87.96% (6.3)

Cluster M 90.72% 88.14% (6.1)

(6,5) All 79.69% 79.81% (5.5)

Cluster F 83.25% 84.7% (7.9)

Cluster M 82.09% 82.82% (4.4)

dian accuracies were observed employing this ap-

proach. They suggest that person-to-person differ-

ences can be ameliorated by an a priori query of the

user’s physicality. It could be possible for future ap-

plications to optimise multiple classiﬁers based on

physical typecasts. A mixture of those can then be

employed for unseen users, consequently improving

expected accuracy. The mixture should be weighted

by the users’ similarity to the physical typecasts pro-

duced while training. Future work should further ex-

plore this possibility with a larger user base.

5 CONCLUSION

This work has presented a wearable system detecting

hand-to-mouth behaviours in real time. It demon-

strated a novel sensing method, associating bio-

impedance spectra with the wearer’s behaviour. The

relationship between the system’s sensitivity and its

electrode conﬁguration has been investigated, opti-

mising the latter to potentially increase detection ef-

ﬁcacy. The system’s performance has been evalu-

ated, examining use cases where it is recurrently worn

and where it encounters a user for the ﬁrst time. It

achieved a median detection accuracy of 92% and

90%, respectively. These, we aspire, may be sufﬁcient

to guide new directions in the treatment and monitor-

ing of compulsive hand-to-mouth habits.

REFERENCES

Ayll

on, D. et al. (2009). Cole equation and parame-

ter estimation from elect. bioimpedance spectroscopy

measurements-a comparative study. In Eng. in

Medicine and Biology Soc., 2009. EMBC 2009. Ann.

Int. Conf. of the IEEE, pages 3779–3782. IEEE.

Azrin, N. and Nunn, R. (1973). Habit-reversal: A method

of eliminating nervous habits and tics. Behaviour re-

search and therapy, 11(4):619–628.

Baydas¸, B. et al. (2007). Effect of a chronic nail-biting habit

on the oral carriage of enterobacteriaceae. Oral micro-

biology and immunology, 22(1):1–4.

Blomqvist, K. et al. (2012). An open-source hardware for

elect. bioimpedance measurement. In Electron. Conf.

(BEC), 2012 13th Biennial Baltic, pages 199–202.

IEEE.

Cole, K. S. and Cole, R. H. (1941). Dispersion and absorp-

tion in dielectrics i. alternating current characteristics.

The J. of Chemical Physics, 9(4):341–351.

Cornish, B. et al. (1993). Improved prediction of extracel-

lular and total body water using impedance loci gener-

ated by multiple frequency bioelect. impedance anal-

ysis. Physics in medicine and biology, 38(3):337.

Gabriel, S. et al. (1996). The dielectric properties of biolog-

ical tissues: Ii. measurements in the frequency range

10 hz to 20 ghz. Physics in medicine and biology,

41(11):2251.

Geselowitz, D. B. (1971). An application of electrocardio-

graphic lead theory to impedance plethysmography.

Biomedical Eng., IEEE Trans. on, (1):38–41.

Madgwick, S. O. et al. (2011). Estimation of imu and marg

orientation using a gradient descent algorithm. In Re-

habilitation Robotics (ICORR), 2011 IEEE Int. Conf.

on, pages 1–7. IEEE.

Martinsen, O. G. et al. (2007). Utilizing characteristic elect.

properties of the epidermal skin layers to detect fake

ﬁngers in biometric ﬁngerprint systems - a pilot study.

Biomedical Eng., IEEE Trans. on, 54(5):891–894.

Murphy, K. P. (2012). Machine learning: a probabilistic

perspective. MIT press.

Pacan, P. et al. (2014). Onychophagia is associated

with impairment of quality of life. Acta dermato-

venereologica, 94(6):703–706.

Pedregosa, F. et al. (2011). Scikit-learn: Machine learn-

ing in Python. J. of Machine Learning Research,

12:2825–2830.

Sato, M. et al. (2012). Touch

e: enhancing touch interaction

on humans, screens, liquids, and everyday objects. In

Proc. of the SIGCHI Conf. on Human Factors in Com-

puting Systems, pages 483–492. ACM.

Schwan, H. P. (1956). Elect. properties of tissue and cell

suspensions. Advances in biological and medical

physics, 5:147–209.

Seoane, F. et al. (2008). An analog front-end enables elect.

impedance spectroscopy system on-chip for biomedi-

cal applicat. Physiological measurement, 29(6):S267.

Sepp

a, V. et al. (2007). Measuring respirational parame-

ters with a wearable bioimpedance device. In 13th

Int. Conf. on Elect. Bioimpedance and the 8th Conf.

on Elect. Impedance Tomography, pages 663–666.

Springer.

Tanaka, O. M. et al. (2008). Nailbiting, or onychophagia: a

special habit. American J. of Orthodontics and Dento-

facial Orthopedics, 134(2):305–308.

Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits