Finger Motion Detection for Human Activities Recognition

using Single sEMG Channel

Yang Qian

, Ichiro Yamada

1,2

and Shin’ichi Warisawa

1,2

School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan

Graduate School of Frontier Sciences, The University of Tokyo, 5-1-5, Kashiwanoha, Kashiwa, Chiba, Japan

Keywords: sEMG, Single sEMG Channel, Finger Motion Detection, Human Activities Recognition.

Abstract: Today’s aging population has recently become a significant problem, requiring a wearable health monitor-

ing system for the elderly who are living alone. One of the focuses of this monitoring system is human ac-

tivities recognition. We propose a wearable sensing method that is based on muscle’s crosstalk information

that uses only one sEMG channel (a pair of electrodes) to recognize five basic finger motions (thumb flex-

ion, index flexion, middle flexion, ring & little flexion, and rest position) related to daily human activities.

In the first step, an inter-electrode distance (IED) experiment was conducted to define the suitable IED for

crosstalk information collection. In this experiment’s recognition part, a conventional feature extraction

method was adopted. The accuracy of each IED was compared and a suitable IED was defined (50 mm). In

the second step, we propose two new features, the summit foot range (SFR) and summits number (SN), to

represent the different patterns of finger motions’ sEMG signals and adopted the minimal Redundancy

Maximal Relevance (mRMR) feature selection method to improve the accuracy. An accuracy of over 87%

was achieved using the improved recognition methodology compared to 81.5% when using the conventional

one.

1 INTRODUCTION

The number of elderly is rapidly increasing, and

there is an urgent need for a wearable health moni-

toring system that is both safe and comfortable for

the elderly who are living alone. One of the focuses

of this monitoring system is human activities recog-

nition. Most of our daily life activities require us to

use our fingers. The motions of the five fingers of a

human hand play a leading role in detailed static

activities such as typing, reading, writing, and using

a mobile phone. Therefore, if the features of the

motions of our fingers can be accurately extracted, it

would be possible to recognize almost all human

activities from only this five fingers’ activity infor-

mation. In particular, as the flexion motions are

often the start of the finger motions while the exten-

sion motions are those that return back to the normal

state, the five fingers’ flexion motions need to be

focused on first.

Two kinds of sensing approaches have mainly

been proposed in the field of finger motion recogni-

tion, vision sensor based and non-vision sensor

based.

There are several vision sensor based approaches.

Lee et al., 2011, for example, have developed a

finger motion recognition method that detects the

finger’s angle change using a video sensor. Finger

motions like moving, clicking, or pointing can be

recognized by analyzing the contour of the tracked

finger. However, because of the immobility of the

video sensor, it is difficult to use this approach for

outdoor recognition and other kinds of moving activ-

ities recognition.

On the other hand, there are mainly two kinds of

sensors for the non-vision sensor based approach.

One is the gyroscope sensor. For example,

Schaechter et al., 2006, have developed a device to

detect finger motions based on a Micro Electro Me-

chanical Systems (MEMS) gyroscope sensor that is

positioned on the fingers. However, using plenty of

gyroscope chips and cables will decrease the flexi-

bility of the fingers and greatly affect the user’s hand

activities.

The surface electromyography (sEMG) sensor is

becoming an exciting tool for use in finger motion

recognition because it can efficiently and accurately

collect the signal from a finger’s detailed motion.

Qian Y., Yamada I. and Warisawa S..

Finger Motion Detection for Human Activities Recognition using Single sEMG Channel.

DOI: 10.5220/0004764700600067

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2014), pages 60-67

ISBN: 978-989-758-010-9

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

However, previous researches could only achieve an

acceptable level of accuracy for finger motion

recognition using multiple sEMG channels (Ishika-

wa et al., 2010; Tenore et al., 2009; Nagata et al.,

2007). The excessive use of channels not only makes

the subjects uncomfortable but also takes lots of

time for the electrodes’ placement. They used more

than one sEMG channel (a pair of electrodes) be-

cause the accuracy when using only one sEMG

channel was not acceptable. For example, Nagata et

al., 2007, have developed a finger motion recogni-

tion system that is based on 96 pairs of electrodes,

which can recognize 18 kinds of finger motions and

achieve an average accuracy of 95%. However,

when the number of electrodes was reduced to one

pair, the accuracy dropped to 33%. Therefore, how

to increase the portability without affecting the accu-

racy has become a significant research topic.

According to the above statement, we propose a

new finger motion recognition methodology using

one sEMG channel that is wearable and convenient.

This paper is organized as follows. The benefit

of using a muscle’s crosstalk information, which is

the basis of our sensing method, is described in

Section 2. The signal acquisition protocol is de-

scribed in Section 3. The recognition methodology

adopted in the inter-electrode distance (IED) exper-

iment is illustrated in Section 4. In Section 5, the

IED experiment is described and a suitable IED is

defined based on the recognition results. In Section

6, the improved recognition methodology is intro-

duced. The recognition results of the improved

recognition methodology are discussed in Section 7.

Finally, the conclusion and future works are present-

ed in Section 8.

2 MUSCLE’S CROSSTALK

A pair of electrodes is usually placed close to each

other, aiming to collect one specific muscle’s sEMG

signal without much crosstalk from the other mus-

cles. If the IED (inter-electrode distance) becomes

larger, other muscles’ crosstalk information will be

recorded. However, it was previously found that the

crosstalk can produce unique sEMG signals’ pat-

terns that are useful for classification. In addition, a

large IED can make the negative effect of elec-

trodes’ displacement smaller by detecting a signal

that contains multiple muscles’ activity information

(Hudgins et al., 1993).

In our case, as we only use one sEMG channel to

collect the finger motions’ signals, we need to record

the signals of multiple muscles together. So, we

enlarged the IED, and thus, the crosstalk information

of the muscles can be recorded.

3 SIGNAL ACQUISITION

PROTOCOL

Eight intact-limbed subjects (3 females and 5 males,

21-46 years old) with no injury history or nerve

problems on their right forearms participated in our

research.

After their right forearms were wiped with first

an alcohol tissue and then a dry tissue, a pair of

Ag/AgCl adhesive electrodes was attached in the

area around the flexor pollicis longus muscle, the

flexor digitorum superficialis muscle, and the flexor

digitorum profundus muscle, which mainly are asso-

ciated with the fingers’ flexion motions, as shown in

Table 1 (Moore et al., 2010). The center of the two

electrodes is on the midline of the forearm’s palmer

surface, and 0.75 of the distance from the wrist to

the olecranon. This placement ensures that we can

collect clear and stable signals from all three mus-

cles, which can be easily segmented.

Table 1: Forearm muscles and their corresponding fingers’

flexion motions.

Muscle Finger motions

Flexor pollicis longus Flexion of thumb

Flexor digitorum superfi-

cialis

Flexion of index, middle,

ring, and little finger

(proximal interphalangeal

joints)

Flexor digitorum profun-

dus

Flexion of index, middle,

ring, and little finger (dis-

tal interphalangeal joints)

The sEMG signal is collected by sEMG active

dipole (emgPLUX) sensor, which is connected to a

wearable signal acquisition device (BioPLUX

re-

search unit) sampling at 1 kHz with a resolution of

12 bits. This device can send the signal (in real-time)

via Bluetooth to the computer. The sEMG signal is

visually inspected on the computer to ensure that it

is stable (MonitorPlux v2.0, PLUX - Engenharia de

Biosensores, Lda.).

The five basic finger motions related to daily ac-

tivities performed by the eight subjects are: thumb

flexion, index flexion, middle flexion, ring & little

flexion, and rest position. As we seldom flex our

ring or little finger separately in daily life, the com-

FingerMotionDetectionforHumanActivitiesRecognitionusingSinglesEMGChannel

bination of the ring and little fingers’ flexion was

performed.

The subjects were asked to perform their finger

motions at a relatively fast speed like they usually

would in daily life activities. In order to cut only the

flexion motion’s signal out in the signal segmenta-

tion step, the final position of each flexion motion

was held for a period of approximately 1 s, resulting

in some muscles’ contraction signals. As to avoid

the effect of muscle fatigue, each motion was re-

peated 10 times, and the subjects had to relax for

approximately 1 min before the next motion started.

A total of 400 finger-motion data were collected.

4 CONVENTIONAL

RECOGNITION

METHODOLOGY

In this section, we illustrate a conventional recogni-

tion methodology adopted in the IED experiment, as

shown in Figure 1.

Figure 1: Conventional recognition methodology of IED

experiment.

4.1 Signal Segmentation Method

In order to cut only the flexion motion’s signal out,

we manually segmented each motion. As shown in

Figure 2, we set the signal length at 400 ms, which

ensured that we cut the flexion motion’s entire sig-

nal out. It is worth noting that the signal segmented

also contained some muscles’ contraction signals as

the subjects performed their flexion motions at dif-

ferent speeds. However, since the flexion motion’s

signal is much larger than the contraction’s signal,

the extracted features mainly belong to the flexion

motion.

4.2 Conventional Feature Extraction

Method

Figure 2: Example of sEMG signals’ segmentation pro-

cess. Each window size in the figure at the top is 400 ms.

The signal segmented in the bottom figure shows that it

also contains the muscles’ contraction signal.

It has previously been demonstrated that Time Do-

main (TD)-Autoregressive (AR) features are useful

and efficient when extracting the features of finger

motions’ sEMG signals (Al-Timemy et al., 2013;

Hargrove et al., 2007; Hudgins et al., 1993). Har-

grove et al., 2007, showed that the TD-AR features

could achieve higher performance than that of other

feature extraction methods such as Fourier transform

and wavelet transform for the detection of hand

motions with sEMG signals.

The TD-AR features we adopted were the AR

model coefficients (order 6), root mean square

(RMS), mean absolute value (MAV), waveform

length (WL), zero crossings (ZC), and slope sign

changes (SSC). It is worth noting that the dead-zone

of the zero crossings and the slope sign changes was

set to 12 mV because the noise became a little larger

after we enlarged the IED.

We adopted the overlapping window method to

extract each feature. The window size was 200 ms

15 2520105

-200

200

100

-100

t (s)

sEMG (mV)

0.20.1

-200

200

100

-100

t (s)

sEMG (mV)

0.40.3

HEALTHINF2014-InternationalConferenceonHealthInformatics

and the interval of the adjacent window was 40 ms.

The average value of each feature for all the win-

dows was calculated, resulting in 11 features for

each sample. Each time domain feature was linearly

normalized to [0,1] before inputting the classifier.

4.3 Classification and Validation

Method

The probabilistic neural network (PNN) was select-

ed as the classifier (Specht et al., 1990). PNN is a

kind of artificial neural network, which has an excel-

lent performance reputation for complex biological

signals like sEMG signals.

As for the validation, we adopted the leave-one-

out-cross-validation (LOOCV) since we have a

relatively small database (400 samples)(Cawley,

2006). In our case, LOOCV involves using a single

observation from the original samples as the valida-

tion data, and the remaining observations as the

training data. The validation was repeated 400 times

and the average accuracy was calculated.

5 INTER-ELECTRODE

DISTANCE EXPERIMENT

In this section, we compare the recognition accuracy

of different IEDs using the conventional recognition

methodology to define a suitable IED.

5.1 Inter-electrode Distance Selection

Figure 3: Placement of pair of electrodes on forearm for

IED experiment. The center of the two electrodes is on the

midline of the forearm’s palmer surface, and at 0.75 of the

distance from the wrist to the olecranon. The IED increas-

es from 30 to 80 mm at an interval of 10 mm.

We selected IEDs of 30, 40, 50, 60, 70, and 80

mm as shown in Figure 3. These IEDs correspond to

the amount of crosstalk information collected rang-

ing from only a few to large amount. In addition, 30

mm means that the two electrodes were placed very

near to each other, which is the conventional place-

ment method.

We separately collected each finger motion’s

sEMG signals from six different IEDs.

5.2 Recognition Results

The accuracy of the six different IEDs using the

conventional recognition methodology are shown in

Figure 4.

Figure 4: Recognition results from six different IEDs.

5.3 Discussion and Conclusion

As shown in Figure 4, the accuracy at 30 mm was

81.5%, which is not very satisfying. However, when

we enlarged the IED to 50 mm, the accuracy in-

creased to the highest level of 86%. So, we defined

50 mm as the suitable IED for collecting a wide

range of crosstalk information without too much

noise, while still maintaining the unique pattern of

each motion’s signal. If the IED is smaller than 50

mm, the crosstalk information of multiple muscles

cannot be fully recorded. And if the IED is larger

than 50 mm, the recorded crosstalk information is

too universal for creating unique signal patterns. In

addition to the crosstalk information, the enlarged

IED is insensitive to the variations in anatomy of the

subjects, which often causes individual differences.

6 IMPROVED RECOGNITION

METHODOLOGY

As the effectiveness of the features and the over-

training of the classifier may significantly affect the

accuracy, apart from defining the suitable IED (50

Olecranon

Wrist

IEDCenter

4030

Inter-electrode Distance (mm)

Accuracy (%)

6050

FingerMotionDetectionforHumanActivitiesRecognitionusingSinglesEMGChannel

mm), we also started to think about proposing some

new features and adopting a feature selection meth-

od to improve the level of accuracy.

Our improved recognition methodology with the

50-mm IED mainly contains two new features for

the features extraction and a feature selection meth-

od adopted to decide the optimal feature set.

In this section, we introduce the methodology that is

shown in Figure 5.

Figure 5: Improved recognition methodology.

6.1 Newly Proposed Features

In this sub-section, we propose two new features to

represent the different patterns of finger motions’

sEMG signals.

6.1.1 Summit Foot Range (SFR)

We inspected the different motions’ sEMG signals,

and found that the foot ranges of the summits are

different for different motions, as shown in Figures 6

and 7. The foot ranges can be interpreted as the

frequency information of the summits. Therefore, a

feature called the summit foot range is proposed to

represent the frequency information of these sum-

mits.

The SFR in an overlapping window is defined as

the following formula:

SFR = ∑ (Foot2 – Foot1) / N,

(1)

where Foot2 and Foot1 are the two feet of each

summit, and N represents the number of summits

found in an overlapping window.

Figure 6: Example of different patterns of finger flexion

motions’ sEMG signals. Ri&Li represents the ring & little

flexion motion.

Figure 7: Example of summits and two feet of one summit

(Foot1 and Foot2), which is 50– 250 ms of middle flexion

motion’s signal in Figure 6.

A MATLAB function called findpeaks

is adopt-

ed to find the summits.

We defined several parameters in this function

so that the patterns of different motions’ sEMG

signals can be clearly extracted.

 MINPEAKHEIGHT: In order to avoid extracting

noise’s features, the minimum height of a sum-

mit should be set. The rest position’s signal can

indicate that the amplitude of noise is 12 mV. So,

the MINPEAKHEIGHT was set to 12 mV.

 MINPEAKDISTANCE: The minimum distance

between summits was set to 3 ms to avoid mis-

detecting small peaks that occur in the neighbor-

hood of a summit.

We adopted the zero crossing method to find the

feet that are near 0 mV when the foot’s amplitude is

not exactly 0 mV (Hudgins et al., 1993).

6.1.2 Summits Number (SN)

The average number of summits found in all the

overlapping windows was also introduced as a fea-

ture to strengthen the SFR by complementing the

Thumb

Index

Ri&Li

Middle

Rest

0.20.1

t (s)

0.40.3

0.10.05

-200

300

100

-100

(

)

sEMG (mV)

0.20.15

Summit

Foot1 Foot2

200

0.25

HEALTHINF2014-InternationalConferenceonHealthInformatics

Figure 8: Example of the rest position’s signal, which

indicates the amplitude of noise. The red and dashed lines

show the MINPEAKHEIGHT of 12 mV.

frequency information of the summits. This feature

is called the summits number.

6.2 Feature Set Optimization by

mRMR

The minimal Redundancy Maximal Relevance

(mRMR) feature selection method can increase the

recognition accuracy by ranking the importance of

the features in regards to both their relevance and

information content (Peng et al., 2005).

We ranked all 13 features by adopting the

mRMR, and compared the accuracy by increasing

the number of input features from a ranking of 1 to

13, as listed in Table 2.

7 DISCUSSION

As listed in Table 2, the accuracy was the highest

when eliminating AR4, which is ranked as the low-

est feature when using the mRMR. So, the optimal

feature set is: SN, AR1, RMS, SFR, WL, AR2, ZC,

MAV, AR6, SSC, AR3, and AR5. The highest accu-

racy we have found thus far is 87.3%, compared to

81.5% for the conventional methodology with the

conventional placement method of IED of 30 mm.

In addition, when we used our improved meth-

odology to recognize the motions’ signals collected

when the IED is 30 mm, the accuracy decreased a

little, as listed in Table 3. This may be because our

newly proposed features are not very suitable for

sEMG signals collected when the IED is 30 mm.

The detailed improvement of each finger mo-

tions’ accuracy is proven by the confusion matrixes

noted in Tables 4 and 5. We determined from the

confusion matrixes that our improved recognition

Table 2: Features’ ranking using mRMR.

Ranking Feature Cumulative accuracy

1 SN 57.8%

2 AR1 71%

3 RMS 75.8%

4 SFR 76%

5 WL 78.3%

6 AR2 84%

7 ZC 84.8%

8 MAV 84.8%

9 AR6 84.8%

10 SSC 87%

11 AR3 86.8%

12 AR5 87.3%

13 AR4 86.8%

Table 3: Comparison of recognition results of convention-

al and improved methodology with IED of 30 and 50 mm.

IED

(mm)

Conventional methodology Improved methodology

30 81.5% 81.3%

50 86% 87.3%

methodology contributed to a universal increase in

almost all the motions’ accuracy. In particular, an

increase of 17.5% was achieved for the flexion of

the middle finger. This shows that our improved

recognition methodology can generally improve the

accuracy of almost all the motions, indicating that

other motions besides the five basic motions can be

accurately recognized as well.

Table 4: Finger motion recognition confusion matrix of

conventional recognition methodology (IED = 30 mm).

Motion Accuracy (%)

Thumb Index Middle Ri&Li Rest

Thumb

71.3

10 1.2 17.5 0

Index 8.8

0 1.2 0

Middle 3.7 0

21.3 0

Ri&Li 7.5 3.7 17.5

71.3

Rest 0 0 0 0

100

The confusion matrix in Table 5 helped us de-

termine that the recognition error mainly comes

from the adjacent fingers. There are basically two

explanations for this phenomenon. One is that the

subjects often could not flex a single finger without

moving the adjacent fingers, causing other finger

motions’ sEMG signals to be collected. The other

explanation is that the crosstalk information of the

adjacent muscles may still be a little universal for

creating the different patterns of the finger motions’

sEMG signals.

0.20.1

-15

-5

-10

t (s)

sEMG (mV)

0.40.3

FingerMotionDetectionforHumanActivitiesRecognitionusingSinglesEMGChannel

Table 5: Finger motion recognition confusion matrix of

improved recognition methodology (IED = 50 mm).

Motion Accuracy (%)

Thumb Index Middle Ri&Li Rest

Thumb

81.3

13.8 1.2 3.7 0

Index 5

91.3

0 2.5 1.2

Middle 0 0

92.5

7.5 0

Ri&Li 6.2 2.5 20

71.3

Rest 0 0 0 0

100

The two new features (SFR and SN) and the

mRMR together contributed to a 1.3% increase in

accuracy (87.3% compared to 86%), which is a

relatively small improvement. However, as noted in

Table 2, the mRMR ranks the SN and SFR in 1st

and 4th

place, respectively, showing they are very

effective features of the sEMG signals for finger

motion recognition. Since we did not normalize the

amplitude of the signals, SFR and SN can have a

robust performance regarding the individual differ-

ences because they are not related to the amplitude

information.

However, although the recognition results by

adopting the mRMR show that only AR4 should be

eliminated, it also indicates that if we do not need to

have the highest level of accuracy, a more compact

feature set can be selected (SN, AR1, RMS, SFR,

WL, and AR2), resulting in an accuracy of 84%.

This result shows us that by adopting the mRMR,

we can determine a relatively suitable feature set

that can significantly reduce the computing time

with only a slight decrease in accuracy.

8 CONCLUSIONS

We proposed a wearable sensing method based on

the muscle’s crosstalk information that uses only one

sEMG channel to recognize five basic finger mo-

tions (thumb flexion, index flexion, middle flexion,

ring & little flexion, and rest position) related to

daily human activities. A suitable inter-electrode

distance was defined (50 mm) from the inter-

electrode distance experiment to improve the accu-

racy. In addition, two new features were proposed

and a feature selection method was adopted, result-

ing in an accuracy of 87.3% compared to 81.5%

when using the conventional methodology with an

IED of 30 mm. Our results show that the improved

recognition methodology is not only effective for

detecting finger motions, but also is insensitive to

individual differences.

The recognition methodology still needs im-

provement. The effectiveness of our methodology in

recognizing other motions besides the five basic

motions should also be reexamined. As for its appli-

cation, we need to adopt the wearable sensing meth-

od and the improved recognition methodology for

recognizing daily human activities like typing, read-

ing, writing, and using a mobile phone.

REFERENCES

Al-Timemy, A., Bugmann, G., Escudero, J., and Outram,

N., 2013. Classification of finger movements for the

dexterous hand prosthesis control with surface elec-

tromyography. IEEE Journal of Biomedical and

Health Informatics, Vol. 17, No. 3, pages 608-618.

Cawley, G. C., 2006. Leave-one-out cross-validation

based model selection criteria for weighted LS-SVMs.

In Proceedings of the International Joint Conference

on Neural Networks (IJCNN’06), pages 1661-1668.

Hargrove, L. J., Englehart, K., and Hudgins, B., 2007. A

comparison of surface and intramuscular myoelectric

signal classification. IEEE Transactions on Biomedi-

cal Engineering, Vol. 54, No. 5, pages 847-853.

Hudgins, B., Parker, P., and Scott, R. N., 1993. A new

strategy for multifunction myoelectric control. IEEE

Transactions on Biomedical Engineering, Vol. 40, No.

1, pages 82-94.

Ishikawa, K., Toda, M., Sakurazawa, S., Akita, J., Kondo,

K., and Nakamura, Y., 2010. Finger motion classifica-

tion using surface-electromyogram signals. In Pro-

ceedings of IEEE/ACIS 9th International Conference

on Computer and Information Science (ICIS), pages

37-42.

Lee, D., and Lee, S., 2011. Vision-based finger action

recognition by angle detection and contour analy-

sis. ETRI Journal, Vol. 33, No. 3, pages 415-422.

Moore, K. L., Arthur F. D., and Anne M. R. A.,

2010. Clinically oriented anatomy. Wolters Kluwer

Health, 6

edition, pages 748-750.

Nagata, K., Ando, K., Magatani, K., and Yamada, M.,

2007. Development of the hand motion recognition

system based on surface EMG using suitable meas-

urement channels for pattern recognition. In Proceed-

ings of Engineering in Medicine and Biology Society,

29th Annual International Conference of the IEEE.

IEEE, pages 5214-5217.

Peng, H., Long, F., and Ding, C., 2005. Feature selection

based on mutual information criteria of max-

dependency, max-relevance, and min-redundancy.

IEEE Transactions on Pattern Analysis and Machine

Intelligence, Vol. 27, No. 8, pages 1226-1238.

Schaechter, J. D., Stokes, C., Connell, B. D., Perdue, K.,

and Bonmassar, G., 2006. Finger motion sensors for

fMRI motor studies. Neuroimage, Vol. 31 No. 4, pages

1549-1559.

Specht, D. F., 1990. Probabilistic neural networks. Neural

Networks, Vol. 3, No. 1, pages 109-118.

Tenore, F. V. G., Ramos, A., Fahmy, A., Acharya, S.,

Etienne-Cummings, and R., Thakor, N. V., 2009. De-

HEALTHINF2014-InternationalConferenceonHealthInformatics

coding of individuated finger movements using sur-

face electromyography. IEEE Transactions on Bio-

medical Engineering, Vol. 56, No. 5, pages 1427-1434.

Notes.

http://www.bioplux.com/home.

http://www.mathworks.co.jp/jp/help/signal/ref/findpeaks.

html?lang=en.

FingerMotionDetectionforHumanActivitiesRecognitionusingSinglesEMGChannel