MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE

ND INDIVIDUAL

Hao Zhang, Guillaume Lopez, Masaki Shuzo, Jean-Jacques Delaunay and Ichiro Yamada

School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan

eywords:

Eating habits monitoring, Wearable sensor, Mastication counting, Chewing sound, Life-style diseases

prevention.

Abstract:

In recent years, an increasing number of people have been suffering from over-weight, reminding the impor-

tance of a balanced dietetic lifestyle. Researches in nutrition and oral health have reported that not only the

calorie intake amount, but also eating speed and the number of chews per bite were also important factors in

obesity. Automatic mastication counting systems based on chewing sound processing have been proposed,

though most of them have difﬁculties in detecting chewing strokes for various food types, and often require

training logic or threshold that need to be customized for each user. To overcome these problems, we have de-

veloped a new model for automatic mastication counting based on new chew feature extraction and detection

methods from natural chewing sound. Chewing sounds collected from 15 persons eating six different food

types were recorded using a wearable bone-conduction microphone placed in ear. The chewing sound anal-

ysis model combining proposed chew feature extraction and detection methods was applied on the collected

data set, showing a good overall accuracy while having better stability to different individuals and food types

comparing to conventional models.

1 INTRODUCTION

In recent years, the importance of dietary habits mon-

itoring for preventing lifestyle-related diseases has

been increasing rapidly, drawing many researchers at-

tention in modern society, since over-weight, which

is increasing dramatically among all ages groups, has

been proved to be related to many other diseases such

as hypertension, diabetes, and heart diseases (Abra-

ham et al., 1971; Hoffmans et al., 1989). Recent stud-

ies revealed that not only the calorie intake and con-

sumption balance, but also eating speed and the num-

ber of chews per bite were also important factors for

obesity (Nicklas et al., 2001).

Clinical studies have lead to develop a kind of

weight scale for plates called Mandometer

, which al-

lows to know the eating amount per bite and eating

speed. (Yasutomi and Masuda, 2008) have developed

a device called ”kamikami sensor” to automatically

count and display the number of mastication during

eating. And the attention of researchers has started to

focus on analyzing eating habits using internal body

sounds such as (Amft et al., 2005; Nishimura and

Kuroda, 2008; Shuzo et al., 2010).

In this paper, we describe and report about the ef-

ﬁciency of our proposed model for stable mastication

counting based on sound signal using a new chew-

ing features extraction method. This paper is orga-

nized as follows. In section 2, the characteristics of

chewingsound during eating process and assumptions

based on these characteristics are demonstrated. In

section 3, all the steps of the data analysis method for

the whole mastication counting process based on the

physical properties are presented. In section 4, exper-

iments and the data preparation for validation are de-

scribed. In section 5, the model construction is illus-

trated by using data from experiments. Finally, a com-

parison study with conventional mastication counting

models is shown in section 6. The conclusions are

presented in section 7.

2 ASSUMPTIONS

2.1 Characteristics of the Chewing

Process

The basic mechanics of mastication have been stud-

ied in detail by Amft et al. (Amft et al., 2005). They

demonstrate that the structure of a single chewing seg-

ment into a chewing sequence is mainly composed of

374

Zhang H., Lopez G., Shuzo M., Delaunay J. and Yamada I..

MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE AND INDIVIDUAL.

DOI: 10.5220/0003771903740377

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2012), pages 374-377

ISBN: 978-989-8425-88-1

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

four phases: The closing of mandible for crushing

the food(P1), a small pause(P2), the opening of the

mandible(P3) in which food that stick to the teeth is

uncompressed, and then another pause(P4).

2.2 Assumptions

According to the characteristics of chewing process,

the following two assumptions are made to construct

our model.

• The amplitude of P1 is larger than P3 in one chew-

ing segment;

• The duration of P4 is longer than P2 in one chew-

ing segment.

3 DATA ANALYSIS METHODS

3.1 Chew Pattern Extraction

The objective of chew pattern extraction is to ﬁnd fea-

tures that can differentiate a chew in the chewing pro-

cess. For all the following pre-processing methods

for chew pattern extraction, we use a common 20 ms

signal frame with no overlap.

3.1.1 Log Energy

Calculation of log energy based on a 20 ms frame is

adopted in order to enlarge the differences among a

chew and pauses in the chewing process.

3.1.2 Zero Crossing Times

Zero crossing

times is a characteristic speech sig-

nal processing method to extract fundamental fre-

quency, which consists in counting the number of

sign-changes along the signal. As the frequency of

a chew is different from the one of pauses, zero cross-

ing times is a good descriptor since it is a rough ap-

proximation of the spectrum characteristics.

3.1.3 Amplitude Differences Accumulation

According to the objective to extract the features that

could demonstrate best the change of a chew, we pro-

posed a new method called Amplitude Differences

Accumulation (ADA), to amplify more clearly the

difference between chews and pauses. The ADA of

the n

frame can be illustrated by the following for-

mula.

ADA

n×N

∑

m=(n−1)×N+1

|x(m) − x(m− 1)|, (1)

where x represents the sound signal, N is the number

of sampling points in each frame.

3.2 Signal Smoothing

The signal from chew pattern extraction may contain

some small vibrations caused by the noise covering

the sound signal. A Butterworth low pass ﬁlter (LPF)

was adopted for the purpose of noise reduction, the

parameters were ﬁxed to 4th order ﬁlter and 2.5 Hz

cut off frequency, considering the reasonable max-

imum chewing cycles per second for human that is

limited by the physical mechanics of the mandible.

3.3 Peak Detection

The peak detection algorithm is to ﬁnd the local max-

ima in a certain interval. We adopt a Matlab function

called ﬁndpeaks

to ﬁnd peaks.

3.4 Rules for Mastication Counting

In the real eating process, the characteristics of eating

signal change a lot due to different situations. How-

ever, according to the eating sound properties, the dif-

ferent signal characteristics can be summarized to the

following two situations.

3.4.1 Situation 1

In this situation, usually happened during eating very

hard or crispy food, the characteristics of a chewing

segment can be demonstrated as one of the following

two possibilities.

• P1 and P3 merge so that P2 disappear;

• P1 and P3 are not merge, but P2 is very narrow.

In that case, after our proposed eating sound sig-

nal processing procedures, the main frequency com-

ponent is only one low frequency.

3.4.2 Situation 2

In the case of food types other than very hard or

crispy, the eating sound signal has not so obvious

characteristics like in situation 1. Indeed, P3 may be

detected as a chew even if the situation obeys to the

assumptions we made based on the physical proper-

ties of the eating process.

Our strategy is to adopt a rule that controls peak

detection according to the characteristics of chewing

sound and the assumptions. The rule consists in regu-

lating the number of chews detected to no more than

three chews in one second, and also not detecting

MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE AND INDIVIDUAL

375

chews that are too near from the previous one. To sat-

isfy these rules, the next peak detected has to be sepa-

rated by at least 15 frames of 20 ms after the frame in

which the previous value have been calculated. The

ﬁxed threshold is in the same direction as time, and

respects the chewing mechanics that there cannot be

more than three chews in one second and the P2 stage

is narrow.

4 EXPERIMENTS

4.1 Sensing Devices

A prototype of a wearable sensing system using bone-

conduction sensor and IC recorder to analyse eating

habits has been developed in previous work (Shuzo

et al., 2010).

4.2 Experiments and Data Collection

Experiments were conducted to collect data for the

validation of our proposed chew pattern extraction

method and rule-based model. Hereafter is a list de-

scribing the experimental conditions.

• The quantity of food intake for one chewing pro-

cess is not deﬁned, participants can eat according

to their will.

• Variety kinds of food with different textures are

included in the experiments.

• Participants eat ﬁve times for each food type in

the experiments.

This experiment has 15 participants, and eating sound

of 6 food types was recorded. We chose the food

types carefully in order to cover different food texture.

Through this experiment, we can obtain 450 sound

ﬁles for establishing the model for mastication count.

5 MODEL CONSTRUCTION

5.1 Introduction

In this section, the different chew pattern extraction

methods demonstrated in section 3 are compared. The

model was constructed based on the best performing

chew pattern extraction method. The detailed perfor-

mances are also reported in this section. The com-

pared models are methods combination of log energy

(LE), zero crossing times (ZCT), amplitude differ-

ences accumulation (ADA), and rule.

5.2 Comparison Study

The overall error rate was calculated based on the ex-

perimental. The performance was examined regard-

ing four parameters, which detailed results are re-

ported in Table 1.

5.3 Discussion

According to the results, the proposed rule can largely

increase the overall accuracy of chews detection.

Also, among the chew pattern extraction methods, the

proposed method called ADA (amplitude differences

accumulation) was proved to perform better.

6 COMPARISON STUDY WITH

CURRENT MASTICATION

COUNTING MODELS

In this section, our model has been compared with

two other kinds of models referenced from Nishimura

et al. and Shuzo et al. works (Nishimura and Kuroda,

2008; Shuzo et al., 2010). The utmost limit of perfor-

mance that each model can achieve were compared.

The details of these models are described in the fol-

lowing sub sections.

6.1 Models for Comparison

6.1.1 Low Passed Filter based Model (Shuzo

et al., 2010)

The model was conducted for the extraction of the

mastication number from the eating sound data. The

number of mastication was counted by adopting a low

pass ﬁlter and peak detection method. There are two

settings in this model. The ﬁrst is the cut-off fre-

quency of the ﬁlter was set to a constant value of

1.5 Hz, the second setting is to change the cut-off

frequency according to the power spectrum of sound

data at that moment.

6.1.2 Log Energy based Model (Nishimura and

Kuroda, 2008)

For this model, log energy was calculated based on 20

ms frame, and then a 4 order Butterworth ﬁlter with

cut-off frequency of 2 Hz was applied to the signal.

There is a threshold for sensitivity control for each

person. The number of mastication can be obtained

by counting the number of times regression coefﬁ-

cients calculated using low pass ﬁlter output values

HEALTHINF 2012 - International Conference on Health Informatics

376

Table 1: Results of comparisons using 5 parameters evaluation.

Comparison Overall Lowest error rate Highest error rate Lowest error rate Highest error rate

situations No. error rate(%) for individual(%) for individual(%) for food type(%) for food type(%)

LE + LPF 35 7 57 16 (Salad) 61 (Fruit jelly)

ZCT + LPF 41 8 61 31 (Salad) 62 (Fruit jelly)

ADA + LPF 30 7 55 13 (Salad) 60 (Fruit jelly)

LE + LPF + rule 8 2 17 4 (Salad) 12 (Banana)

ZCT + LPF + rule 11 3 20 8 (Fruit jelly) 43 (Banana)

ADA + LPF + rule 7 3 14 2 (Salad) 11 (Marshmallow)

of 9 frames crosses zero with a negative value slope

and the local peak of low pass ﬁlter output is larger

than the sensitivity threshold.

6.2 Comparison Results

The overall accuracy of the mastication counting

models were compared (Table 2). The low pass ﬁl-

ter based model with ﬁxed cut-off frequency of 1.5

Hz was named as M1, the low pass ﬁlter based model

with adaptive cut-offfrequencywas named as M2, the

log energy based model was named as M3, and the

model proposed in section 5 was named as M.

Table 2: Comparison of overall accuracy for 4 models.

Methods M M1 M2 M3

Accuracy(%) 93 55 85 77

7 CONCLUSIONS

In this paper, we proposed a data analysis model for

realizing automatic mastication counting for any indi-

vidual and food type with high accuracy (93%). The

proposed model reduces the inﬂuence of individual

chewing ways differences (speed, strength, etc.) to a

large extent, and is little inﬂuenced by chewing sound

of different food types (soft, hard, crispy, etc.).

ACKNOWLEDGEMENTS

This research was supported by Japan Science and

Technology agency (JST)’s strategic sector for cre-

ation of advanced integrated sensing technologies for

realizing safe and secure societies: research project

on ”Development of a Physiological and Environmen-

tal Information Processing Platform and its Applica-

tion to the Metabolic Syndrome Measures”.

REFERENCES

Abraham, S., Collins, G., and Nordsieck, M. (1971). Re-

lationship of childhood weight status to morbidity in

adults. In HSMHA Health Reports. United States.

Health Services and Mental Health Administration.

Amft, O., Stger, M., and Trster, G. (2005). Analysis of

chewing sounds for dietary monitoring. In Proceed-

ings of 7th International Conference on Ubiquitous

Computing. Springer.

Hoffmans, A. F., Kromhout, D., and de Lezenne Coulande,

C. (1989). Body mass index at the age of 18 and its

effects on 32-year-mortality from coronary heart dis-

ease and cancer. a nnested case-control study among

the entire 1932 dutch male birth cohort. Journal of

Clinical Epidemiology Vol. 42, Issue 6, pages 513–

520.

Nicklas, T., Baranowski, T., Cullen, K., and Berenson, G.

(2001). Eating patterns, dietary quality and obesity.

Journal of the American College of Nutrition Vol. 20,

Issue 6, pages 599–608.

Nishimura, J. and Kuroda, T. (2008). Eating habits moni-

toring using wireless wearable in-ear microphone. In

Proceedings of International Symposium on Wireless

Pervasive Communications (ISWPC08). IEEE.

Shuzo, M., Komori, S., Takashima, T., Lopez, G., Tatsuta,

S., Yanagimoto, S., Warisawa, S., Delaunay, J.-J., and

Yamada, I. (2010). Wearable eating habit sensing sys-

tem using internal body sound. Journal of Advanced

Mechanical Design, Systems, and Manufacturing Vol.

4, Issue 1, pages 158–166.

Yasutomi, K. and Masuda, Y. (2008). Example of food

education for young japanese obese by wearing the

’kamikami sensor’ while eating. Journal of Japanese

Society for Masticatory Science and Health Promo-

tion Vol. 18, Issue 2, pages 149–150.

Notes

http://www.mando.se/en/start-page/start-page-/1.aspx

http://en.wikipedia.org/wiki/Zero crossing

http://www.mathworks.com/help/toolbox/signal/

ﬁndpeaks.html

MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE AND INDIVIDUAL

377