MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE
A
ND INDIVIDUAL
Hao Zhang, Guillaume Lopez, Masaki Shuzo, Jean-Jacques Delaunay and Ichiro Yamada
School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
K
eywords:
Eating habits monitoring, Wearable sensor, Mastication counting, Chewing sound, Life-style diseases
prevention.
Abstract:
In recent years, an increasing number of people have been suffering from over-weight, reminding the impor-
tance of a balanced dietetic lifestyle. Researches in nutrition and oral health have reported that not only the
calorie intake amount, but also eating speed and the number of chews per bite were also important factors in
obesity. Automatic mastication counting systems based on chewing sound processing have been proposed,
though most of them have difficulties in detecting chewing strokes for various food types, and often require
training logic or threshold that need to be customized for each user. To overcome these problems, we have de-
veloped a new model for automatic mastication counting based on new chew feature extraction and detection
methods from natural chewing sound. Chewing sounds collected from 15 persons eating six different food
types were recorded using a wearable bone-conduction microphone placed in ear. The chewing sound anal-
ysis model combining proposed chew feature extraction and detection methods was applied on the collected
data set, showing a good overall accuracy while having better stability to different individuals and food types
comparing to conventional models.
1 INTRODUCTION
In recent years, the importance of dietary habits mon-
itoring for preventing lifestyle-related diseases has
been increasing rapidly, drawing many researchers at-
tention in modern society, since over-weight, which
is increasing dramatically among all ages groups, has
been proved to be related to many other diseases such
as hypertension, diabetes, and heart diseases (Abra-
ham et al., 1971; Hoffmans et al., 1989). Recent stud-
ies revealed that not only the calorie intake and con-
sumption balance, but also eating speed and the num-
ber of chews per bite were also important factors for
obesity (Nicklas et al., 2001).
Clinical studies have lead to develop a kind of
weight scale for plates called Mandometer
1
, which al-
lows to know the eating amount per bite and eating
speed. (Yasutomi and Masuda, 2008) have developed
a device called ”kamikami sensor to automatically
count and display the number of mastication during
eating. And the attention of researchers has started to
focus on analyzing eating habits using internal body
sounds such as (Amft et al., 2005; Nishimura and
Kuroda, 2008; Shuzo et al., 2010).
In this paper, we describe and report about the ef-
ficiency of our proposed model for stable mastication
counting based on sound signal using a new chew-
ing features extraction method. This paper is orga-
nized as follows. In section 2, the characteristics of
chewingsound during eating process and assumptions
based on these characteristics are demonstrated. In
section 3, all the steps of the data analysis method for
the whole mastication counting process based on the
physical properties are presented. In section 4, exper-
iments and the data preparation for validation are de-
scribed. In section 5, the model construction is illus-
trated by using data from experiments. Finally, a com-
parison study with conventional mastication counting
models is shown in section 6. The conclusions are
presented in section 7.
2 ASSUMPTIONS
2.1 Characteristics of the Chewing
Process
The basic mechanics of mastication have been stud-
ied in detail by Amft et al. (Amft et al., 2005). They
demonstrate that the structure of a single chewing seg-
ment into a chewing sequence is mainly composed of
374
Zhang H., Lopez G., Shuzo M., Delaunay J. and Yamada I..
MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE AND INDIVIDUAL.
DOI: 10.5220/0003771903740377
In Proceedings of the International Conference on Health Informatics (HEALTHINF-2012), pages 374-377
ISBN: 978-989-8425-88-1
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
four phases: The closing of mandible for crushing
the food(P1), a small pause(P2), the opening of the
mandible(P3) in which food that stick to the teeth is
uncompressed, and then another pause(P4).
2.2 Assumptions
According to the characteristics of chewing process,
the following two assumptions are made to construct
our model.
The amplitude of P1 is larger than P3 in one chew-
ing segment;
The duration of P4 is longer than P2 in one chew-
ing segment.
3 DATA ANALYSIS METHODS
3.1 Chew Pattern Extraction
The objective of chew pattern extraction is to find fea-
tures that can differentiate a chew in the chewing pro-
cess. For all the following pre-processing methods
for chew pattern extraction, we use a common 20 ms
signal frame with no overlap.
3.1.1 Log Energy
Calculation of log energy based on a 20 ms frame is
adopted in order to enlarge the differences among a
chew and pauses in the chewing process.
3.1.2 Zero Crossing Times
Zero crossing
2
times is a characteristic speech sig-
nal processing method to extract fundamental fre-
quency, which consists in counting the number of
sign-changes along the signal. As the frequency of
a chew is different from the one of pauses, zero cross-
ing times is a good descriptor since it is a rough ap-
proximation of the spectrum characteristics.
3.1.3 Amplitude Differences Accumulation
According to the objective to extract the features that
could demonstrate best the change of a chew, we pro-
posed a new method called Amplitude Differences
Accumulation (ADA), to amplify more clearly the
difference between chews and pauses. The ADA of
the n
th
frame can be illustrated by the following for-
mula.
ADA
n
=
n×N
m=(n1)×N+1
|x(m) x(m 1)|, (1)
where x represents the sound signal, N is the number
of sampling points in each frame.
3.2 Signal Smoothing
The signal from chew pattern extraction may contain
some small vibrations caused by the noise covering
the sound signal. A Butterworth low pass filter (LPF)
was adopted for the purpose of noise reduction, the
parameters were fixed to 4th order filter and 2.5 Hz
cut off frequency, considering the reasonable max-
imum chewing cycles per second for human that is
limited by the physical mechanics of the mandible.
3.3 Peak Detection
The peak detection algorithm is to find the local max-
ima in a certain interval. We adopt a Matlab function
called findpeaks
3
to find peaks.
3.4 Rules for Mastication Counting
In the real eating process, the characteristics of eating
signal change a lot due to different situations. How-
ever, according to the eating sound properties, the dif-
ferent signal characteristics can be summarized to the
following two situations.
3.4.1 Situation 1
In this situation, usually happened during eating very
hard or crispy food, the characteristics of a chewing
segment can be demonstrated as one of the following
two possibilities.
P1 and P3 merge so that P2 disappear;
P1 and P3 are not merge, but P2 is very narrow.
In that case, after our proposed eating sound sig-
nal processing procedures, the main frequency com-
ponent is only one low frequency.
3.4.2 Situation 2
In the case of food types other than very hard or
crispy, the eating sound signal has not so obvious
characteristics like in situation 1. Indeed, P3 may be
detected as a chew even if the situation obeys to the
assumptions we made based on the physical proper-
ties of the eating process.
Our strategy is to adopt a rule that controls peak
detection according to the characteristics of chewing
sound and the assumptions. The rule consists in regu-
lating the number of chews detected to no more than
three chews in one second, and also not detecting
MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE AND INDIVIDUAL
375
chews that are too near from the previous one. To sat-
isfy these rules, the next peak detected has to be sepa-
rated by at least 15 frames of 20 ms after the frame in
which the previous value have been calculated. The
fixed threshold is in the same direction as time, and
respects the chewing mechanics that there cannot be
more than three chews in one second and the P2 stage
is narrow.
4 EXPERIMENTS
4.1 Sensing Devices
A prototype of a wearable sensing system using bone-
conduction sensor and IC recorder to analyse eating
habits has been developed in previous work (Shuzo
et al., 2010).
4.2 Experiments and Data Collection
Experiments were conducted to collect data for the
validation of our proposed chew pattern extraction
method and rule-based model. Hereafter is a list de-
scribing the experimental conditions.
The quantity of food intake for one chewing pro-
cess is not defined, participants can eat according
to their will.
Variety kinds of food with different textures are
included in the experiments.
Participants eat five times for each food type in
the experiments.
This experiment has 15 participants, and eating sound
of 6 food types was recorded. We chose the food
types carefully in order to cover different food texture.
Through this experiment, we can obtain 450 sound
files for establishing the model for mastication count.
5 MODEL CONSTRUCTION
5.1 Introduction
In this section, the different chew pattern extraction
methods demonstrated in section 3 are compared. The
model was constructed based on the best performing
chew pattern extraction method. The detailed perfor-
mances are also reported in this section. The com-
pared models are methods combination of log energy
(LE), zero crossing times (ZCT), amplitude differ-
ences accumulation (ADA), and rule.
5.2 Comparison Study
The overall error rate was calculated based on the ex-
perimental. The performance was examined regard-
ing four parameters, which detailed results are re-
ported in Table 1.
5.3 Discussion
According to the results, the proposed rule can largely
increase the overall accuracy of chews detection.
Also, among the chew pattern extraction methods, the
proposed method called ADA (amplitude differences
accumulation) was proved to perform better.
6 COMPARISON STUDY WITH
CURRENT MASTICATION
COUNTING MODELS
In this section, our model has been compared with
two other kinds of models referenced from Nishimura
et al. and Shuzo et al. works (Nishimura and Kuroda,
2008; Shuzo et al., 2010). The utmost limit of perfor-
mance that each model can achieve were compared.
The details of these models are described in the fol-
lowing sub sections.
6.1 Models for Comparison
6.1.1 Low Passed Filter based Model (Shuzo
et al., 2010)
The model was conducted for the extraction of the
mastication number from the eating sound data. The
number of mastication was counted by adopting a low
pass filter and peak detection method. There are two
settings in this model. The first is the cut-off fre-
quency of the filter was set to a constant value of
1.5 Hz, the second setting is to change the cut-off
frequency according to the power spectrum of sound
data at that moment.
6.1.2 Log Energy based Model (Nishimura and
Kuroda, 2008)
For this model, log energy was calculated based on 20
ms frame, and then a 4 order Butterworth filter with
cut-off frequency of 2 Hz was applied to the signal.
There is a threshold for sensitivity control for each
person. The number of mastication can be obtained
by counting the number of times regression coeffi-
cients calculated using low pass filter output values
HEALTHINF 2012 - International Conference on Health Informatics
376
Table 1: Results of comparisons using 5 parameters evaluation.
Comparison Overall Lowest error rate Highest error rate Lowest error rate Highest error rate
situations No. error rate(%) for individual(%) for individual(%) for food type(%) for food type(%)
LE + LPF 35 7 57 16 (Salad) 61 (Fruit jelly)
ZCT + LPF 41 8 61 31 (Salad) 62 (Fruit jelly)
ADA + LPF 30 7 55 13 (Salad) 60 (Fruit jelly)
LE + LPF + rule 8 2 17 4 (Salad) 12 (Banana)
ZCT + LPF + rule 11 3 20 8 (Fruit jelly) 43 (Banana)
ADA + LPF + rule 7 3 14 2 (Salad) 11 (Marshmallow)
of 9 frames crosses zero with a negative value slope
and the local peak of low pass filter output is larger
than the sensitivity threshold.
6.2 Comparison Results
The overall accuracy of the mastication counting
models were compared (Table 2). The low pass fil-
ter based model with fixed cut-off frequency of 1.5
Hz was named as M1, the low pass filter based model
with adaptive cut-offfrequencywas named as M2, the
log energy based model was named as M3, and the
model proposed in section 5 was named as M.
Table 2: Comparison of overall accuracy for 4 models.
Methods M M1 M2 M3
Accuracy(%) 93 55 85 77
7 CONCLUSIONS
In this paper, we proposed a data analysis model for
realizing automatic mastication counting for any indi-
vidual and food type with high accuracy (93%). The
proposed model reduces the influence of individual
chewing ways differences (speed, strength, etc.) to a
large extent, and is little influenced by chewing sound
of different food types (soft, hard, crispy, etc.).
ACKNOWLEDGEMENTS
This research was supported by Japan Science and
Technology agency (JST)’s strategic sector for cre-
ation of advanced integrated sensing technologies for
realizing safe and secure societies: research project
on ”Development of a Physiological and Environmen-
tal Information Processing Platform and its Applica-
tion to the Metabolic Syndrome Measures”.
REFERENCES
Abraham, S., Collins, G., and Nordsieck, M. (1971). Re-
lationship of childhood weight status to morbidity in
adults. In HSMHA Health Reports. United States.
Health Services and Mental Health Administration.
Amft, O., Stger, M., and Trster, G. (2005). Analysis of
chewing sounds for dietary monitoring. In Proceed-
ings of 7th International Conference on Ubiquitous
Computing. Springer.
Hoffmans, A. F., Kromhout, D., and de Lezenne Coulande,
C. (1989). Body mass index at the age of 18 and its
effects on 32-year-mortality from coronary heart dis-
ease and cancer. a nnested case-control study among
the entire 1932 dutch male birth cohort. Journal of
Clinical Epidemiology Vol. 42, Issue 6, pages 513–
520.
Nicklas, T., Baranowski, T., Cullen, K., and Berenson, G.
(2001). Eating patterns, dietary quality and obesity.
Journal of the American College of Nutrition Vol. 20,
Issue 6, pages 599–608.
Nishimura, J. and Kuroda, T. (2008). Eating habits moni-
toring using wireless wearable in-ear microphone. In
Proceedings of International Symposium on Wireless
Pervasive Communications (ISWPC08). IEEE.
Shuzo, M., Komori, S., Takashima, T., Lopez, G., Tatsuta,
S., Yanagimoto, S., Warisawa, S., Delaunay, J.-J., and
Yamada, I. (2010). Wearable eating habit sensing sys-
tem using internal body sound. Journal of Advanced
Mechanical Design, Systems, and Manufacturing Vol.
4, Issue 1, pages 158–166.
Yasutomi, K. and Masuda, Y. (2008). Example of food
education for young japanese obese by wearing the
’kamikami sensor’ while eating. Journal of Japanese
Society for Masticatory Science and Health Promo-
tion Vol. 18, Issue 2, pages 149–150.
Notes
1
http://www.mando.se/en/start-page/start-page-/1.aspx
2
http://en.wikipedia.org/wiki/Zero crossing
3
http://www.mathworks.com/help/toolbox/signal/
findpeaks.html
MASTICATION COUNTING METHOD ROBUST TO FOOD TYPE AND INDIVIDUAL
377