4.5 CBF: Similarity Measure
Aggregation
Some SMs are reliable to predict class, depending on
the selected n-gram and s-split. 175 metrics N-SM/S
are computed according to the possible combinations
(SM,N,S). The proposed voting scheme aims to catch
the strength and the complementarity of each SM and
to propose a nice combination. This can be done ei-
ther by vectors manipulating (means or product) or
voting scheme.
A step-forward process on the training set leads
to a single measure, 2-Jaccard/3, with an accuracy of
100% on the training set, but 98.7% (F1-measure =
0.987) on the testing set.
To enforce complementarity, we decided to keep
all the metrics with an accuracy in the train part that
have less than 10% of relative difference with the best
one, and using them in a weighted voting scheme, de-
fined as follow: Accuracy(m)
2
/
p
Occurence(m).
Accuracy(m): is the accuracy of the metric m on
the training set (validation step). This allows us to
increase the gap between high and poor metrics.
Occurence(m): this is the occurrence of the SM in
the kept metrics from the training set. Thus, it lowers
the influence of too frequent SMs and allows more
complementarity: each SM brings a different kind of
information.
An hard vote obtained from the 175 metrics
(weigths=1) permits 93.3% of good recognition on
the testing set based on the training profiles. With this
weighted vote combining the 175 metrics, test accu-
racy increases up to 98.8%. With the selected metrics
based on the 10%-relative difference,99.5% of test ac-
curacy is reached (F1-measure = 0.995), close to the
state-of-art result: 99.8% (Bagnall et al., 2017). Also,
weighted similarity vote we proposed obtains better
accuracy than the 7 of 9 algorithms cited in (Bagnall
et al., 2017).
As said previously, n-gram are highly sensitive to
noise, by dismissing n-gram from this voting scheme,
the prediction reaches 99.8% of accuracy and a F1-
measure of 99.8%, as well as Bag of SFA Symbols or
COTE for CBF in (Bagnall et al., 2017).
4.6 ORIENTOI’s Results
For the ORIENTOI’s dataset (ORIENTOI in the se-
quel), and only using the best metric (1-Jaccard/1),
an accuracy of 85.3% (F1-measure = 0.844) is ob-
tained and once again, the ”step forward” stops into
the first step. Our vote method allows to reach an ac-
curacy of 97.2% (F1-measure = 0.971). N-gram pro-
cess seems to be useful to classify for this dataset:
only 95.9% accuracy without n-gram (F1-measure =
0.961). This could be explained by the fact that in
an application, multiple choices exists and the n-gram
help the method to take into consideration the impor-
tance of transition between pages. The s-split is less
important in ORIENTOI. Due to the high redundancy
of the core loop (cycle of main interest actions), the
s-split is less effective.
4.7 UCR Results
In (Bagnall et al., 2017), 9 algorithms are tested on
85 UCR datasets. We used the same benchmark to
validate our approach on 28 of them.
Table 6 details accuracy results for the 6 types of
data in UCR and recall the state-of-art best scores (p0
from Table 6). The relevance of the N-gram process
(p1, p1a) depends on the used dataset. Mean accuracy
with the previous SM vote protocol is 66.3% with-
out n-gram and s-split process. Adding n-gram up-
grades this accuracy to 70.9%, and using just s-split
upgrade it to 78.1% (p2, p2a). Both processes give a
close score: 78.2%. This shows the importance of s-
split, but does not mean that metrics with n-gram isn’t
reliable. Furthermore, metrics without n-gram reach
state-of-art for 3 of the 28 used datasets.
Our weighted vote scheme has better results than
an hard vote scheme (mean accucary of 60.8%) on
these 28 datasets without n-gram and s-split. And
for full metrics (N-SM/S), weighted vote scheme was
78.2%, and 77.7% for the hard vote scheme, pointing
the usefulness of lowering too redundant SM on kept
metrics.
Our method seems to be efficient in some kind of
data, such as motion capture, ECG or simulated and
less in others, such as spectrographs, as you can see
in the table 6.
4.8 UCR Results: Adapted
Quantization Step
A fixed quantization (0.5-step) could not be relevant
for each dataset. So adapted step was explored for
each dataset of UCR among these values: 5, 10, 15,
20 and 30 (p1a, p2a). Results with this adapted quan-
tization are noted p1a and p2a in table 6. Few datasets
have lower result with adapted step, and some dataset
have notable better results, such as SonyAIBORobot-
Surface1, Wine, BirdChicken and SyntheticControl.
Note that we only test 5 adapted steps in this case,
and an optimal quantization could be learnt from the
training set. Symbolic Aggregate approximation on
CBF has also been compared without better success,
but could be investigated for the other datasets.
Symbolic Translation of Time Series using Piecewise N-gram Similarity Voting
331