![](bg1.png)
DECISION TREE INDUCTION FROM COUNTEREXAMPLES
Nicolas Cebron, Fabian Richter and Rainer Lienhart
Multimedia Computing Lab, University of Augsburg, Universitaetsstr. 6a, Augsburg, Germany
Keywords:
Decision trees, Counterexamples, Machine learning, Data mining, Decision making.
Abstract:
While it is well accepted in human learning to learn from counterexamples or mistakes, classic machine
learning algorithms still focus only on correctly labeled training examples.We replace this rigid paradigm by
using complementary probabilities to describe the probability that a certain class does not occur. Based on
the complementary probabilities, we design a decision tree algorithm that learns from counterexamples. In
a classification problem with K classes, K − 1 counterexamples correspond to one correctly labeled training
example. We demonstrate that even when only a partial amount of counterexamples is available, we can still
obtain good performance.
1 INTRODUCTION
The goal of supervised classification is to deduce a
function from examples in a dataset that maps input
objects to desired outputs. By using a set of labeled
training examples, we can train a classifier that can be
used to predict the nominal target variable for unseen
test data. To achieve this, the learner has to generalize
from the presented data to unseen situations. While
a plethora of algorithms for supervised classification
has been developed, only a few works deviate from
this classical setting.
In this paper, we focus our attention on decision
trees. Especially in multi-class problems, they are a
reliable and effective technique. They usually per-
form well and offer a simple representation in form
of a tree or a set of rules that can be deduced from
it. They have been used a lot in situations where a
decision must be made effectively and reliably, e.g.
in medical decision making (Podgorelec et al., 2002).
However, like all inductive methods in machine learn-
ing, the performance of this classifier is based on cor-
rectly labeled training examples. Finding the correct
class label for an example when generating a train-
ing set for the classifier can be difficult – especially
when there is a large number of possible classes. In
the work of (Joshi et al., 2010), it has been shown that
the human error rate and the time needed to find the
correct label grows with the number of classes; at the
same time the user distress increases. In some situ-
ations, it might not even be possible for the human
expert to determine the correct class label out of ma-
ny possible class labels. In a normal classification set-
ting, we would have to ignore this example.
As an example, we stick to the domain of medical
decision making, where we have two common situa-
tions in which the human expert has problems provid-
ing the correct class label:
1. Ambiguous Information: different class labels
(e.g. diseases) may be possible, but there is a lack
of information to explicitly choose one of them.
For example, it is unclear whether a person with
headache symptoms is suffering from a cold or
has the flu (or another type of disease).
2. Rare Cases: the determination of the class label
may be difficult because of missing expertise in a
special field. For example, it may be difficult to
classify rare (so-called orphan) diseases.
In this work, we want to introduce a new paradigm
in supervised classification: we do not obtain the la-
bel information itself, but the labels of the classes that
this example does not belong to. We call these exam-
ples counterexamples. For the preceding examples in
medical decision making, it can be very easy to spec-
ify the diseases that are not likely (e.g. not typhlitis,
not heartburn, etc. for a headache symptom) in order
to narrow down the set of possible classes. We argue
that in many real world settings, it is much easier for
the human expert to provide a counterexample instead
of determining the correct class label. This does not
only apply to the domain of medical decision mak-
ing, it is also true for many other domains like image,
music or text classification.
525
Cebron N., Richter F. and Lienhart R. (2012).
DECISION TREE INDUCTION FROM COUNTEREXAMPLES.
In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods, pages 525-528
DOI: 10.5220/0003730405250528
Copyright
c
SciTePress