HIERARCHICAL OBJECT CLASSIFICATION USING IMAGENET

DOMAIN ONTOLOGIES

Haider Ali

University of Leoben

Department of Mathematics and Information Technology

Franz-Josef-Strasse 18, A-8700 Leoben, Austria

Keywords:

Object classiﬁcation, Object hierarchies, ImageNet, Object detection and Domain Ontologies.

Abstract:

We present a binary tree based object classiﬁcation method in this paper. The binary tree builds a group of

classes using ImageNet domain ontologies. A binary decision function is introduced in the root node of the

decision tree using the positive samples of the ﬁrst group for training. The decision function continues dividing

the groups in sub-sequent groups when approaching the leaf nodes and provides positive and negative samples

for multi-class problems. We have tested our method on the PASCAL Visual Object Classes Challenge 2006

(VOC2006) dataset and have achieved comparable accuracy for group classiﬁcation. The results show that the

proposed method is a powerful class binarization technique for hierarchical objects group classiﬁcation.

1 INTRODUCTION

The recent approaches in object classiﬁcation sys-

tems shows that Support Vector Machines provides

better recognition rates as compare to the other exist-

ing approaches. We introduce an object classiﬁcation

method using hierarchical object class structure using

ImageNet domain ontologies. We have introduced a

conceptual ontology based phenomena to group ob-

ject class hierarchies. For example mammal was in-

troduced as a group having two member groups carni-

vore and ungulate hoofed mammal. These two groups

have some association to the mammal group based on

their characteristics. The root nodes of each subgroup

are deﬁned as classes have common relationship to

the subgroup. This is new way out for training group

level classiﬁer using the group labels instead of train-

ing directly root nodes. The prediction score gener-

ated by the group level classiﬁer is used to choose the

sub-group training samples. A base-level classiﬁer is

trained using the minimum positive samples identi-

ﬁed by the group/sub-group level classiﬁer prediction

scores. We have tested our approach on a publicly

available well known dataset provided for VOC chal-

lenge in 2006. The results we provide shows that the

group classiﬁer performs signiﬁcantly better than the

individual classiﬁers.

This paper is organized as following. The next

section describes the related work including the de-

scription and how we differ as compared to other ap-

proaches. Section 3 will provide our approach and

how we build a decision function providing the ex-

periments and our results. The last section provides

error analysis and continues with future work.

1.1 Related Work

Madzarov et al. (G. et al., 2009) has introduced a

novel architecture using binary decision tree for solv-

ing multiclass problem. They have introduced a clus-

tering based method to training binary decision func-

tions using SVM. They claims that their method per-

form faster than single class one-against-one and for

multi-class one-against-all available methods and pro-

vides better recognition rates. A popular approach is

introduced by Marcin et al. (Marszałek and Schmid,

2007) using lexical semantic networks for the object

recognition task. They introduce the concept of visual

appearance based learning by deﬁning the inter-class

relationship using semantic hierarchy of discrimina-

tive classiﬁers. A higher level layered object cate-

gorization architecture has been proposed by Lei et

al. (Cheng, 2009) for object categorization using hier-

archical category information. The object categories

are built with bottom-up and top-down approaches us-

ing cognitive rules using inter-category relationship at

higher level concepts.

Mailot et al. (Maillot et al., 2004) carried out a

534

Ali H. (2010).

HIERARCHICAL OBJECT CLASSIFICATION USING IMAGENET DOMAIN ONTOLOGIES.

In Proceedings of the International Conference on Computer Vision Theory and Applications, pages 534-536

DOI: 10.5220/0002851905340536

 SciTePress

similar work using machine learning and knowledge

representation techniques using visual concept on-

tology. They introduced visual concept ontological

concepts(spatial, color and texture concepts and re-

lations) as an intermediate relations using machine

learning and knowledge domain. We have used the

domain ontological concept deﬁned in the work of

Jia Deng et al. (Deng et al., 2009). They combined

the image datasets using WordNet hierarchy utilizing

subtrees and synsets of millions of images. We car-

ried our working using ImageNet domain ontologies

to generalize the task due to its strong logical group-

ing.

2 OBJECT CLASS HIERARCHIES

The hierarchical classiﬁcation structure contains the

group having all the sub-sequent nodes. The Root

node is trained with all training examples and there-

after test with the test data. Individual classiﬁers are

trained on training data selected by the group clas-

siﬁer. The process continues until we reach from a

group to class level classiﬁer training. The idea is to

use the reduced training examples based on the pre-

diction score of the group level classiﬁer. The global

workﬂow of the system is presented in Fig. 1.

Figure 1: Pipeline for hierarchical object classiﬁcation sys-

tem using ImageNet Domain ontologies.

We followed a rather simple but powerful ap-

proach to built the histograms using a scale-invariant

feature transform (SIFT) (Lowe, 2004). We built a

histogram for each image descriptor and concatenate

them for each image. We have chosen SIFT features

because of their invariance to image scale and rotation

as well as robust for illumination, noise, viewpoint

and partial occlusions as well as highly distinctive.

3 THE ALGORITHM

We have chosen N number of groups for a given set

of training examples. These training examples be-

longs to a ﬁxed number of classes C = c

, ..., c

. We

have trained a binary classiﬁer to assign these train-

ing examples to the corresponding correct groups and

classes. The basic algorithm is based on the following

steps:-

• Step 1: Training Examples X := (x

, . . . , x

)

where X ⊆ R

• Step 2: Classes C := (c

, . . . , c

) where C ⊆ R

and K ≤ N

• Step 3: For all given x

• Step 4: Group Labels N := (n

, . . . , n

)

where N ⊆ R

and T < K

• Step 5: Sub Group Labels S

:= (s

, . . . , s

)

where S

⊆ N

• Step 6: x

are assigned to its relevant c

• Step 7: Return x

where C

is the corresponding

class of x

4 EXPERIMENTS AND RESULTS

In this section we evaluate the hierarchical object

classiﬁcation method. This section covers the evalu-

ation scheme, SVM parameter selection and dataset

details. We provide an empirical evaluation of the

provided method.

4.1 Evaluation Scheme

We have trained the SVM on the ”train” VOC 2006

dataset. We train SVM classiﬁer for each group and

use to the prediction score to choose the positive sam-

ples for each sub-sequent group. We have chosen full

training data to train the root classiﬁer, although the

individual classiﬁers are trained on training data se-

lected by the group classiﬁer. We carried out our ex-

periments using SVMlight with the default parame-

ters, although the choice of the C-parameter selection

depends on the average precision on the ”val” dataset.

We did not choose the weight factor because we found

that the ”train” set was normalized and have equal

weight to the positive and negative samples.

4.2 Results

The results we provide show a consistent improve-

ment at each group level. Depending on the prediction

HIERARCHICAL OBJECT CLASSIFICATION USING IMAGENET DOMAIN ONTOLOGIES

535

score of each group the choice of positive samples for

each sub-sequent group is quite sensitive task. The

results are quite promising and provide sufﬁcient mo-

tivation to explore and optimize the sub-sequent tree

nodes by improving the decision function. The recog-

nition rate is provided in the Table 1.

Table 1: VOC2006 results - The group scores are obtained

on VAL dataset while the rest are on test dataset.

Groups Wheeled Vehicle Carnivore Hoofed Mammal

83.28 70.24 69.65

Classes Bicycle—53.17 Cow—15.97 Cat—34.61

Bus— 43.07 Horse—11.95 Dog—30.77

Car— 61.06 Sheep—26.36

Motorbike—25.16

5 CONCLUSIONS

We have analyzed and found that the proposed algo-

rithm provide a comparable accuracy for classiﬁca-

tion when used to classify a particular group. The ini-

tial idea worked quite well but thereafter need to be

reﬁned to identify the cause of failure when moving

to the sub-group and then to the base classiﬁer. Es-

pecially for those classes where we already have very

few training samples in the whole dataset; if not iden-

tiﬁed correctly at root-node fails completely when

reaching down to the base classiﬁer. The available ob-

ject classiﬁcation datasets for Pascal VOC challenges

contain very few positive examples for some classes

and are not balanced. Although training SVM or

boosting algorithm with very few training examples is

an active area of research for machine learning com-

munity (Hu et al., 2007), (Mutch and Lowe, 2008)

and (Janez Brank and Mladenic, 2003). A compara-

tive study could be carried out using existing learning

techniques like boosting to training the hierarchical

classiﬁcation tree and to compare them with SVM ap-

proach.

ACKNOWLEDGEMENTS

This work was supported in part by the Austrian Sci-

ence Fund FWF (S9104-N13 SP4). The research

leading to these results has also received funding from

the European Communitys Seventh Framework Pro-

gramme (FP7/2007- 2013) under grant agreements n

216886 (PASCAL2 Network of Excellence).

REFERENCES

Cheng, L. Y. J. Y. N. Z. H. (2009). Layered object catego-

rization. In ICPR 2008. 19th International Conference

on Pattern Recognition, pages 1–4.

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-

Fei, L. (2009). ImageNet: A Large-Scale Hierarchical

Image Database. In CVPR09.

G., M., D., G., and I., C. (2009). A multi-class svm classiﬁer

utilizing binary decision tree. Informatica, 33.

Hu, Q., Yu, D., and Xie, Z. (2007). Selecting samples and

features for svm based on neighborhood model. In

RSFDGrC ’07: Proceedings of the 11th International

Conference on Rough Sets, Fuzzy Sets, Data Min-

ing and Granular Computing, pages 508–517, Berlin,

Heidelberg. Springer-Verlag.

Janez Brank, Marko Grobelnik, N. M.-F. and Mladenic, D.

(2003). Training text classiﬁers with svm on very few

positive examples. Technical Report.

Lowe, D. G. (2004). Distinctive image features from scale-

invariant keypoints. International Journal of Com-

puter Vision, 60:91–110.

Maillot, N., Thonnat, M., and Hudelot, C. (2004). Ontology

based object learning and recognition: application to

image retrieval. In Tools with Artiﬁcial Intelligence,

2004. ICTAI 2004. 16th IEEE International Confer-

ence on, pages 620–625.

Marszałek, M. and Schmid, C. (2007). Semantic hierar-

chies for visual object recognition. In Conference on

Computer Vision & Pattern Recognition.

Mutch, J. and Lowe, D. (2008). Object class recognition

and localization using sparse features with limited re-

ceptive ﬁelds. International Journal of Computer Vi-

sion, 80(1):45–57.

VISAPP 2010 - International Conference on Computer Vision Theory and Applications

536