![](bg6.png)
outperform a single SVM with optimal hyperparame-
ters.
Although 200 or 2000 SVMs must be combined
in an ensemble random-subset SVM, the number of
computations for the subset-kernels would not exceed
that for a single (full-set) SVM because an SVM re-
quires at least O(N
2
) to O(N
3
) computations.
We employed a linear SVM to combine the ker-
nels and obtained the optimal kernel weights. How-
ever, this final SVM took up the majority of the com-
putation time of the ensemble random-subset SVM
because it had to be trained for as many samples as
the large-attribute training samples.
In this study, we used all the outputs from subset
kernels for the training samples; however, we can ap-
ply feature selection and sample selection for the final
linear SVM, as this may help reduce the computation
time and improve the generalization performance si-
multaneously.
REFERENCES
Baudat, G. “Feature Vector Selection and Projection Using
Kernels”, in NeuroComputing, Vol.55, No.1, pp.21-
38, 2003.
Breiman, L. “Bagging Predictors”, Machine Learning,
Vol.24, pp.123-140, 1996.
Chang, C. C. and Lin, C. J. “LIBSVM: a
library for support vector machines”,
http://www.csie.ntu.edu.tw/˜cjlin/libsvm, 2001.
Chapelle, O. “Training a Support Vector Machine in the Pri-
mal”, in Large-Scale Kernel Machines, pp.29-50, The
MIT Press, 2007.
Cristianini, N. and Taylor, J. S. An Introduction to Sup-
port Vector Machines and other kernel-based learning
methods, Cambridge University Press, 2000.
Demir, B. and Erturk, S. “Hyperspectral Image Classifica-
tion Using Relevance Vector Machines”, IEEE Geo-
science and Remote Sensing Letters, Vol.4, No.4,
pp.586-590, 2007.
Fischler, M. A. and Bolles, R. C. “Random Sample Con-
sensus: A Paradigm for Model Fitting with Appli-
cations to Image Analysis and Automated Cartogra-
phy”, Communications of the ACM, Vol.24, pp.381-
395, 1981.
Freund, Y. and Schapire, R. E. “Experiments with a New
Boosting Algorithm”,in Proc. of International Conf.
on Machine Learning (ICML96), pp.148-156, 1996.
Keerthi, S. S. and Chapelle, O. and DeCoste, D. “Build-
ing SVMs with Reduced Classifier Complexity”, in
Large-Scale Kernel Machines, pp.251-274, The MIT
Press, 2007.
Lin, K.-M. and Lin, C.-J. “A Study on Reduced Support
Vector Machines”, IEEE Transactions on Neural Net-
works, Vol.14, pp.1449-1459, 2003.
Nishida, K. and Kurita, T. “RANSAC-SVM for Large-Scale
Datasets”, in proc. International COnference on Pat-
tern Recognition (ICPR2008), 2008. (CD-ROM).
Oosugi, Y. and Uehara, K. “Constructing a Minimal
Instance-base by Storing Prototype Instances”, in J.
of Information Processing, Vol.39, No.11, pp.2949-
2959, 1998. (in Japanese).
Sch¨olkopf, B. and Burges, C. J. C. and Smola, A. J. Ad-
vances in Kernel Methods - Support Vector Learning,
The MIT Press, 1999.
Vapnik, V. N. Statistical Learning Theory, John Wiley &
Sons, 1998.
Zhu, J. and Hastie, T. “Kernel Logistic Regression and
the Import Vector Machine”, J.of Computational and
Graphical Statistics, Vol.14, No.1, pp.185-205, 2005.
LIBSVM data set, http://www.csie.ntu.edu.tw/˜cjlin/libsvm
tools/datasets/binary.html#cod-rna, 2006.
ENSEMBLE RANDOM-SUBSET SVM
339