Learning with Kernel Random Field and Linear SVM

Haruhisa Takahashi

2014

Abstract

Deep learning methods, which include feature extraction in the training process, are achieving success in pattern recognition and machine learning fields but require huge parameter setting, and need the selection from various methods. On the contrary, Support Vector Machines (SVMs) have been popularly used in these fields in light of the simple algorithm and solid reasons based on the learning theory. However, it is difficult to improve recognition performance in SVMs beyond a certain level of capacity, in that higher dimensional feature space can only assure the linear separability of data as opposed to separation of the data manifolds themselves. We propose a new framework of kernel machine that generates essentially linearly separable kernel features. Our method utilizes pretraining process based on a kernel generative model and the mean field Fisher score with a higher-order autocorrelation kernel. Thus derived features are to be separated by a liner SVM, which exhibits far better generalization performance than any kernel-based SVMs. We show the experiments on the face detection using the appearance based approach, and that our method can attain comparable results with the state-of-the-art face detection methods based on AdaBoost, SURF, and cascade despite of smaller data size and no preprocessing.

References

  1. Alvira M. and Rifkin R. 2001, 'An Empirical Comparison of SNoW and SVMs for Face Detection', in MIT CBCL Memos (1993 - 2004).
  2. Bengio Y., Courville A. and Vincent P. 2012, 'Representation Learning: A Review and New Perspectives' , arXiv:1206.5538 [cs.LG], Cornell University Library [Oct 2012].
  3. Erhan D., Bengio Y., Courville A., Manzagol P.A., Vincent P. and Bengio S., 2010, 'Why Does Unsupervised Pretraining Help Deep Learning?, Journal of Machine Learning Research 11, 625-660 .
  4. Mangasarian O.L. and Musicant D.R. 2001, 'Lagrangian Support Vector Machines', Journal of Machine Learning Research 1, 161-177.
  5. Horikawa Yo. 2004, 'Comparison of Support Vector Machines with Autocorrelation Kernels for Invariant Texture Classification', Proceedings of the 17th Omt. Conf. on Pat. recog. (ICPR'04), 4647-4651.
  6. Jaakkola ,T and Haussler, D. 1998, 'Exploiting Generative Models in Discriminative Classifiers' In Advances in Neural Information Processing Systems 11, pp 487- 493. MIT Press.
  7. Lafferty J. Zhu X., and Liu Y. 2004,'Kernel conditional random fields: representation and clique selection', Proc. of the twenty-first int. conf. on Machine learning, Canada, Page: 64 .
  8. Lafferty J., McCallum A. and Pereira F. 2004, 'Exponential Families for Conditional Random Fields', Conditional Random Fields: ACM Int. Conf. Proc. Series; Vol. 70, . 20th Conf.on Uncertainty in artificial intelligence, Banff, Canada, 2 - 9.
  9. Li, J. , Zhang Y. 2013, 'Learning SURF Cascade for Fast and Accurate Object Detection', CVPR, 3468-3475.
  10. Pham, M. T., Gao, Y. , Hoang,V. D. D., and T. J.Cham, T. J., 2010, 'Fast Polygonal Integration and Its Application in Extending Haar-like Features to Improve Object Detection', Proc. IEEE conf. on Comp. and Pat. Recog. (CVPR), San Francisco.
  11. Roscher, R., 'Kernel Discriminative Random Fields for land cover classification', Pattern Recognition in Remote Sensing (PRRS), 2010 IAPR Workshop on Date of Conference, 22-22 Aug.
  12. Viola P. A. and Jones M. J. 2004, 'Robust real-time face detection', IJCV, 57(2), 137-154.
Download


Paper Citation


in Harvard Style

Takahashi H. (2014). Learning with Kernel Random Field and Linear SVM . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 167-174. DOI: 10.5220/0004792601670174


in Bibtex Style

@conference{icpram14,
author={Haruhisa Takahashi},
title={Learning with Kernel Random Field and Linear SVM},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={167-174},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004792601670174},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Learning with Kernel Random Field and Linear SVM
SN - 978-989-758-018-5
AU - Takahashi H.
PY - 2014
SP - 167
EP - 174
DO - 10.5220/0004792601670174