GPU Solver with Chi-square Kernels for SVM Classification of Big Sparse Problems

Krzysztof Sopyla, Pawel Drozda

2014

Abstract

This paper presents the ongoing research on the GPU SVM solutions for classification of big sparse datasets. In particular, after the success of implementation of RBF kernel for sparse matrix formats in previous work we decided to evaluate Chi2 and Exponential Chi2 kernels. Moreover, the details of GPU solver are pointed. Experimental session summarizes results of GPU SVM classification for different sparse data formats and different SVM kernels and demonstrates that solution for Exponential Chi2 achieves significant accelerations in GPU SVM processing, while the results for Chi2 kernel are very far from satisfactory.

References

  1. Acir, N. and Guzelis, C. (2004). An application of support vector machine in bioinformatics: automated recognition of epileptiform patterns in eeg using svm classifier designed by a perturbation method. In Proceedings of the Third international conference on Advances in Information Systems, ADVIS'04, pages 462-471, Berlin, Heidelberg. Springer-Verlag.
  2. Bell, N. and Garl, M. (2008). Efficient sparse matrix-vector multiplication on cuda. Technical report, NVidia.
  3. Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory, COLT 7892, pages 144-152, New York, NY, USA. ACM.
  4. Cao, L. J., Keerthi, S. S., Ong, C. J., Zhang, J. Q., Periyathamby, U., Fu, X. J., and Lee, H. P. (2006). Parallel sequential minimal optimization for the training of support vector machines. IEEE Transactions on Neural Networks, 17(4):1039-1049.
  5. Carpenter, A. (2009). Cusvm: A cuda implementation of support vector classification and regression. Technical report.
  6. Catanzaro, B., Sundaram, N., and Keutzer, K. (2008). Fast support vector machine training and classification on graphics processors. In Proceedings of the 25th international conference on Machine learning, ICML 7808, pages 104-111, New York, NY, USA. ACM.
  7. Chang, C.-C. and Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27.
  8. Chang, E. Y., Zhu, K., Wang, H., Bai, H., Li, J., Qiu, Z., and Cui, H. (2007). Psvm: Parallelizing support vector machines on distributed computers. In NIPS.
  9. Chapelle, O., Haffner, P., and Vapnik, V. N. (1999). Support vector machines for histogram-based image classification. Neural Networks, IEEE Transactions on, pages 1055-1064.
  10. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Mach. Learn., 20(3):273-297.
  11. Cotter, A., Srebro, N., and Keshet, J. (2011). A gpu-tailored approach for training kernelized svms. In Proceedings of the 17th ACM SIGKDD conference, KDD 7811, pages 805-813.
  12. Fan, R.-E., Chen, P.-H., and Lin, C.-J. (2005). Working set selection using the second order information for training svm. JOURNAL OF MACHINE LEARNING RESEARCH, 6:1889-1918.
  13. Gorecki, P., Artiemjew, P., Drozda, P., and Sopyla, K. (2012). Categorization of similar objects using bag of visual words and support vector machines. In Filipe, J. and Fred, A. L. N., editors, ICAART (1), pages 231-236. SciTePress.
  14. Graf, H. P., Cosatto, E., Bottou, L., Durdanovic, I., and Vapnik, V. (2005). Parallel support vector machines: The cascade svm. In In Advances in Neural Information Processing Systems, pages 521-528. MIT Press.
  15. Harris, M. (2008). Optimizing Parallel Reduction in CUDA. Technical report, nVidia.
  16. Herrero-Lopez, S., Williams, J. R., and Sanchez, A. (2010). Parallel multiclass classification using svms on gpus. In Proceedings of the 3rd Workshop on GeneralPurpose Computation on Graphics Processing Units, GPGPU 7810, pages 2-11, New York, NY, USA. ACM.
  17. Joachims, T. (1998). Text categorization with support vector machines: learning with many relevant features. In Nédellec, C. and Rouveirol, C., editors, Proceedings of ECML-98, 10th European Conference on Machine Learning, number 1398, pages 137-142. Springer Verlag, Heidelberg, DE.
  18. Joachims, T. (1999). Advances in kernel methods. chapter Making large-scale support vector machine learning practical, pages 169-184. MIT Press, Cambridge, MA, USA.
  19. Joachims, T., Finley, T., and Yu, C. J. (2009). Cutting-plane training of structural svms. Mach. Learn., 77(1):27- 59.
  20. Keerthi, S., Shevade, S., Bhattacharyya, C., and Murthy, K. (2001). Improvements to platt's smo algorithm for svm classifier design. Neural Computation, 13(3):637-649.
  21. Lazebnik, S., Schmid, C., and Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2, CVPR 7806, pages 2169-2178, Washington, DC, USA. IEEE Computer Society.
  22. Lin, T.-K. and Chien, S.-Y. (2010). Support vector machines on gpu with sparse matrix format. Machine Learning and Applications, Fourth International Conference on, 0:313-318.
  23. Platt, J. (1998). Fast training of support vector machines using sequential minimal optimization. In Advances in Kernel Methods - Support Vector Learning. MIT Press.
  24. Sopyla, K., Drozda, P., and Gorecki, P. (2012). Svm with cuda accelerated kernels for big sparse problems. In Proceedings of the ICAISC, volume 7267 of Lecture Notes in Computer Science, pages 439-447. Springer.
  25. Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. Springer-Verlag New York, Inc., New York, NY, USA.
  26. Vázquez, F., Garzón, E. M., Martinez, J. A., and Fernández, J. J. (2009). The sparse matrix vector product on gpus. Technical report, University of Almeria.
  27. Volkov, V. and Demmel, J. W. (2008). Benchmarking gpus to tune dense linear algebra. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing, SC 7808, pages 31:1-31:11, Piscataway, NJ, USA. IEEE Press.
  28. Zanni, L., Serafini, T., and Zanghirati, G. (2006). Parallel software for training large scale support vector machines on multiprocessor systems. J. Mach. Learn. Res., 7:1467-1492.
  29. Zhao, H. X. and Magoules, F. (2011). Parallel support vector machines on multi-core and multiprocessor systems. In Proceedings of the 11th International Conference on Artificial Intelligence and Applications (AIA 2011).
Download


Paper Citation


in Harvard Style

Sopyla K. and Drozda P. (2014). GPU Solver with Chi-square Kernels for SVM Classification of Big Sparse Problems . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 331-336. DOI: 10.5220/0004922603310336


in Bibtex Style

@conference{icpram14,
author={Krzysztof Sopyla and Pawel Drozda},
title={GPU Solver with Chi-square Kernels for SVM Classification of Big Sparse Problems},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={331-336},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004922603310336},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - GPU Solver with Chi-square Kernels for SVM Classification of Big Sparse Problems
SN - 978-989-758-018-5
AU - Sopyla K.
AU - Drozda P.
PY - 2014
SP - 331
EP - 336
DO - 10.5220/0004922603310336