Margin-based Refinement for Support-Vector-Machine Classification

Helene Dörksen, Volker Lohweg

2017

Abstract

In real-world scenarios it is not always possible to generate an appropriate number of measured objects for machine learning tasks. At the learning stage, for small/incomplete datasets it is nonetheless often possible to get high accuracies for several arbitrarily chosen classifiers. The fact is that many classifiers might perform accurately, but decision boundaries might be inadequate. In this situation, the decision supported by marginlike characteristics for the discrimination of classes might be taken into account. Accuracy as an exclusive measure is often not sufficient. To contribute to the solution of this problem, we present a margin-based approach originated from an existing refinement procedure. In our method, margin value is considered as optimisation criterion for the refinement of SVM models. The performance of the approach is evaluated on a real-world application dataset for Motor Drive Diagnosis coming from the field of intelligent autonomous systems in the context of Industry 4.0 paradigm as well as on several UCI Repository samples with different numbers of features and objects.

References

  1. Ahmad, K., Meier, U., and Kwasnicka, H. (2010). Fuzzy logic based signal classification with cognitive radios for standard wireless technologies. In Cognitive Radio Oriented Wireless Networks Communications (CROWNCOM), 2010 Proceedings of the Fifth International Conference on, pages 1-5.
  2. Alpaydin, E. (2010). Introduction to Machine Learning. The MIT Press, Cambridge, 2 edition.
  3. Bator, M., Dicks, A., M önks, U., and Lohweg, V. (2012). Feature Extraction and Reduction Applied to Sensorless Drive Diagnosis. In Hoffmann, F. and Hüllermeier, E., editors, 22. Workshop Computational Intelligence, volume 45 of Schriftenreihe des Instituts für Angewandte Informatik - Automatisierungstechnik am Karlsruher Institut für Technologie, pages 163- 178. KIT Scientific Publishing.
  4. Bayer, C., Bator, M., Mönks, U., Dicks, A., EngeRosenblatt, O., and Lohweg, V. (2013). Sensorless Drive Diagnosis Using Automated Feature Extraction, Significance Ranking and Reduction. In Seatzu, C. and Zurawski, R., editors, ETFA 2013, pages 1-4.
  5. Benner, P., Mehrmann, V., and Sorensen, D. C. D. C., editors (2005). Dimension reduction of large-scale systems : proceedings of a workshop held in Oberwolfach, Germany, October 19-25, 2003, Lecture notes in computational science and engineering. Springer.
  6. Blum, A. L. and Langley, P. (1997). Selection of relevant features and examples in machine learning. Artificial Intelligence, 97:245-271.
  7. Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory, COLT 7892, pages 144-152, New York, NY, USA. ACM.
  8. Chapelle, O., Vapnik, V., and Bengio, Y. (2002a). Model selection for small sample regression. Machine Learning, 48(1-3):9-23.
  9. Chapelle, O., Vapnik, V., Bousquet, O., and Mukherjee, S. (2002b). Choosing multiple parameters for support vector machines. Mach. Learn., 46(1-3).
  10. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3):273-297.
  11. Do, H., Kalousis, A., Woznica, A., and Hilario, M. (2009). Margin and radius based multiple kernel learning. In Buntine, W., Grobelnik, M., Mladenic, D., and Shawe-Taylor, J., editors, Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2009, Bled, Slovenia, September 7-11, Proceedings, Part I. Springer, Berlin, Heidelberg.
  12. Dörksen, H. and Lohweg, V. (2014). Combinatorial refinement of feature weighting for linear classification. In Emerging Technology and Factory Automation (ETFA), 2014 IEEE, pages 1-7.
  13. Dörksen, H., Mönks, U., and Lohweg, V. (2014). Fast classification in industrial Big Data environments. In Emerging Technology and Factory Automation (ETFA), 2014 IEEE, pages 1-7.
  14. Freund, Y. and Schapire, R. E. (1996). Experiments with a New Boosting Algorithm. Thirteenth International Conference on Machine Learning.
  15. Guyon, I., Gunn, S., Nikravesh, M., and Zadeh, L. A. (2006). Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing). Springer-Verlag New York, Inc, Secaucus and NJ and USA.
  16. Hagan, M. T., Demuth, H. B., and Beale, M. (1996). Neural Network Design. PWS Publishing Co., Boston, MA, USA.
  17. Kuncheva, L. I. (2004). Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience.
  18. Lee, J. A. and Verleysen, M. (2007). Nonlinear Dimensionality Reduction. Springer Publishing Company, Incorporated, 1st edition.
  19. Ma, Y. and Guo, G. (2014). Support vector machines applications. Springer.
  20. Niggemann, O. and Lohweg, V. (2015). On the diagnosis of cyber-physical production systems. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA., pages 4119-4126.
  21. Pei, J., Tseng, V. S., Cao, L., Motoda, H., and Xu, G., editors (2013). Advances in Knowledge Discovery and Data Mining, 17th Pacific-Asia Conference, PAKDD 2013, Gold Coast, Australia, April 14-17, 2013, Proceedings, Part I-III, Lecture Notes in Computer Science. Springer.
  22. Schölkopf, B. and Smola, A. J. (2002). Learning with kernels: Support vector machines, regularization, optimization, and beyond. Adaptive computation and machine learning. MIT Press, Cambridge and Mass.
  23. Sra, S., Nowozin, S., and Wright, S. J. (2012). Optimization for Machine Learning. Neural information processing series. MIT Press.
  24. Vapnik, V. N. (1995). Statistical Learning Theory. Wiley, London, 1 edition.
  25. Weinberger, K. Q. and Saul, L. K. (2009). Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res., 10:207-244.
  26. Yi, W.-J., Sarkar, O., Mathavan, S., and Saniie, J. (2014). Wearable sensor data fusion for remote health assessment and fall detection. In Electro/Information Technology (EIT), 2014 IEEE International Conference on, pages 303-307.
  27. Zhou, Z. H., Roli, F., and Kittler, J. (2013). Multiple Classifier Systems: 11th International Workshop, MCS 2013, Nanjing, China, May 15-17, 2013. Proceedings. Lecture Notes in Computer Science, vol. 7872. Springer London, Limited.
Download


Paper Citation


in Harvard Style

Dörksen H. and Lohweg V. (2017). Margin-based Refinement for Support-Vector-Machine Classification . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 293-300. DOI: 10.5220/0006115502930300


in Bibtex Style

@conference{icpram17,
author={Helene Dörksen and Volker Lohweg},
title={Margin-based Refinement for Support-Vector-Machine Classification},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2017},
pages={293-300},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006115502930300},
isbn={978-989-758-222-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Margin-based Refinement for Support-Vector-Machine Classification
SN - 978-989-758-222-6
AU - Dörksen H.
AU - Lohweg V.
PY - 2017
SP - 293
EP - 300
DO - 10.5220/0006115502930300