Incorporating Privileged Information to Improve Manifold Ordinal Regression

M. Pérez-Ortiz, P. A. Gutiérrez, C. Hervás-Martínez

2014

Abstract

Manifold learning covers those learning algorithms where high-dimensional data is assumed to lie on a lowdimensional manifold (usually nonlinear). Specific classification algorithms are able to preserve this manifold structure. On the other hand, ordinal regression covers those learning problems where the objective is to classify patterns into labels from a set of ordered categories. There have been very few works combining both ordinal regression and manifold learning. Additionally, privileged information refers to some special features which are available during classifier training, but not in the test phase. This paper contributes a new algorithm for combining ordinal regression and manifold learning, based on the idea of constructing a neighbourhood graph and obtaining the shortest path between all pairs of patterns. Moreover, we propose to exploit privileged information during graph construction, in order to obtain a better representation of the underlying manifold. The approach is tested with one synthetic experiment and 5 real ordinal datasets, showing a promising potential.

References

  1. Asuncion, A. and Newman, D. (2007). UCI machine learning repository.
  2. Baccianella, S., Esuli, A., and Sebastiani, F. (2009). Evaluation measures for ordinal regression. In Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA 09), pages 283-287, Pisa, Italy.
  3. Belkin, M. and Niyogi, P. (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. In NIPS, volume 14, pages 585-591.
  4. Cardoso, J. S. and da Costa, J. F. P. (2007). Learning to classify ordinal data: The data replication method. Journal of Machine Learning Research, 8:1393-1429.
  5. Cheng, J., Wang, Z., and Pollastri, G. (2008). A neural network approach to ordinal regression. In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN2008, IEEE World Congress on Computational Intelligence), pages 1279-1284. IEEE Press.
  6. Chu, W. and Ghahramani, Z. (2005). Gaussian processes for ordinal regression. Journal of Machine Learning Research, 6:1019-1041.
  7. Chu, W. and Keerthi, S. S. (2007). Support vector ordinal regression. Neural Computation, 19(3):792-815.
  8. Cruz-Ramírez, M., Hervás-Martínez, C., SánchezMonedero, J., and Gutiérrez, P. A. (2014). Metrics to guide a multi-objective evolutionary algorithm for ordinal classification. Neurocomputing, 135:21-31.
  9. Deng, W.-Y., Zheng, Q.-H., Lian, S., Chen, L., and Wang, X. (2010). Ordinal extreme learning machine. Neurocomputation, 74(1-3):447-456.
  10. Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik, 1(1):269- 271.
  11. Frank, E. and Hall, M. (2001). A simple approach to ordinal classification. In Proc. of the 12th Eur. Conf. on Machine Learning, pages 145-156.
  12. Gutiérrez, P. A., Pérez-Ortiz, M., Fernandez-Navarro, F., Sánchez-Monedero, J., and Hervás-Martínez, C. (2012). An Experimental Study of Different Ordinal Regression Methods and Measures. In 7th International Conference on Hybrid Artificial Intelligence Systems (HAIS), volume 7209 of Lecture Notes in Computer Science, pages 296-307.
  13. He, X. and Niyogi, P. (2003). Locality preserving projections. In NIPS, volume 16, pages 234-241.
  14. Kira, K. and Rendell, L. A. (1992). The feature selection problem: Traditional methods and a new algorithm. In AAAI, pages 129-134.
  15. Li, L. and Lin, H.-T. (2007). Ordinal Regression by Extended Binary Classification. In Advances in Neural Inform. Processing Syst. 19.
  16. Lin, H.-T. and Li, L. (2012). Reduction from cost-sensitive ordinal ranking to weighted binary classification. Neural Computation, 24(5):1329-1367.
  17. Liu, Y., Liu, Y., and Chan, K. C. C. (2011a). Ordinal regression via manifold learning. In Burgard, W. and Roth, D., editors, Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI'11), pages 398-403. AAAI Press.
  18. Liu, Y., Liu, Y., Chan, K. C. C., and Zhang, J. (2012). Neighborhood preserving ordinal regression. In Proceedings of the 4th International Conference on Internet Multimedia Computing and Service (ICIMCS12), pages 119-122, New York, NY, USA. ACM.
  19. Liu, Y., Liu, Y., Zhong, S., and Chan, K. C. (2011b). Semi-supervised manifold ordinal regression for image ranking. In Proceedings of the 19th ACM international conference on Multimedia (ACM MM2011), pages 1393-1396, New York, NY, USA. ACM.
  20. McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society, 42(2):109- 142.
  21. PASCAL (2011). Pascal (pattern analysis, statistical modelling and computational learning) machine learning benchmarks repository.
  22. R. Herbrich, T. G. and Obermayer, K. (2000). Large margin rank boundaries for ordinal regression. In Smola, A., Bartlett, P., Schölkopf, B., and Schuurmans, D., editors, Advances in Large Margin Classifiers, pages 115-132. MIT Press.
  23. Shashua, A. and Levin, A. (2003). Ranking with large margin principle: Two approaches. In Advances in Neural Information Processing Systems (NIPS), pages 937- 944. MIT Press, Cambridge.
  24. Sun, B.-Y., Li, J., Wu, D. D., Zhang, X.-M., and Li, W.-B. (2010). Kernel discriminant learning for ordinal regression. IEEE Transactions on Knowledge and Data Engineering, 22:906-910.
  25. Tenenbaum, J. B., De Silva, V., and Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323.
  26. Vapnik, V. and Vashist, A. (2009).
  27. A new learning paradigm: Learning using privileged information. Neural Networks, 22(5-6):544-557.
  28. Verwaeren, J., Waegeman, W., and De Baets, B. (2012). Learning partial ordinal class memberships with kernel-based proportional odds models. Computational Statistics & Data Analysis, 56(4):928-942.
  29. Wang, H., Huang, H., and Ding, C. H. (2010). Discriminant laplacian embedding. In AAAI.
  30. Zhou, D., Weston, J., Gretton, A., Bousquet, O., and Schölkopf, B. (2004). Ranking on data manifolds. In Proceedings of the Seventeenth Annual Conference on Neural Information Processing Systems (NIPS2003), pages 169-176.
Download


Paper Citation


in Harvard Style

Pérez-Ortiz M., A. Gutiérrez P. and Hervás-Martínez C. (2014). Incorporating Privileged Information to Improve Manifold Ordinal Regression . In Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014) ISBN 978-989-758-054-3, pages 187-194. DOI: 10.5220/0005075801870194


in Bibtex Style

@conference{ncta14,
author={M. Pérez-Ortiz and P. A. Gutiérrez and C. Hervás-Martínez},
title={Incorporating Privileged Information to Improve Manifold Ordinal Regression},
booktitle={Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)},
year={2014},
pages={187-194},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005075801870194},
isbn={978-989-758-054-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)
TI - Incorporating Privileged Information to Improve Manifold Ordinal Regression
SN - 978-989-758-054-3
AU - Pérez-Ortiz M.
AU - A. Gutiérrez P.
AU - Hervás-Martínez C.
PY - 2014
SP - 187
EP - 194
DO - 10.5220/0005075801870194