Metric Learning in Dimensionality Reduction

Alexander Schulz, Barbara Hammer

2015

Abstract

The emerging big dimensionality in digital domains causes the need of powerful non-linear dimensionality reduction techniques for a rapid and intuitive visual data access. While a couple of powerful non-linear dimensionality reduction tools have been proposed in the last years, their applicability is limited in practice: since a non-linear projection is no longer characterised by semantically meaningful data dimensions, the visual display provides only very limited interpretability which goes beyond mere neighbourhood relationships and, hence, interactive data analysis and further expert insight are hindered. In this contribution, we propose to enhance non-linear dimensionality reduction techniques by a metric learning framework. This allows us to quantify the relevance of single data dimensions and their correlation with respect to the given visual display; on the one side, this explains its most relevant factors; on the other side, it opens the way towards an interactive data analysis by changing the data representation based on the learned metric from the visual display.

References

  1. Bellet, A., Habrard, A., and Sebban, M. (2013). A survey on metric learning for feature vectors and structured data. CoRR, abs/1306.6709.
  2. Biehl, M., Hammer, B., Merényi, E., Sperduti, A., and Villmann, T., editors (2011). Learning in the context of very high dimensional data (Dagstuhl Seminar 11341), volume 1.
  3. Biehl, M., Hammer, B., Schneider, P., and Villmann, T. (2009). Metric learning for prototype based classification. In Bianchini, M., Maggini, M., and Scarselli, F., editors, Innovations in Neural Information - Paradigms and Applications, Studies in Computational Intelligence 247, pages 183-199. Springer.
  4. Biehl, M., Schneider, P., Smith, D., Stiekema, H., Taylor, A., Hughes, B., Shackleton, C., Stewart, P., and Arlt, W. (2012). Matrix relevance lvq in steroid metabolomics based classification of adrenal tumors. In ESANN.
  5. Brown, E. T., Liu, J., Brodley, C. E., and Chang, R. (2012). Dis-function: Learning distance functions interactively. In Visual Analytics Science and Technology (VAST), 2012 IEEE Conference on, pages 83-92. IEEE.
  6. Bunte, K., Biehl, M., and Hammer, B. (2012a). A general framework for dimensionality reducing data visualization mapping. Neural Computation, 24(3):771-804.
  7. Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., and Biehl, M. (2012b). Limited rank matrix learning, discriminative dimension reduction and visualization. Neural Networks, 26:159-173.
  8. Committee on the Analysis of Massive Data, Committee on Applied and Theoretical Statistics, Board on Mathematical Sciences and Their Applications, Division on Engineering and Physical Sciences, and National Research Council (2013). Frontiers in Massive Data Analysis. National Academic Press.
  9. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32:407-499.
  10. Endert, A., Fiaux, P., and North, C. (2012). Semantic interaction for visual text analytics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 473-482. ACM.
  11. Gisbrecht, A. and Hammer, B. (2014). Data visualization by nonlinear dimensionality reduction. WIREs Data Mining and Knowledge Discovery.
  12. Gisbrecht, A., Schulz, A., and Hammer, B. (2014). Parametric nonlinear dimensionality reduction using kernel t-sne. Neurocomputing.
  13. Goldberger, J., Roweis, S., Hinton, G., and Salakhutdinov, R. (2004). Neighbourhood components analysis. In Advances in Neural Information Processing Systems 17, pages 513-520. MIT Press.
  14. Hammer, B., Gisbrecht, A., and Schulz, A. (2013). Applications of discriminative dimensionality reduction. In ICPRAM.
  15. Hammer, B., He, H., and Martinetz, T. (2014). Learning and modeling big data. In Verleysen, M., editor, ESANN, pages 343-352.
  16. Jin, Y. and Hammer, B. (2014). Computational intelligence in big data [guest editorial]. IEEE Comp. Int. Mag., 9(3):12-13.
  17. Khalil, T. (2012). Big data is a big deal. White House.
  18. Lee, J. and Verleysen, M. (2009). Quality assessment of dimensionality reduction: Rank-based criteria quality assessment of dimensionality reduction: Rank-based criteria quality assessment of dimensionality reduction: Rank-based criteria quality assessment of dimensionality reduction: rank-based criteria. Neurocomputing, 72(7-9):1431-1443.
  19. Lee, J. A., Renard, E., Bernard, G., Dupont, P., and Verleysen, M. (2013). Type 1 and 2 mixtures of kullbackleibler divergences as cost functions in dimensionality reduction based on similarity preservation. Neurocomputing, 112:92-108.
  20. Lee, J. A. and Verleysen, M. (2007). Nonlinear Dimensionality Reduction. Springer.
  21. Lee, J. A. and Verleysen, M. (2010). Scale-independent quality criteria for dimensionality reduction. Pattern Recognition Letters, 31:2248-2257.
  22. Mokbel, B., Paassen, B., and Hammer, B. (2014). Adaptive distance measures for sequential data. In Verleysen, M., editor, ESANN, pages 265-270.
  23. Peltonen, J., Sandholm, M., and Kaski, S. (2013). Information retrieval perspective to interactive data visualization. In Hlawitschka, M. and Weinkauf, T., editors, Proceedings of Eurovis 2013, The Eurographics Conference on Visualization. The Eurographics Association.
  24. Riedmiller, M. and Braun, H. (1993). A direct adaptive method for faster backpropagation learning: The rprop algorithm. In Proceedings of the IEEE International Conference on Neural Networks, pages 586- 591. IEEE Press.
  25. Roweis, S. T. and Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. SCIENCE, 290:2323-2326.
  26. Rüping, S. (2006). Learning Interpretable Models. PhD thesis, Dortmund University.
  27. Schulz, A., Gisbrecht, A., and Hammer, B. (2014). Relevance learning for dimensonality reduction. In Verleysen, M., editor, ESANN, pages 165-170.
  28. Simoff, S. J., Böhlen, M. H., and Mazeika, A., editors (2008). Visual Data Mining - Theory, Techniques and Tools for Visual Analytics, volume 4404 of Lecture Notes in Computer Science. Springer.
  29. Tenenbaum, J., da Silva, V., and Langford, J. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290:2319-2323.
  30. van der Maaten, L. and Hinton, G. (2008). Visualizing high-dimensional data using t-sne. Journal of Machine Learning Research, 9:2579-2605.
  31. van der Maaten, L., Postma, E., and van den Herik, H. (2009). Dimensionality reduction: A comparative review. Technical report, Tilburg University Technical Report, TiCC-TR 2009-005.
  32. Vellido, A., Martin-Guerroro, J., and Lisboa, P. (2012). Making machine learning models interpretable. In ESANN'12.
  33. Venna, J., Peltonen, J., Nybo, K., Aidos, H., and Kaski, S. (2010). Information retrieval perspective to nonlinear dimensionality reduction for data visualization. Journal of Machine Learning Research, 11:451-490.
  34. Ward, M., Grinstein, G., and Keim, D. A. (2010). Interactive Data Visualization: Foundations, Techniques, and Application. A. K. Peters, Ltd.
  35. Yang, Z., Peltonen, J., and Kaski, S. (2013). Scalable optimization of neighbor embedding for visualization. In ICML (2), volume 28 of JMLR Proceedings, pages 127-135. JMLR.org.
  36. Zhai, Y., Ong, Y.-S., and Tsang, I. (2014). The emerging ”big dimensionality”. Computational Intelligence Magazine, IEEE, 9(3):14-26.
Download


Paper Citation


in Harvard Style

Schulz A. and Hammer B. (2015). Metric Learning in Dimensionality Reduction . In Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-076-5, pages 232-239. DOI: 10.5220/0005200802320239


in Bibtex Style

@conference{icpram15,
author={Alexander Schulz and Barbara Hammer},
title={Metric Learning in Dimensionality Reduction},
booktitle={Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2015},
pages={232-239},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005200802320239},
isbn={978-989-758-076-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Metric Learning in Dimensionality Reduction
SN - 978-989-758-076-5
AU - Schulz A.
AU - Hammer B.
PY - 2015
SP - 232
EP - 239
DO - 10.5220/0005200802320239