Multi-level Visualisation using Gaussian Process Latent Variable Models

Shahzad Mumtaz, Darren R. Flower, Ian T. Nabney

2014

Abstract

Projection of a high-dimensional dataset onto a two-dimensional space is a useful tool to visualise structures and relationships in the dataset. However, a single two-dimensional visualisation may not display all the intrinsic structure. Therefore, hierarchical/multi-level visualisation methods have been used to extract more detailed understanding of the data. Here we propose a multi-level Gaussian process latent variable model (MLGPLVM). MLGPLVM works by segmenting data (with e.g. K-means, Gaussian mixture model or interactive clustering) in the visualisation space and then fitting a visualisation model to each subset. To measure the quality of multi-level visualisation (with respect to parent and child models), metrics such as trustworthiness, continuity, mean relative rank errors, visualisation distance distortion and the negative log-likelihood per point are used. We evaluate the MLGPLVM approach on the ‘Oil Flow’ dataset and a dataset of protein electrostatic potentials for the ‘Major Histocompatibility Complex (MHC) class I’ of humans. In both cases, visual observation and the quantitative quality measures have shown better visualisation at lower levels.

References

  1. Bishop, C. and James, G. (1993). Analysis of Multiphase Flows Using Dual-Energy Gamma Densitometry and Neural Networks. Nuclear Instruments and Methods in Physics Research, 327(2-3):580-593.
  2. Bishop, C. M. and Svensen, M. (1998). GTM: The generative topographic mapping. Neural Compuatation, 10(1):215-234.
  3. Bishop, C. M. and Tipping, M. E. (1998). A hierarchical latent variable model for data visualization. IEEE Trans. Pattern Anal. Mach. Intell., 20(3):281-293.
  4. Corduneanu, A. and Bishop, C. M. (2001). Variational bayesian model selection for mixture distributions. In Artificial intelligence and Statistics, volume 2001, pages 27-34. Morgan Kaufmann Waltham, MA.
  5. Doytchinova, I. A., Guan, P., and Flower, D. R. (2004). Identifying human MHC supertypes uisng bioinformatics methods. The Journal of Immunology, 172:4314-4323.
  6. Hormann, K. and Agathos, A. (2001). The point in polygon problem for arbitrary polygons. Comput. Geom. Theory Appl., 20(3):131-144.
  7. Iwata, T., Duvenaud, D., and Ghahramani, Z. (2012). Warped mixtures for nonparametric cluster shapes. arXiv preprint arXiv:1206.1846.
  8. Lampinen, J. and Oja, E. (1992). Clustering properties of hierarchical self-organizing maps. Journal of Mathematical Imaging and Vision, 2:261-272.
  9. Larkin, J. H. and Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11(1):65-100.
  10. Lawrence, N. and Hyvrinen, A. (2005). Probabilistic non-linear principal component analysis with gaussian process latent variable models. Journal of Machine Learning Research, 6:1783-1816.
  11. Lawrence, N. D. (2004). Gaussian process latent variable models for visualisation of high dimensional data. In Thrun, S., Saul, L., and Schölkopf, B., editors, Advances in Neural Information Processing Systems 16, pages 329-336, Cambridge, MA. MIT Press.
  12. Lawrence, N. D. (2006). Local distance preservation in the GPLVM through back constraints. In In ICML, pages 513-520. ACM Press.
  13. Lawrence, N. D. (2008). Large scale learning with the gaussian process latent variable model. Technical report, university of sheffield, United Kingdom.
  14. Lawrence, N. D., Seeger, M., and Herbrich, R. (2003). Fast sparse gaussian process methods: The informative vector machine. In Advances in Neural Information Processing Systems 15, pages 609-616. MIT Press.
  15. Lee, D., Redfern, O., and Orengo, C. (2007). Predicting protein function from sequence and structure. Nature Reviews Molecular Cell Biology, 8:995-1005.
  16. Lowe, D. and Tipping, M. E. (1996). Neuroscale: Novel topographic feature extraction using RBF networks. In NIPS, pages 543-549.
  17. MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Le Cam, L. M. and Neyman, J., editors, Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability - Vol. 1, pages 281-297. University of California Press, Berkeley, CA, USA.
  18. McLachlan, G. and Basford, K. (1988). Mixture Models: Inference and Applications to Clustering. Marcel Dekker, New York.
  19. Miikkulainen, R. (1990). Script recognition with hierarchical feature maps. Connection Science, 2:83-101.
  20. Mumtaz, S., Flower, D. R., and Nabney, I. T. (2013). Multi-level visualisation. Technical report, NCRG, Aston University, Birmingham, UK. http://eprints.aston.ac.uk/.
  21. Mumtaz, S., Nabney, I. T., and Flower, D. (2012). Novel visualization methods for protein data. In Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), 2012 IEEE Symposium on, pages 198 -205.
  22. Murtagh, F. and Contreras, P. (2011). Methods of hierarchical clustering. CoRR, abs/1105.0121.
  23. Quionero-candela, J., Rasmussen, C. E., and Herbrich, R. (2005). A unifying view of sparse approximate gaussian process regression. Journal of Machine Learning Research, 6:1939-1959.
  24. Schreck, T., von Landesberger, T., and Bremm, S. (2010). Techniques for precision-based visual analysis of projected data. Information Visualization, 9(3):181-193.
  25. Shneiderman, B. (2002). Inventing discovery tools: combining information visualization with data mining. Information Visualization, 1(1):5-12.
  26. Tino, P. and Nabney, I. T. (2002). Hierarchical GTM: Constructing localized nonlinear projection manifolds in a principled way. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 24, pages 639-656.
  27. Tipping, M. E. and Bishop, C. M. (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B, 61:611-622.
  28. Venna, J. and Kaski, S. (2001). Neighborhood preservation in nonlinear projection methods: An experimental study. In Proceedings of the International Conference on Artificial Neural Networks, ICANN 7801, pages 485-491, London, UK, UK. Springer-Verlag.
  29. Versino, C. and Gambardella, L. M. (1996). Learning fine motion by using the hierarchical extended kohonen map. In MAP, Proceedings Proc. ICANN96, International Conference on Artificial Neural Networks, pages 221-226. Springer-Verlag.
  30. Vicente, D. and Vellido, A. (2004). Review of hierarchical models for data clustering and visualization. Tendencias de la Minera de Datos en Espaa, Espaola de Minera de Datos.
  31. Wang, X., Ti n?o, P., and Fardal, M. A. (2008). Multiple manifolds learning framework based on hierarchical mixture density model. In Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II, ECML PKDD 7808, pages 566-581, Berlin, Heidelberg. Springer-Verlag.
Download


Paper Citation


in Harvard Style

Mumtaz S., Flower D. and Nabney I. (2014). Multi-level Visualisation using Gaussian Process Latent Variable Models . In Proceedings of the 5th International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2014) ISBN 978-989-758-005-5, pages 122-129. DOI: 10.5220/0004686801220129


in Bibtex Style

@conference{ivapp14,
author={Shahzad Mumtaz and Darren R. Flower and Ian T. Nabney},
title={Multi-level Visualisation using Gaussian Process Latent Variable Models},
booktitle={Proceedings of the 5th International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2014)},
year={2014},
pages={122-129},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004686801220129},
isbn={978-989-758-005-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2014)
TI - Multi-level Visualisation using Gaussian Process Latent Variable Models
SN - 978-989-758-005-5
AU - Mumtaz S.
AU - Flower D.
AU - Nabney I.
PY - 2014
SP - 122
EP - 129
DO - 10.5220/0004686801220129