Locally Linear Embedding based on Rank-order Distance

Xili Sun, Yonggang Lu

2016

Abstract

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

References

  1. Belkin, M. & Niyogi, P., (2003) Laplacian eigenmaps for dimensionality reduction and data representation, Neural computation, 15 (6), 1373-1396.
  2. Comon, P., (1992). Independent component analysis, Higher-Order Statistics, 29-38.
  3. Ding, C., He, X, & Zha, H. et al., (2002). Adaptive dimension reduction for clustering high dimensional data. Proceeding of the 2002 IEEE International Conference on Data Mining, pp. 147-154.
  4. Fukunaga, K., (1990). Introduction to statistical pattern recognition, Academic press.
  5. He, L.M., Jin, W. & Yang, X.B. et al., (2013). An algorithm research of supervised LLE based on mahalanobis distance and extreme learning machine. In: Consumer Electronics, Communications and Networks (CECNet), 2013 3rd International Conference on. IEEE, pp. 76- 79.
  6. Hinton, G. E. & Roweis, S.T., (2002). Stochastic neighbor embedding. In: Advances in neural information processing systems. pp. 833-840.
  7. Hubert, L. & Schultz, J., (1976). Quadratic assignment as a general data analysis strategy, British Journal of Mathematical and Statistical Psychology, 29 (2), 190- 241.
  8. Hull, J. J., (1994). A database for handwritten text recognition research, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16 (5), 550-554.
  9. Kouropteva, O., Okun, O. & Pietikäinen, M., (2002). Selection of the optimal parameter value for the locally linear embedding algorithm, Proc. 1st Int. Conf. Fuzzy Syst. Knowl. Discov., pp. 359 -363.
  10. LeCun, Y., Bottou, L. & Bengio, Y. et al., (1998) Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86 (11), 2278-2324.
  11. Pan, Y. & Ge, S.S., (2009). A. Al Mamun, Weighted locally linear embedding for dimension reduction, Pattern Recognition, 42 (5), 798-811.
  12. Roweis, S.T. & Saul, L.K., (2000) Nonlinear dimensionality reduction by locally linear embedding, Science, 290 (5500), 2323-2326.
  13. Saul, L.K. & Roweis, S.T., (2003). Think globally, fit locally: unsupervised learning of low dimensional manifolds, The Journal of Machine Learning Research, 4, 119-155.
  14. Tenenbaum, B. et al., (2000). A global geometric framework for nonlinear dimensionality reduction, Science, 290 (5500), 2319-2323.
  15. van der Maaten, J.P.L., Postma, E.O. & van den Herik, H.J., (2009). Dimensionality reduction: A comparative review, Tilburg Univ., Tilburg, The Netherlands, Tech. Rep. TiCC-TR 2009-005.
  16. Varini, C. A., (2006). Degenhard, T.W. Nattkemper, ISOLLE: LLE with geodesic distance, Neurocomputing, 69 (13), 1768-1771.
  17. Wold, S., Esbensen, K. & Geladi, P., (1987). Principal component analysis, Chemometrics and intelligent laboratory systems, 2 (1), 37-52.
  18. Zhang, L. & Wang, N., (2007). Locally linear embedding based on image Euclidean distance. In: Automation and Logistics, 2007 IEEE International Conference on. IEEE, pp. 1914-1918.
  19. Zhang, X.F. & Huang, S.B., (2012). Mahalanobis Distance Measurement Based Locally Linear Embedding Algorithm, Pattern Recognition and Artificial Intelligence, 25, pp. 318-324.
  20. Zhang, Z. & Wang, J., (2006). MLLE: Modified locally linear embedding using multiple weights. In: Advances in Neural Information Processing Systems, pp. 1593- 1600.
  21. Zhang, Z. & Zha, H., (2004). Principal manifolds and nonlinear dimensionality reduction via tangent space alignment, Journal of Shanghai University (English Edition), 8 (4), 406-424.
  22. Zhu, C., Wen, F. & Sun, J., (2011). A rank-order distance based clustering algorithm for face tagging. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, pp. 481-488.
  23. Zhuo, L., Cheng, B. & Zhang, J., (2014). A Comparative study of dimensionality reduction methods for largescale image retrieval, Neurocomputing, 141, 202-210.
Download


Paper Citation


in Harvard Style

Sun X. and Lu Y. (2016). Locally Linear Embedding based on Rank-order Distance . In Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-173-1, pages 162-169. DOI: 10.5220/0005658601620169


in Bibtex Style

@conference{icpram16,
author={Xili Sun and Yonggang Lu},
title={Locally Linear Embedding based on Rank-order Distance},
booktitle={Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2016},
pages={162-169},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005658601620169},
isbn={978-989-758-173-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Locally Linear Embedding based on Rank-order Distance
SN - 978-989-758-173-1
AU - Sun X.
AU - Lu Y.
PY - 2016
SP - 162
EP - 169
DO - 10.5220/0005658601620169