Evaluation of Local Descriptors for Automatic Image Annotation

Ladislav Lenc

2017

Abstract

Feature extraction is the first and often also the crucial step in many computer vision applications. In this paper we aim at evaluation of three local descriptors for the automatic image annotation (AIA) task. We utilize local binary patterns (LBP), patterns of oriented edge magnitudes (POEM) and local derivative patterns (LDP). These descriptors are successfully used in many other domains such as face recognition. However, the utilization of them in the AIA field is rather infrequent. The annotation algorithm is based on the K-nearest neighbours (KNN) classifier where labels from $K$ most similar images are ``transferred'' to the annotated one. We propose a label transfer method that assigns variable number of labels to each image. It is compared with an existing approach using constant number of labels. The proposed method is evaluated on three image datasets: Li photography, IAPR-TC12 and ESP. We show that the results of the utilized local descriptors are comparable to, and in many cases outperform the texture features usually used in AIA. We also show that the proposed label transfer method increases the overall system performance. The proposed method is evaluated on three image datasets: Li photography, IAPR-TC12 and ESP. We show that the results of the utilized local descriptors are comparable to, and in many cases outperform the texture features usually used in AIA. We also show that the proposed label transfer method increases the overall system performance.

References

  1. Ahonen, T., Hadid, A., and Pietikainen, M. (2006). Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):2037-2041.
  2. Blei, D. M. and Jordan, M. I. (2003). Modeling annotated data. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 127-134. ACM.
  3. Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993-1022.
  4. Carneiro, G., Chan, A. B., Moreno, P. J., and Vasconcelos, N. (2007). Supervised learning of semantic classes for image annotation and retrieval. IEEE transactions on pattern analysis and machine intelligence, 29(3):394- 410.
  5. Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), volume 1, pages 886-893. IEEE.
  6. Feng, S., Manmatha, R., and Lavrenko, V. (2004). Multiple bernoulli relevance models for image and video annotation. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, volume 2, pages II1002. IEEE.
  7. Giordano, D., Kavasidis, I., Palazzo, S., and Spampinato, C. (2015). Nonparametric label propagation using mutual local similarity in nearest neighbors. Computer Vision and Image Understanding, 131:116-127.
  8. Guillaumin, M., Mensink, T., Verbeek, J., and Schmid, C. (2009). Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In 2009 IEEE 12th international conference on computer vision, pages 309-316. IEEE.
  9. Makadia, A., Pavlovic, V., and Kumar, S. (2008). A new baseline for image annotation. In European conference on computer vision, pages 316-329. Springer.
  10. Makadia, A., Pavlovic, V., and Kumar, S. (2010). Baselines for image annotation. International Journal of Computer Vision, 90(1):88-105.
  11. Manjunath, B. S. and Ma, W.-Y. (1996). Texture features for browsing and retrieval of image data. IEEE Transactions on pattern analysis and machine intelligence, 18(8):837-842.
  12. Murthy, V. N., Maji, S., and Manmatha, R. (2015). Automatic image annotation using deep learning representations. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pages 603-606. ACM.
  13. Ojala, T., Pietikäinen, M., and Harwood, D. (1996). A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1):51-59.
  14. Tian, G., Fu, H., and Feng, D. D. (2008). Automatic medical image categorization and annotation using lbp and mpeg-7 edge histograms. In 2008 International Conference on Information Technology and Applications in Biomedicine, pages 51-53. IEEE.
  15. Von Ahn, L. and Dabbish, L. (2004). Labeling images with a computer game. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 319-326. ACM.
  16. Vu, N.-S., Dee, H. M., and Caplier, A. (2012). Face recognition using the poem descriptor. Pattern Recognition, 45(7):2478-2488.
  17. Wang, J. Z., Li, J., and Wiederhold, G. (2001). Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Transactions on pattern analysis and machine intelligence, 23(9):947-963.
  18. Zhang, B., Gao, Y., Zhao, S., and Liu, J. (2010). Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE transactions on image processing, 19(2):533- 544.
  19. Zhang, D., Islam, M. M., and Lu, G. (2012). A review on automatic image annotation techniques. Pattern Recognition, 45(1):346-362.
Download


Paper Citation


in Harvard Style

Lenc L. (2017). Evaluation of Local Descriptors for Automatic Image Annotation . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-220-2, pages 527-534. DOI: 10.5220/0006194305270534


in Bibtex Style

@conference{icaart17,
author={Ladislav Lenc},
title={Evaluation of Local Descriptors for Automatic Image Annotation},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2017},
pages={527-534},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006194305270534},
isbn={978-989-758-220-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Evaluation of Local Descriptors for Automatic Image Annotation
SN - 978-989-758-220-2
AU - Lenc L.
PY - 2017
SP - 527
EP - 534
DO - 10.5220/0006194305270534