Automatic Tag Extraction from Social Media for Visual Labeling

Shuhua Liu, Thomas Forss

2015

Abstract

Visual labeling or automated visual annotation is of great importance to the efficient access and management of multimedia content. Many methods and techniques have been proposed for image annotation in the last decade and they have shown reasonable performance on standard datasets. Great progress has been made especially in recent couple of years with the development of deep learning models for image content analysis and extraction of content-based concept labels. However, concept objects labels are much more friendly to machine than to users. We consider that more relevant and user-friendly visual labels need to include “context” descriptors. In this study we explore the possibilities to leverage social media content as a resource for visual labeling. We developed a tag extraction system that applies heuristic rules and term weighting method to extract image tags from associated Tweet. The system retrieves tweet-image pairs from public Twitter accounts, analyzes the Tweet, and generates labels for the images. We elaborate on different visual labeling methods, tag analysis and tag refinement methods.

References

  1. Liu, X., Zhang, S., Wei, F., & Zhou, M, June. Recognizing Named Entities in Tweets. 2011. ACL (pp. 359-367).
  2. Chen M., A. Zheng, K. Weinberger, Fast Image Tagging, 30th International Conference on Machine Learning (ICML), 2013
  3. Sjöberg M., M. Koskela, S. Ishikawa and J. Laaksonen. Large-Scale Visual Concept Detection with Explicit Kernel Maps and Power Mean SVM. In Proceesdings of ACM ICMR 2013, Dallas, Texas, USA
  4. Sjöberg Mats., J. Schlüter, B. Ionescu and M. Schedl. FAR at MediaEval 2013 Violent Scenes Detection: Concept-based Violent Scenes Detection in Movies. In Proceedings of MediaEval 2013 Multi-media Benchmark Workshop, Barcelona, Spain, October 18-19, 2013.
  5. Jin, Y., Khan, L., Wang, L., and Awad, M. Image Annotations By Combining Multiple Evidence & Wordnet. Proc. of ACM Multimedia, Singapore, 2005
  6. Jain R. and P. Sinha. Content without context is meaningless. In Proceedings of the international conference on Multimedia (MM'10), pages 1259- 1268, Firenze, Italy, 2010. ACM.
  7. Sun A. and S. S. Bhowmick. Image tag clarity: in search of visual-representative tags for social images. In Proceedings of the first SIGMM workshop on Social media (WSM'09), pages 19-26, Beijing, China, 2009. ACM.
  8. Sun A. and S. S. Bhowmick. Quantifying tag representativeness of visual content of social images. In Proceedings of the international conference on Multimedia (MM'10), pages 471-480, Firenze, Italy, 2010. ACM.
  9. Sun Aixin, Sourav S. Bhowmick, Khanh Tran Nam Nguyen, Ge Bai, Image-Based Social Image Retrieval: An Empirical Evaluation, American Society for Information Science and Technology, 2011.
  10. Tang J., S. Yan, R. Hong, G.-J. Qi, and T.-S. Chua. Inferring semantic concepts from communitycontributed images and noisy tags. In Proceedings of the 17th ACM international conference on Multimedia (MM'09), pages 223-232, Beijing, China, 2009. ACM.
  11. Rorissa A., A comparative study of flickr tags and index terms in a general image collection. Journal of the American Society for Information Science and Technology (JASIST), 61(11):2230-2242, 2010.
  12. Ding, E. K. Jacob, M. Fried, I. Toma, E. Yan, S. Foo, and S. Milojevic. Upper tag ontology for integrating social tagging data. Journal of the American Society for Information Science and Technology (JASIST), 61(3):505- 521, 2010.
  13. Liu D., X.-S. Hua, L. Yang, M. Wang, and H.-J. Zhang. Tag ranking. In Proceedings of the 18th international conference on World wide web (WWW'09), pages 351-360, Madrid, Spain, 2009. ACM.
  14. Liu D., X.-S. Hua, M. Wang, and H.-J. Zhang. Image retagging. In Proceedings of the international conference on Multimedia (MM'10), pages 491-500, Firenze, Italy, 2010. ACM.
  15. Chua T.-S., J.Tang, R.Hong, H.Li, Z.Luo,and Y.Zheng. Nus-wide: a real-world web image database from national university of Singapore. In Proceeding of the ACM International Conference on Image and Video Retrieval (CIVR'09), pp 48:1-48:9, Santorini, Fira, Greece, 2009.
  16. Sawant Neela, Jia Li and James Z. Wang, " Automatic Image Semantic Interpretation using Social Action and Tagging Data," Multimedia Tools and Applications, Special Issue on Survey Papers in Multimedia by World Experts, vol. 51, no. 1, pp. 213-246, 2011.
  17. Wang Changhu, Feng Jing, Lei Zhang, Hong-Jiang Zhang, “Image Annotation Refinement using Random Walk with Restarts”, MM'06, October 23-27, 2006, Santa Barbara, California, USA
  18. Liu Dong, Xian-Sheng Hua, Meng Wang, HongJiang Zhang, Microsoft Advanced Technology Center, ICME 2009
  19. Makadia Ameesh, Vladimir Pavlovic and Sanjiv Kumar, A New Baseline for Image Annotation, Google Research, New York, NY, & Rutgers University, Piscataway, NJ, 2008
  20. Noel George E. and Gilbert L. Peterson, Context-Driven Image Annotation Using ImageNet, Proceedings of the Twenty-Sixth International Florida Artificial Intelligence Research Society Conference, IEEE 2013
  21. Wang, Feng Jing, Lei Zhang, Hong-Jiang Zhang, Image Annotation Refinement using Random Walk with Restarts, MM'06, Oct 23-27, 2006, Santa Barbara, California, USA
  22. Klavans Judith L., Raul Guerra, Rebecca LaPlante, Robert Stein, and Edward Bachta, Beyond Flickr: Not All Image Tagging Is Created Equal, Language-Action Tools for Cognitive Artificial Agents, the 2011 AAAI Workshop (WS-11-14)
  23. Stvilia, B. and Jörgensen, C., Member activities and quality of tags in a collection of historical photographs in Flickr. Journal of the American Society for Information Science and Technology, 61:2477-2489, 2010
  24. Vander Wal, Folksonomy Definition and Wikipedia, Nov. 2005, http://www.vanderwal.net/random/entrysel.php? blog=1750
  25. Trant, J. Tagging, Folksonomy and Art Museums: Early Experiments and Ongoing Research, Journal of Digital Information, Vol 10, No 1, 2009
Download


Paper Citation


in Harvard Style

Liu S. and Forss T. (2015). Automatic Tag Extraction from Social Media for Visual Labeling . In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, (IC3K 2015) ISBN 978-989-758-158-8, pages 504-510. DOI: 10.5220/0005638505040510


in Bibtex Style

@conference{kdir15,
author={Shuhua Liu and Thomas Forss},
title={Automatic Tag Extraction from Social Media for Visual Labeling},
booktitle={Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, (IC3K 2015)},
year={2015},
pages={504-510},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005638505040510},
isbn={978-989-758-158-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, (IC3K 2015)
TI - Automatic Tag Extraction from Social Media for Visual Labeling
SN - 978-989-758-158-8
AU - Liu S.
AU - Forss T.
PY - 2015
SP - 504
EP - 510
DO - 10.5220/0005638505040510