Travel Blog Assistant System (TBAS) - An Example Scenario of How to Enrich Text with Images and Images with Text using Online Multimedia Repositories

Marco Bressan, Gabriela Csurka, Yves Hoppenot, Jean-Michel Renders

Abstract

In this paper we present a Travel Blog Assistant System that facilitates the travel blog writing by automatically selecting for each blog paragraph written by the user the most relevant images from an uploaded image set. In order to do this, the system first automatically adds metadata to the traveler’s photos based both on a Generic Visual Categorizer (visual keywords) and by exploiting cross-content web repositories (textual keywords). For a given paragraph, the system ranks the uploaded images according to the similarity between the extracted metadata and the paragraph. The technology developed and presented here has potential beyond travel blogs, which served just as an illustrative example. Clearly, the same methodology can be used by professional users in the fields of multimedia document generation and automatic illustration and captioning.

References

  1. Technorati: http://technorati.com/pop/blogs/ (2007)
  2. IAPR-TC12-Benchmark: http://eureka.vu.edu.au/ grubinger/iapr/tc12 benchmark.html (2006)
  3. Flickr: http://www.flickr.com (2007)
  4. Realtravel: http://realtravel.com/ (2007)
  5. Travelpod: http://www.travelpod.com/ (2007)
  6. Travelblog: http://www.travelblog.org/ (2007)
  7. Trippert: http://trippert.com/ (2007)
  8. Footstops: http://footstops.com/ (2007)
  9. Travbuddy: http://www.travbuddy.com/ (2007)
  10. Travellerspoint: http://www.travellerspoint.com (2007)
  11. Google-Earth: http://earth.google.com (2007)
  12. AltaMap: http://www.geomicro.com/gis/ (2007)
  13. Perronnin, F., Dance, C.: Fisher kernels on visual vocabularies for image categorization. In: CVPR. (2007)
  14. Clinchant, S., Renders, J.M., Csurka, G.: Xrce's participation to imageclefphoto 2007. In: Working Notes of the 2007 CLEF Workshop. (2007)
  15. Clough, P., Grubinger, M., Deselaers, T., Hanbury, A., Müller, H.: Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Working Notes of the 2006 CLEF Workshop. (2006) http://www.clef-campaign.org/2006/working notes/.
  16. Grubinger, M., Clough, P., Hanbury, A., Müller, H.: Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Working Notes of the 2007 CLEF Workshop. (2007) http://www.clef-campaign.org/2007/working notes/.
  17. World-Gazetteer: http://www.world-gazetteer.com (2007)
  18. GNS: http://eart-hinfo.nga.mil/gns/html (2007) GEOnet Names Server.
  19. Getty: http://www.getty.edu/research/conducting research/ vocabularies/tgn (2007) Thesaurus of Geographical Names.
  20. Wikipedia: http://www.wikipedia.org (2007)
  21. Everingham, M., Zisserman, A., Williams, C., Gool, L.V.: The pascal visual object classes challenges (2007) http://www.pascal-network.org/challenges/VOC/.
  22. Barnard, K., Duygulu, P., de Freitas, N., Forsyth, D., Blei, D., Jordan, M.I.: Matching words and pictures. Journal of Machine Learning Research 3 (2003)
  23. Monay, F., Gatica-Perez, D.: Plsa-based image auto-annotation: Constraining the latent space. In: ACM MM. (2004)
  24. Pan, J., Yang, H., Faloutsos, C., Duygulu, P.: Gcap: Graph-based automatic image captioning. In: CVPR Workshop on Multimedia Data and Document Engineering. (2004)
  25. Feng, S., Lavrenko, V., Manmatha, R.: Multiple bernoulli relevance models for image and video annotation. In: CVPR. (2004)
  26. Wang, X., Zhang, L., F. Jing, W.Y.M.: Annosearch: Image auto-annotation by search. In: CVPR. (2006)
  27. Jeon, J., Manmatha, R.: Automatic image annotation of news images with large vocabularies and low quality training data. In: ACM MM. (2004)
  28. Maillot, N., Chevallet, J.P., Valea, V., Lim, J.H.: Ipal inter-media pseudo-relevance feedback approach to imageclef 2006 photo retrieval. In: CLEF 2006 Working Notes. (2006)
  29. Chang, Y.C., Chen, H.H.: Approaches of using a word-image ontology and an annotated image corpus as intermedia for cross-language image retrieval. In: CLEF 2006 Working Notes. (2006)
Download


Paper Citation


in Harvard Style

Bressan M., Csurka G., Hoppenot Y. and Renders J. (2008). Travel Blog Assistant System (TBAS) - An Example Scenario of How to Enrich Text with Images and Images with Text using Online Multimedia Repositories . In Metadata Mining for Image Understanding - Volume 1: MMIU, (VISIGRAPP 2008) ISBN 978-989-8111-24-1, pages 90-104. DOI: 10.5220/0002338200900104


in Bibtex Style

@conference{mmiu08,
author={Marco Bressan and Gabriela Csurka and Yves Hoppenot and Jean-Michel Renders},
title={Travel Blog Assistant System (TBAS) - An Example Scenario of How to Enrich Text with Images and Images with Text using Online Multimedia Repositories},
booktitle={Metadata Mining for Image Understanding - Volume 1: MMIU, (VISIGRAPP 2008)},
year={2008},
pages={90-104},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002338200900104},
isbn={978-989-8111-24-1},
}


in EndNote Style

TY - CONF
JO - Metadata Mining for Image Understanding - Volume 1: MMIU, (VISIGRAPP 2008)
TI - Travel Blog Assistant System (TBAS) - An Example Scenario of How to Enrich Text with Images and Images with Text using Online Multimedia Repositories
SN - 978-989-8111-24-1
AU - Bressan M.
AU - Csurka G.
AU - Hoppenot Y.
AU - Renders J.
PY - 2008
SP - 90
EP - 104
DO - 10.5220/0002338200900104