A UTILITY CENTERED APPROACH FOR EVALUATING AND OPTIMIZING GEO-TAGGING

Albert Weichselbraun

Abstract

Geo-tagging is the process of annotating a document with its geographic focus by extracting a unique locality that describes the geographic context of the document as a whole (Amitay et al., 2004). Accurate geographic annotations are crucial for geospatial applications such as Google Maps or the IDIOM Media Watch on Climate Change (Hubmann-Haidvogel et al., 2009), but many obstacles complicate the evaluation of such tags. This paper introduces an approach for optimizing geo-tagging by applying the concept of utility from economic theory to tagging results. Computing utility scores for geo-tags allows a fine grained evaluation of the tagger’s performance in regard to multiple dimensions specified in use case specific domain ontologies and provides means for addressing problems such as different scope and coverage of evaluation corpora. The integration of external data sources and evaluation ontologies with user profiles ensures that the framework considers use case specific requirements. The presented model is instrumental in comparing different geotagging settings, evaluating the effect of design decisions, and customizing geo-tagging to a particular use cases.

References

  1. Alani, H., Jones, C. B., and Tudhope, D. (2001). Voronoibased region approximation for geographical information retrieval with gazetteers. International Journal of Geographical Information Science, 15(4):287-306.
  2. Allan, J., Carterette, B., and Lewis, J. (2005). When will information retrieval be “good enough”? In SIGIR 7805: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 433-440, New York, NY, USA. ACM.
  3. Amitay, E., Har'El, N., Sivan, R., and Soffer, A. (2004). Web-a-where: geotagging web content. In SIGIR 7804: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, pages 273-280, New York, NY, USA. ACM.
  4. Clough, P. and Sanderson, M. (2004). A proposal for comparative evaluation of automatic annotation for georeferenced documents. In Proceedings of the Workshop on Geographic Information Retrieval at SIGIR 2004.
  5. Hersh, W., Turpin, A., Price, S., Chan, B., Kramer, D., Sacherek, L., and Olson, D. (2000). Do batch and user evaluations give the same results? In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 17-24, New York, NY, USA. ACM.
  6. Hubmann-Haidvogel, A., Scharl, A., and Weichselbraun, A. (2009). Multiple coordinated views for searching and navigating web content repositories. Information Sciences, 179(12):1813-1821.
  7. Leidner, J. L. (2006). An evaluation dataset for the toponym resolution task. Computers, Environment and Urban Systems, 30:400-417.
  8. Martins, B., Silva, M. J., and Chaves, M. S. (2005). Challenges and resources for evaluating geographical ir. In GIR 7805: Proceedings of the 2005 workshop on Geographic information retrieval, pages 65-69, New York, NY, USA. ACM.
  9. Nielsen, J. and Landauer, T. K. (1993). A mathematical model of the finding of usability problems. In Proceedings of the INTERACT 7893 and CHI 7893 conference on Human factors in computing systems, pages 206-213, Amsterdam, The Netherlands. ACM.
  10. Riekert, W.-F. (2002). Automated retrieval of information in the internet by using thesauri and gazetteers as knowledge sources. Journal of Universal Computer Science, 8(6):581-590.
  11. Turpin, A. H. and Hersh, W. (2001). Why batch and user evaluations do not give the same results. In SIGIR 7801: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 225-231, New York, NY, USA. ACM.
  12. Warren, D. H. D. and Pereira, F. C. N. (1982). An efficient easily adaptable system for interpreting natural language queries. Computational Linguistics, 8(3- 4):110-122.
  13. Weichselbraun, A. (2008). Strategies for optimizing querying third party resources in semantic web applications. In Cordeiro, J., Shishkov, B., Ranchordas, A., and Helfer, M., editors, 3rd International Conference on Software and Data Technologies, pages 111-118, Porto, Portugal.
Download


Paper Citation


in Harvard Style

Weichselbraun A. (2009). A UTILITY CENTERED APPROACH FOR EVALUATING AND OPTIMIZING GEO-TAGGING . In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2009) ISBN 978-989-674-011-5, pages 134-139. DOI: 10.5220/0002289401340139


in Bibtex Style

@conference{kdir09,
author={Albert Weichselbraun},
title={A UTILITY CENTERED APPROACH FOR EVALUATING AND OPTIMIZING GEO-TAGGING},
booktitle={Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2009)},
year={2009},
pages={134-139},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002289401340139},
isbn={978-989-674-011-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2009)
TI - A UTILITY CENTERED APPROACH FOR EVALUATING AND OPTIMIZING GEO-TAGGING
SN - 978-989-674-011-5
AU - Weichselbraun A.
PY - 2009
SP - 134
EP - 139
DO - 10.5220/0002289401340139