RELEVANCE FEEDBACK AS AN INDICATOR TO SELECT THE BEST SEARCH ENGINE - Evaluation on TREC Data

Gilles Hubert, Josiane Mothe

2007

Abstract

This paper explores information retrieval system variability and takes advantage of the fact two systems can retrieve different documents for a given query. More precisely, our approach is based on data fusion (fusion of system results) by taking into account local performances of each system. Our method considers the relevance of the very first documents retrieved by different systems and from this information selects the system that will perform the retrieval for the user. We found that this principle improves the performances of about 9%. Evaluation is based on different years of TREC evaluation program (TREC 3, 5, 6 and 7), TREC-adhoc tracks. It considers the two and five best systems that participate to TREC the corresponding year.

References

  1. Harman, D., 1994. Overview of the Third Text REtrieval Conference (TREC-3), 3rd Text Retrieval Conference, NIST Special Publication 500-226, pp 1-19.
  2. Beitzel, S. M., Frieder, O., Jensen, E. C., Grossman, D. Chowdhury A., Goharian, N., 2003. Disproving the fusion hypothesis: an analysis of data fusion via effective information retrieval strategies. SAC'03, ACM symposium on Applied computing, pp. 823-827.
  3. Buckley, C., Harman, D., 2004. The NRRC reliable information access (RIA) workshop. 27th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 528-529.
  4. Fox E. A., Shaw, J. A., 1994. Combination of Multiple Searches. 2nd Text Retrieval Conference (TREC-2), NIST Special Publication 500-215, pp. 243-252.
  5. He, B., Ounis, I., 2003. University of Glasgow at the Robust Track - A query-based Model Selection Approach for Poorly-performing Queries. 12th Text Retrieval Conference (TREC-12), NIST Special Publication 500-255, pp. 636-645.
  6. Lee, J., 1997. Analysis of multiple evidence combination. 22th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 267-276.
  7. Mothe, J., Tanguy, L., 2005. Linguistic features to predict query difficulty - A case study on previous TREC campaign. SIGIR workshop on Predicting Query Difficulty - Methods and Applications, pp. 7-10.
  8. Voohrees E., Harman, D., 2001. Overview of TREC 2001. 10th Text Retrieval Conference, NIST Special Publication 500-255, pp. 1-15.
Download


Paper Citation


in Harvard Style

Hubert G. and Mothe J. (2007). RELEVANCE FEEDBACK AS AN INDICATOR TO SELECT THE BEST SEARCH ENGINE - Evaluation on TREC Data . In Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 3: ICEIS, ISBN 978-972-8865-90-0, pages 184-189. DOI: 10.5220/0002361301840189


in Bibtex Style

@conference{iceis07,
author={Gilles Hubert and Josiane Mothe},
title={RELEVANCE FEEDBACK AS AN INDICATOR TO SELECT THE BEST SEARCH ENGINE - Evaluation on TREC Data},
booktitle={Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 3: ICEIS,},
year={2007},
pages={184-189},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002361301840189},
isbn={978-972-8865-90-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 3: ICEIS,
TI - RELEVANCE FEEDBACK AS AN INDICATOR TO SELECT THE BEST SEARCH ENGINE - Evaluation on TREC Data
SN - 978-972-8865-90-0
AU - Hubert G.
AU - Mothe J.
PY - 2007
SP - 184
EP - 189
DO - 10.5220/0002361301840189