Evaluation of Information Retrieval Models and Query Performance Predictors for Amharic Adhoc Task

Tilahun Yeshambel, Josiane Mothe, Yaregal Assabie

2023

Abstract

Query performance prediction (QPP) is the task of evaluating the quality of retrieval results of a query within the context of a retrieval model. Although several research activities have been carried out on QPP for many languages, query performance predictors are not studied yet for Amharic adhoc information retrieval (IR) task. In this paper, we present the effect of various IR models on Amharic queries, and make some analysis on the computed features for QPP methods from both Indri and Terrier indexes based on the Amharic Adhoc Information Retrieval Test Collection (2AIRTC). We conducted various experiments to attest the quality of Amharic queries and performance of IR models on 2AIRTC which is TREC-like test collection. The correlation degree between predictors is used to measure the dependence between various query performance predictors, or between a predictor and a retrieval score. Our finding shows that Jelinek-Mercer model outperformed the BM25 and Dirichlet models. The finding also indicates the correlation matrices between the query-IDF predictors and the evaluation measures show very low Pearson correlation coefficient values.

Paper Citation

in Harvard Style

Yeshambel T., Mothe J. and Assabie Y. (2023). Evaluation of Information Retrieval Models and Query Performance Predictors for Amharic Adhoc Task. In Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-671-2, SciTePress, pages 98-108. DOI: 10.5220/0012224000003598

in Bibtex Style

@conference{kdir23,
author={Tilahun Yeshambel and Josiane Mothe and Yaregal Assabie},
title={Evaluation of Information Retrieval Models and Query Performance Predictors for Amharic Adhoc Task},
booktitle={Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2023},
pages={98-108},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012224000003598},
isbn={978-989-758-671-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Evaluation of Information Retrieval Models and Query Performance Predictors for Amharic Adhoc Task
SN - 978-989-758-671-2
AU - Yeshambel T.
AU - Mothe J.
AU - Assabie Y.
PY - 2023
SP - 98
EP - 108
DO - 10.5220/0012224000003598
PB - SciTePress