Personalization of Dataset Retrieval Results Using a Data Valuation Method

Malick Ebiele, Malika Bendechache, Eamonn Clinton, Rob Brennan

2024

Abstract

In this paper, we propose a data valuation method that is used for Dataset Retrieval (DR) results re-ranking. Dataset retrieval is a specialization of Information Retrieval (IR) where instead of retrieving relevant documents, the information retrieval system returns a list of relevant datasets. To the best of our knowledge, data valuation has not yet been applied to dataset retrieval. By leveraging metadata and users’ preferences, we estimate the personal value of each dataset to facilitate dataset ranking and filtering. With two real users (stakeholders) and four simulated users (users’ preferences generated using a uniform weight distribution), we studied the user satisfaction rate. We define users’ satisfaction rate as the probability that users find the datasets they seek in the top k = {5,10} of the retrieval results. Previous studies of fairness in rankings (position bias) have shown that the probability or the exposure rate of a document drops exponentially from the top 1 to the top 10, from 100% to about 20%. Therefore, we calculated the Jaccard score@5 and Jaccard score@10 between our approach and other re-ranking options. It was found that there is a 42.24% and a 56.52% chance on average that users will find the dataset they are seeking in the top 5 and top 10, respectively. The lowest chance is 0% for the top 5 and 33.33% for the top 10; while the highest chance is 100% in both cases. The dataset used in our experiments is a real-world dataset and the result of a query sent to a National mapping agency data catalog. In the future, we are planning to extend the experiments performed in this paper to publicly available data catalogs.

Paper Citation

in Harvard Style

Ebiele M., Bendechache M., Clinton E. and Brennan R. (2024). Personalization of Dataset Retrieval Results Using a Data Valuation Method. In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-716-0, SciTePress, pages 122-134. DOI: 10.5220/0013044100003838

in Bibtex Style

@conference{kdir24,
author={Malick Ebiele and Malika Bendechache and Eamonn Clinton and Rob Brennan},
title={Personalization of Dataset Retrieval Results Using a Data Valuation Method},
booktitle={Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2024},
pages={122-134},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013044100003838},
isbn={978-989-758-716-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Personalization of Dataset Retrieval Results Using a Data Valuation Method
SN - 978-989-758-716-0
AU - Ebiele M.
AU - Bendechache M.
AU - Clinton E.
AU - Brennan R.
PY - 2024
SP - 122
EP - 134
DO - 10.5220/0013044100003838
PB - SciTePress