Automatic Construction of Benchmarks for RDF Keyword Search Systems Evaluation

Angelo Neves, Luiz Leme, Yenier Izquierdo, Marco Casanova

2021

Abstract

Keyword search systems provide users with a friendly alternative to access Resource Description Framework (RDF) datasets. The evaluation of such systems requires adequate benchmarks, consisting of RDF datasets and keyword queries, with their correct answers. However, the sets of correct answers such benchmarks provide for each query are often incomplete, mostly because they are manually built with experts’ help. The central contribution of this paper is an offline method that helps build RDF keyword search benchmarks automatically, leading to more complete sets of correct answers, called solution generators. The paper focuses on computing sets of generators and describes heuristics that circumvent the combinatorial nature of the problem. The paper then describes five benchmarks, constructed with the proposed method and based on three real datasets, DBpedia, IMDb, and Mondial, and two synthetic datasets, LUBM and BSBM. Finally, the paper compares the constructed benchmarks with keyword search benchmarks published in the literature.

Paper Citation

in Harvard Style

Neves A., Leme L., Izquierdo Y. and Casanova M. (2021). Automatic Construction of Benchmarks for RDF Keyword Search Systems Evaluation. In Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-509-8, pages 126-137. DOI: 10.5220/0010519401260137

in Bibtex Style

@conference{iceis21,
author={Angelo Neves and Luiz Leme and Yenier Izquierdo and Marco Casanova},
title={Automatic Construction of Benchmarks for RDF Keyword Search Systems Evaluation},
booktitle={Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2021},
pages={126-137},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010519401260137},
isbn={978-989-758-509-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - Automatic Construction of Benchmarks for RDF Keyword Search Systems Evaluation
SN - 978-989-758-509-8
AU - Neves A.
AU - Leme L.
AU - Izquierdo Y.
AU - Casanova M.
PY - 2021
SP - 126
EP - 137
DO - 10.5220/0010519401260137