Learning to Rank for Query Auto-Complete with Language Modelling in Enterprise Search

Colin Daly, Colin Daly, Lucy Hederman, Lucy Hederman

2024

Abstract

Query Auto-Completion (QAC) is of particular importance to the field of Enterprise Search, where query suggestions can steer searchers to use the appropriate organisational jargon/terminology and avoid submitting queries that produce no results. The order in which QAC candidates are presented to users (for a given prefix) can be influenced by signals, such as how often the prefix appears in the corpus, most popular completions, most frequently queried, anchor text and other of a document, or what queries are currently trending in the organisation. We measure the individual contribution of each of these heuristic signals and supplement them with a feature based on Large Language Modelling (LLM) to detect jargon/terminology. We use Learning To Rank (LTR) to combine the weighted features to create a QAC ranking model for a live Enterprise Search service. In an online A/B test over a 12-week period processing 100,000 queries, our results show that the addition of our jargon/terminology detection LLM feature to the heuristic LTR model results in a Mean Reciprocal Rank score increase of 3.8%.

Paper Citation

in Harvard Style

Daly C. and Hederman L. (2024). Learning to Rank for Query Auto-Complete with Language Modelling in Enterprise Search. In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-716-0, SciTePress, pages 15-26. DOI: 10.5220/0012888100003838

in Bibtex Style

@conference{kdir24,
author={Colin Daly and Lucy Hederman},
title={Learning to Rank for Query Auto-Complete with Language Modelling in Enterprise Search},
booktitle={Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2024},
pages={15-26},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012888100003838},
isbn={978-989-758-716-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Learning to Rank for Query Auto-Complete with Language Modelling in Enterprise Search
SN - 978-989-758-716-0
AU - Daly C.
AU - Hederman L.
PY - 2024
SP - 15
EP - 26
DO - 10.5220/0012888100003838
PB - SciTePress