LLMQuoter: Enhancing RAG Capabilities Through Efficient Quote Extraction from Large Contexts

Yuri Bezerra, Li Weigang

2025

Abstract

We introduce LLMQuoter, a lightweight, distillation-based model designed to enhance Retrieval-Augmented Generation (RAG) by extracting the most relevant textual evidence for downstream reasoning tasks. Built on the LLaMA-3B architecture and fine-tuned with Low-Rank Adaptation (LoRA) on a 15,000-sample subset of HotpotQA, LLMQuoter adopts a “quote-first-then-answer” strategy, efficiently identifying key quotes before passing curated snippets to reasoning models. This workflow reduces cognitive overhead and outperforms full-context approaches like Retrieval-Augmented Fine-Tuning (RAFT), achieving over 20-point accuracy gains across both small and large language models. By leveraging knowledge distillation from a high-performing teacher model, LLMQuoter achieves competitive results in a resource-efficient fine-tuning setup. It democratizes advanced RAG capabilities, delivering significant performance improvements without requiring extensive model retraining. Our results highlight the potential of distilled quote-based reasoning to streamline complex workflows, offering a scalable and practical solution for researchers and practitioners alike.

Paper Citation

in Harvard Style

Bezerra Y. and Weigang L. (2025). LLMQuoter: Enhancing RAG Capabilities Through Efficient Quote Extraction from Large Contexts. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 1335-1342. DOI: 10.5220/0013358700003890

in Bibtex Style

@conference{icaart25,
author={Yuri Bezerra and Li Weigang},
title={LLMQuoter: Enhancing RAG Capabilities Through Efficient Quote Extraction from Large Contexts},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={1335-1342},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013358700003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - LLMQuoter: Enhancing RAG Capabilities Through Efficient Quote Extraction from Large Contexts
SN - 978-989-758-737-5
AU - Bezerra Y.
AU - Weigang L.
PY - 2025
SP - 1335
EP - 1342
DO - 10.5220/0013358700003890
PB - SciTePress