Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases

Elias Lumer, Vamse Kumar Subbiah, James A. Burke, Pradeep Honaganahalli Basavaraju, Austin Huber

2025

Abstract

Recent advancements in tool-equipped agents (LLMs) have enabled complex tasks like secure database interactions and code development. However, scaling tool capacity beyond agent reasoning or model limits remains a challenge. In this paper, we address these challenges by introducing Toolshed Knowledge Bases, a tool knowledge base (vector database) designed to store enhanced tool representations and optimize tool selection for large-scale tool-equipped agents. Additionally, we propose Advanced RAG-Tool Fusion, a novel ensemble of tool-applied advanced retrieval-augmented generation (RAG) techniques across the pre-retrieval, intra-retrieval, and post-retrieval phases, without requiring fine-tuning. During pre-retrieval, tool documents are enhanced with key information and stored in the Toolshed Knowledge Base. Intra-retrieval focuses on query planning and transformation to increase retrieval accuracy. Post-retrieval refines the retrieved tool documents, enables self-reflection, and equips the tools to the agent. Furthermore, by varying both the total number of tools (tool-M) an agent has access to and the tool selection threshold (top-k), we address trade-offs between retrieval accuracy, agent performance, and token cost. Our approach achieves 46%, 56%, and 47% absolute improvements on the ToolE single-tool, ToolE multi-tool and Seal-Tools benchmarks, respectively (recall@5).

Paper Citation

in Harvard Style

Lumer E., Subbiah V., Burke J., Basavaraju P. and Huber A. (2025). Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 1180-1191. DOI: 10.5220/0013303000003890

in Bibtex Style

@conference{icaart25,
author={Elias Lumer and Vamse Subbiah and James Burke and Pradeep Basavaraju and Austin Huber},
title={Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={1180-1191},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013303000003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases
SN - 978-989-758-737-5
AU - Lumer E.
AU - Subbiah V.
AU - Burke J.
AU - Basavaraju P.
AU - Huber A.
PY - 2025
SP - 1180
EP - 1191
DO - 10.5220/0013303000003890
PB - SciTePress