Text-to-SQL Experiments with Engineering Data Extracted from CAD Files

Júlio Campos, Grettel García, Jefferson A. de Sousa, Eduardo Corseuil, Yenier Izquierdo, Melissa Lemos, Melissa Lemos, Marco Casanova, Marco Casanova

2025

Abstract

The development of Natural Language (NL) interfaces to access relational databases attracted renewed interest with the use of Large Language Models (LLMs) to translate NL questions to SQL queries. This translation task is often referred to as text-to-SQL, a problem far from being solved for real-world databases. This paper addresses the text-to-SQL task for a specific type of real-world relational database storing data extracted from engineering CAD files. The paper introduces a prompt strategy tuned to the text-to-SQL task over such databases and presents a performance analysis of LLMs of different sizes. The experiments indicated that GPT-4o achieved the highest accuracy (96%), followed by Llama 3.1 70B Instruct (86%). Quantized versions of Gemma 2 27B and Llama 3.1 8B had a very limited performance. The main challenges faced in the text-to-SQL task involved SQL complexity and balancing speed and accuracy when using quantized open-source models.

Paper Citation

in Harvard Style

Campos J., García G., A. de Sousa J., Corseuil E., Izquierdo Y., Lemos M. and Casanova M. (2025). Text-to-SQL Experiments with Engineering Data Extracted from CAD Files. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 343-350. DOI: 10.5220/0013436800003929

in Bibtex Style

@conference{iceis25,
author={Júlio Campos and Grettel García and Jefferson A. de Sousa and Eduardo Corseuil and Yenier Izquierdo and Melissa Lemos and Marco Casanova},
title={Text-to-SQL Experiments with Engineering Data Extracted from CAD Files},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS},
year={2025},
pages={343-350},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013436800003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS
TI - Text-to-SQL Experiments with Engineering Data Extracted from CAD Files
SN - 978-989-758-749-8
AU - Campos J.
AU - García G.
AU - A. de Sousa J.
AU - Corseuil E.
AU - Izquierdo Y.
AU - Lemos M.
AU - Casanova M.
PY - 2025
SP - 343
EP - 350
DO - 10.5220/0013436800003929
PB - SciTePress