HERO-GPT: Zero-Shot Conversational Assistance in Industrial Domains Exploiting Large Language Models

Luca Strano, Claudia Bonanno, Francesco Ragusa, Francesco Ragusa, Giovanni Farinella, Giovanni Farinella, Giovanni Farinella, Antonino Furnari, Antonino Furnari

2024

Abstract

We introduce HERO-GPT, a Multi-Modal Virtual Assistant built on a Multi-Agent System designed to swiftly adapt to any procedural context minimizing the need for training on context-specific data. In contrast to traditional approaches to conversational agents, HERO-GPT utilizes a series of dynamically interchangeable documents instead of datasets, hand-written rules, or conversational examples, to provide information on the given scenario. This paper presents the system’s capability to adapt to an industrial domain scenario through the integration of a GPT-based Large Language Model and an object detector to support Visual Question Answering. HERO-GPT is capable of offering conversational guidance on various aspects of industrial contexts, including information on Personal Protective Equipment (PPE), machinery, procedures, and best practices. Experiments performed in an industrial laboratory with real users demonstrate HERO-GPT’s effectiveness. Results indicate that users clearly prefer the proposed virtual assistant over traditional supporting materials such as paper-based manuals in the considered scenario. Moreover, the performance of the proposed system are shown to be comparable or superior to those of traditional approaches, while requiring little domain-specific data for the setup of the system.

Paper Citation

in Harvard Style

Strano L., Bonanno C., Ragusa F., Farinella G. and Furnari A. (2024). HERO-GPT: Zero-Shot Conversational Assistance in Industrial Domains Exploiting Large Language Models. In Proceedings of the 4th International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE; ISBN 978-989-758-693-4, SciTePress, pages 74-82. DOI: 10.5220/0012688900003720

in Bibtex Style

@conference{improve24,
author={Luca Strano and Claudia Bonanno and Francesco Ragusa and Giovanni Farinella and Antonino Furnari},
title={HERO-GPT: Zero-Shot Conversational Assistance in Industrial Domains Exploiting Large Language Models},
booktitle={Proceedings of the 4th International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE},
year={2024},
pages={74-82},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012688900003720},
isbn={978-989-758-693-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 4th International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE
TI - HERO-GPT: Zero-Shot Conversational Assistance in Industrial Domains Exploiting Large Language Models
SN - 978-989-758-693-4
AU - Strano L.
AU - Bonanno C.
AU - Ragusa F.
AU - Farinella G.
AU - Furnari A.
PY - 2024
SP - 74
EP - 82
DO - 10.5220/0012688900003720
PB - SciTePress