VLLM Guided Human-Like Guidance Navigation Generation

Masaki Nambata, Tsubasa Hirakawa, Takayoshi Yamashita, Hirobobu Fujiyoshi, Takehito Teraguchi, Shota Okubo, Takuya Nanri

2025

Abstract

In the field of Advanced Driver Assistance Systems (ADAS), car navigation systems have become an essential part of modern driving. However, the guidance provided by existing car navigation systems is often difficult to understand, making it difficult for drivers to understand solely through voice instructions. This challenge has led to growing interest in Human-like Guidance (HLG), a task focused on delivering intuitive navigation instructions that mimic the way a passenger would guide a driver. Despite this, previous studies have used rule-based systems to generate HLG datasets, which have resulted in inflexible and low-quality due to limited textual representation. In contrast, high-quality datasets are crucial for improving model performance. In this study, we propose a method to automatically generate high-quality navigation sentences from image data using a Large Language Model with a novel prompting approach. Additionally, we introduce a Mixture of Experts (MoE) framework for data cleaning to filter out unreliable data. The resulting dataset is both expressive and consistent. Furthermore, our proposed MoE evaluation framework makes it possible to perform appropriate evaluation from multiple perspectives, even for complex tasks such as HLG.

Paper Citation

in Harvard Style

Nambata M., Hirakawa T., Yamashita T., Fujiyoshi H., Teraguchi T., Okubo S. and Nanri T. (2025). VLLM Guided Human-Like Guidance Navigation Generation. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 456-463. DOI: 10.5220/0013191100003912

in Bibtex Style

@conference{visapp25,
author={Masaki Nambata and Tsubasa Hirakawa and Takayoshi Yamashita and Hirobobu Fujiyoshi and Takehito Teraguchi and Shota Okubo and Takuya Nanri},
title={VLLM Guided Human-Like Guidance Navigation Generation},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2025},
pages={456-463},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013191100003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - VLLM Guided Human-Like Guidance Navigation Generation
SN - 978-989-758-728-3
AU - Nambata M.
AU - Hirakawa T.
AU - Yamashita T.
AU - Fujiyoshi H.
AU - Teraguchi T.
AU - Okubo S.
AU - Nanri T.
PY - 2025
SP - 456
EP - 463
DO - 10.5220/0013191100003912
PB - SciTePress