Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots

Christian Hofmann, Christopher May, Patrick Ziegler, Iliya Ghotbiravandi, Jörg Franke, Sebastian Reitelshöfer

2025

Abstract

Large Language Models (LLMs) and Vision Language Models (VLMs) enable robots to perform complex tasks. However, many of today’s mobile robots cannot carry the computing hardware required to run these models on board. Furthermore, access via communication systems to external computers running these models is often impractical. Therefore, lightweight object detection models are often utilized to enable mobile robots to semantically perceive their environment. In addition, mobile robots are used in different environments, which also change regularly. Thus, an automated adaptation of object detectors would simplify the deployment of mobile robots. In this paper, we present a method for automated environment-specific individualization and adaptation of lightweight object detectors using LLMs and VLMs, which includes the automated identification of relevant object classes. We comprehensively evaluate our method and show its successful application in principle, while also pointing out shortcomings regarding semantic ambiguities and the application of VLMs for pseudo-labeling datasets with bounding box annotations.

Paper Citation

in Harvard Style

Hofmann C., May C., Ziegler P., Ghotbiravandi I., Franke J. and Reitelshöfer S. (2025). Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 851-862. DOI: 10.5220/0013162500003912

in Bibtex Style

@conference{visapp25,
author={Christian Hofmann and Christopher May and Patrick Ziegler and Iliya Ghotbiravandi and Jörg Franke and Sebastian Reitelshöfer},
title={Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2025},
pages={851-862},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013162500003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots
SN - 978-989-758-728-3
AU - Hofmann C.
AU - May C.
AU - Ziegler P.
AU - Ghotbiravandi I.
AU - Franke J.
AU - Reitelshöfer S.
PY - 2025
SP - 851
EP - 862
DO - 10.5220/0013162500003912
PB - SciTePress