Automatic Error Correction of GPT-Based Robot Motion Generation by Partial Affordance of Tool

Takahiro Suzuki, Yuta Ando, Manabu Hashimoto

2024

Abstract

In this research, we proposed a technique that, given a simple instruction such as ”Please make a cup of coffee” as would commonly be used when one human gives another human an instruction, determines an appropriate robot motion sequence and the tools to be used for that task and generates a motion trajectory for a robot to execute the task. The proposed method uses a large language model (GPT) to determine robot motion sequences and tools to be used. However, GPT may select tools that do not exist in the scene or are not appropriate. To correct this error, our research focuses on function and functional consistency. An everyday object has a role assigned to each region of that object, such as ”scoop” or ”contain”. There are also constraints such as the fact that a ladle must have scoop and grasp functions. The proposed method judges whether the tools in the scene are inconsistent with these constraints, and automatically corrects the tools as necessary. Experimental results confirmed that the proposed method was able to generate motion sequences a from simple instruction and that the proposed method automatically corrects errors in GPT outputs.

Paper Citation

in Harvard Style

Suzuki T., Ando Y. and Hashimoto M. (2024). Automatic Error Correction of GPT-Based Robot Motion Generation by Partial Affordance of Tool. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 314-323. DOI: 10.5220/0012375000003660

in Bibtex Style

@conference{visapp24,
author={Takahiro Suzuki and Yuta Ando and Manabu Hashimoto},
title={Automatic Error Correction of GPT-Based Robot Motion Generation by Partial Affordance of Tool},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP},
year={2024},
pages={314-323},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012375000003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP
TI - Automatic Error Correction of GPT-Based Robot Motion Generation by Partial Affordance of Tool
SN - 978-989-758-679-8
AU - Suzuki T.
AU - Ando Y.
AU - Hashimoto M.
PY - 2024
SP - 314
EP - 323
DO - 10.5220/0012375000003660
PB - SciTePress