2D Motion Generation Using Joint Spatial Information with 2CM-GPT

Ryota Inoue, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi

2025

Abstract

Various methods have been proposed for generating human motion from text due to advancements in large language models and diffusion models. However, most research has focused primarily on 3D motion generation. While 3D motion enables realistic representations, the creation and collection of datasets using motion-capture technology is costly, and its application to downstream tasks, such as pose-guided human video generation, is limited. Therefore, we propose 2D Convolutional Motion Generative Pre-trained Transformer (2CM-GPT), a method for generating two-dimensional (2D) motion from text. 2CM-GPT is based on the framework of Mo-tionGPT, a method for 3D motion generation, and uses a motion tokenizer to convert 2D motion into motion tokens while learning the relationship between text and motion using a language model. Unlike MotionGPT, which utilizes 1D convolution for processing 3D motion, 2CM-GPT uses 2D convolution for processing 2D motion. This enables more effective capture of spatial relationships between joints. Evaluation experiments demonstrated that 2CM-GPT is effective in both motion reconstruction and text-guided 2D motion generation. The generated 2D motion is also shown to be effective for pose-guided human video generation.

Paper Citation

in Harvard Style

Inoue R., Hirakawa T., Yamashita T. and Fujiyoshi H. (2025). 2D Motion Generation Using Joint Spatial Information with 2CM-GPT. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 582-590. DOI: 10.5220/0013179600003912

in Bibtex Style

@conference{visapp25,
author={Ryota Inoue and Tsubasa Hirakawa and Takayoshi Yamashita and Hironobu Fujiyoshi},
title={2D Motion Generation Using Joint Spatial Information with 2CM-GPT},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2025},
pages={582-590},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013179600003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - 2D Motion Generation Using Joint Spatial Information with 2CM-GPT
SN - 978-989-758-728-3
AU - Inoue R.
AU - Hirakawa T.
AU - Yamashita T.
AU - Fujiyoshi H.
PY - 2025
SP - 582
EP - 590
DO - 10.5220/0013179600003912
PB - SciTePress