Absolute-ROMP: Absolute Multi-Person 3D Mesh Prediction from a Single Image

Bilal Abdulrahman, Zhigang Zhu

2023

Abstract

Recovering multi-person 3D poses and shapes with absolute scales from a single RGB image is a challenging task due to the inherent depth and scale ambiguity from a single view. Current works on 3D pose and shape estimation tend to mainly focus on the estimation of the 3D joint locations relative to the root joint , usually defined as the one closest to the shape centroid, in case of humans defined as the pelvis joint. In this paper, we build upon an existing multi-person 3D mesh predictor network, ROMP, to create Absolute-ROMP. By adding absolute root joint localization in the camera coordinate frame, we are able to estimate multi-person 3D poses and shapes with absolute scales from a single RGB image. Such a single-shot approach allows the system to better learn and reason about the inter-person depth relationship, thus improving multi-person 3D estimation. In addition to this end to end network, we also train a CNN and transformer hybrid network, called TransFocal, to predict the focal length of the image’s camera. Absolute-ROMP estimates the 3D mesh coordinates of all persons in the image and their root joint locations normalized by the focal point. We then use TransFocal to obtain focal length and get absolute depth information of all joints in the camera coordinate frame. We evaluate Absolute-ROMP on the root joint localization and root-relative 3D pose estimation tasks on publicly available multi-person 3D pose datasets. We evaluate TransFocal on dataset created from the Pano360 dataset and both are applicable to in-the-wild images and videos, due to real time performance.

Paper Citation

in Harvard Style

Abdulrahman B. and Zhu Z. (2023). Absolute-ROMP: Absolute Multi-Person 3D Mesh Prediction from a Single Image. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 69-79. DOI: 10.5220/0011629500003417

in Bibtex Style

@conference{visapp23,
author={Bilal Abdulrahman and Zhigang Zhu},
title={Absolute-ROMP: Absolute Multi-Person 3D Mesh Prediction from a Single Image},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={69-79},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011629500003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - Absolute-ROMP: Absolute Multi-Person 3D Mesh Prediction from a Single Image
SN - 978-989-758-634-7
AU - Abdulrahman B.
AU - Zhu Z.
PY - 2023
SP - 69
EP - 79
DO - 10.5220/0011629500003417
PB - SciTePress