Multi-View Inversion for 3D-aware Generative Adversarial Networks

Florian Barthel, Florian Barthel, Anna Hilsmann, Peter Eisert, Peter Eisert

2024

Abstract

Current 3D GAN inversion methods for human heads typically use only one single frontal image to reconstruct the whole 3D head model. This leaves out meaningful information when multi-view data or dynamic videos are available. Our method builds on existing state-of-the-art 3D GAN inversion techniques to allow for consistent and simultaneous inversion of multiple views of the same subject. We employ a multi-latent extension to handle inconsistencies present in dynamic face videos to re-synthesize consistent 3D representations from the sequence. As our method uses additional information about the target subject, we observe significant enhancements in both geometric accuracy and image quality, particularly when rendering from wide viewing angles. Moreover, we demonstrate the editability of our inverted 3D renderings, which distinguishes them from NeRF-based scene reconstructions.

Paper Citation

in Harvard Style

Barthel F., Hilsmann A. and Eisert P. (2024). Multi-View Inversion for 3D-aware Generative Adversarial Networks. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 194-203. DOI: 10.5220/0012371000003660

in Bibtex Style

@conference{visapp24,
author={Florian Barthel and Anna Hilsmann and Peter Eisert},
title={Multi-View Inversion for 3D-aware Generative Adversarial Networks},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP},
year={2024},
pages={194-203},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012371000003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP
TI - Multi-View Inversion for 3D-aware Generative Adversarial Networks
SN - 978-989-758-679-8
AU - Barthel F.
AU - Hilsmann A.
AU - Eisert P.
PY - 2024
SP - 194
EP - 203
DO - 10.5220/0012371000003660
PB - SciTePress