Alias-Free GAN for 3D-Aware Image Generation

Attila Szabó, Yevgeniy Puzikov, Sahan Ayvaz, Sonia Aurelio, Peter Gehler, Reza Shirvany, Malte Alf

2024

Abstract

In this work we build a 3D-aware generative model that produces high quality results with fast inference times. A 3D-aware model generates images and offers control over camera parameters to the user, so that an object can be shown from different viewpoints. The model we build combines the best of two worlds in a very direct way: alias-free Generative Adversarial Networks (GAN) and a Neural Radiance Field (NeRF) rendering, followed by image super-resolution. We show that fast and high-quality image synthesis is possible with careful modifications of the well designed architecture of StyleGAN3. Our design overcomes the problem of viewpoint inconsistency and aliasing artefacts that a direct application of lower-resolution NeRF would exhibit. We show experimental evaluation on two standard benchmark datasets, FFHQ and AFHQv2 and achieve the best or competitive performance on both. Our method does not sacrifice speed, we can render images at megapixel resolution at interactive frame rates.

Paper Citation

in Harvard Style

Szabó A., Puzikov Y., Ayvaz S., Aurelio S., Gehler P., Shirvany R. and Alf M. (2024). Alias-Free GAN for 3D-Aware Image Generation. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 221-232. DOI: 10.5220/0012432700003660

in Bibtex Style

@conference{visapp24,
author={Attila Szabó and Yevgeniy Puzikov and Sahan Ayvaz and Sonia Aurelio and Peter Gehler and Reza Shirvany and Malte Alf},
title={Alias-Free GAN for 3D-Aware Image Generation},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={221-232},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012432700003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Alias-Free GAN for 3D-Aware Image Generation
SN - 978-989-758-679-8
AU - Szabó A.
AU - Puzikov Y.
AU - Ayvaz S.
AU - Aurelio S.
AU - Gehler P.
AU - Shirvany R.
AU - Alf M.
PY - 2024
SP - 221
EP - 232
DO - 10.5220/0012432700003660
PB - SciTePress