Generative Texture Super-Resolution via Differential Rendering

Milena Bagdasarian, Peter Eisert, Peter Eisert, Anna Hilsmann

2024

Abstract

Image super-resolution is a well-studied field that aims at generating high-resolution images from low-resolution inputs while preserving fine details and realistic features. Despite significant progress on regular images, inferring high-resolution textures of 3D models poses unique challenges. Due to the non-contiguous arrangement of texture patches, intended for wrapping around 3D meshes, applying conventional image super-resolution techniques to texture maps often results in artifacts and seams at texture discontinuities on the mesh. Additionally, obtaining ground truth data for texture super-resolution becomes highly complex due to the labor intensive process of hand-crafting ground truth textures for each mesh. We propose a generative deep learning network for texture map super-resolution using a differentiable renderer and calibrated reference images. Combining a super-resolution generative adversarial network (GAN) with differentiable rendering, we guide our network towards learning realistic details and seamless texture map super-resolution without a high-resolution ground truth of the texture. Instead, we use high-resolution reference images. Through the differentiable rendering approach, we include model knowledge such as 3D meshes, projection matrices, and calibrated images to bridge the domain gap between 2D image super-resolution and texture map super-resolution. Our results show textures with fine structures and improved detail, which is especially of interest in virtual and augmented reality environments depicting humans.

Paper Citation

in Harvard Style

Bagdasarian M., Eisert P. and Hilsmann A. (2024). Generative Texture Super-Resolution via Differential Rendering. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 282-289. DOI: 10.5220/0012303300003660

in Bibtex Style

@conference{visapp24,
author={Milena Bagdasarian and Peter Eisert and Anna Hilsmann},
title={Generative Texture Super-Resolution via Differential Rendering},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2024},
pages={282-289},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012303300003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - Generative Texture Super-Resolution via Differential Rendering
SN - 978-989-758-679-8
AU - Bagdasarian M.
AU - Eisert P.
AU - Hilsmann A.
PY - 2024
SP - 282
EP - 289
DO - 10.5220/0012303300003660
PB - SciTePress