Leveraging Local Domains for Image-to-Image Translation

Anthony Dell’Eva, Fabio Pizzati, Fabio Pizzati, Massimo Bertozzi, Raoul de Charette

2022

Abstract

Image-to-image (i2i) networks struggle to capture local changes because they do not affect the global scene structure. For example, translating from highway scenes to offroad, i2i networks easily focus on global color features but ignore obvious traits for humans like the absence of lane markings. In this paper, we leverage human knowledge about spatial domain characteristics which we refer to as ’local domains’ and demonstrate its benefit for image-to-image translation. Relying on a simple geometrical guidance, we train a patch-based GAN on few source data and hallucinate a new unseen domain which subsequently eases transfer learning to target. We experiment on three tasks ranging from unstructured environments to adverse weather. Our comprehensive evaluation setting shows we are able to generate realistic translations, with minimal priors, and training only on a few images. Furthermore, when trained on our translations images we show that all tested proxy tasks are significantly improved, without ever seeing target domain at training.

Paper Citation

in Harvard Style

Dell’Eva A., Pizzati F., Bertozzi M. and de Charette R. (2022). Leveraging Local Domains for Image-to-Image Translation. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 179-189. DOI: 10.5220/0010848900003124

in Bibtex Style

@conference{visapp22,
author={Anthony Dell’Eva and Fabio Pizzati and Massimo Bertozzi and Raoul de Charette},
title={Leveraging Local Domains for Image-to-Image Translation},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={179-189},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010848900003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Leveraging Local Domains for Image-to-Image Translation
SN - 978-989-758-555-5
AU - Dell’Eva A.
AU - Pizzati F.
AU - Bertozzi M.
AU - de Charette R.
PY - 2022
SP - 179
EP - 189
DO - 10.5220/0010848900003124
PB - SciTePress