The Risk of Image Generator-Specific Traces in Synthetic Training Data

Georg Wimmer, Dominik Söllinger, Andreas Uhl

2024

Abstract

Deep learning based methods require large amounts of annotated training data. Using synthetic images to train deep learning models is a faster and cheaper alternative to gathering and manually annotating training data. However, synthetic images have been demonstrated to exhibit a unique model-specific fingerprint that is not present in real images. In this work, we investigate the effect of such model-specific traces on the training of CNN-based classifiers. Two different methods are applied to generate synthetic training data, a conditional GAN-based image-to-image translation method (BicycleGAN) and a conditional diffusion model (Palette). Our results show that CNN-based classifiers can easily be fooled by generator-specific traces contained in synthetic images. As we will show, classifiers can learn to discriminate based on the traces left by the generator, instead of class-specific features.

Paper Citation

in Harvard Style

Wimmer G., Söllinger D. and Uhl A. (2024). The Risk of Image Generator-Specific Traces in Synthetic Training Data. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 199-206. DOI: 10.5220/0012420600003660

in Bibtex Style

@conference{visapp24,
author={Georg Wimmer and Dominik Söllinger and Andreas Uhl},
title={The Risk of Image Generator-Specific Traces in Synthetic Training Data},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2024},
pages={199-206},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012420600003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - The Risk of Image Generator-Specific Traces in Synthetic Training Data
SN - 978-989-758-679-8
AU - Wimmer G.
AU - Söllinger D.
AU - Uhl A.
PY - 2024
SP - 199
EP - 206
DO - 10.5220/0012420600003660
PB - SciTePress