Object Detector Differences When using Synthetic and Real Training Data

Martin Georg Ljungqvist, Otto Nordander, Arvid Mildner, Tony Liu, Pierre Nugues

2022

Abstract

To train well-behaved generalizing neural networks, sufficiently large and diverse datasets are needed. Collecting data while adhering to privacy legislation becomes increasingly difficult and annotating these large datasets is both a resource-heavy and time-consuming task. An approach to overcome these difficulties is to use synthetic data since it is inherently scalable and can be automatically annotated. However, how training on synthetic data affects the layers of a neural network is still unclear. In this paper, we train the YOLOv3 object detector on real and synthetic images from city environments. We perform a similarity analysis using Centered Kernel Alignment (CKA) to explore the effects of training on synthetic data on a layer-wise basis. The analysis captures the architecture of the detector while showing both different and similar patterns between different models. With this similarity analysis we want to give insights on how training synthetic data affects each layer and to give a better understanding of the inner workings of complex neural networks. The results show that the largest similarity between a detector trained on real data and a detector trained on synthetic data was in the early layers, and the largest difference was in the head part.

Paper Citation

in Harvard Style

Ljungqvist M., Nordander O., Mildner A., Liu T. and Nugues P. (2022). Object Detector Differences When using Synthetic and Real Training Data. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 48-59. DOI: 10.5220/0010778200003124

in Bibtex Style

@conference{visapp22,
author={Martin Georg Ljungqvist and Otto Nordander and Arvid Mildner and Tony Liu and Pierre Nugues},
title={Object Detector Differences When using Synthetic and Real Training Data},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={48-59},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010778200003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - Object Detector Differences When using Synthetic and Real Training Data
SN - 978-989-758-555-5
AU - Ljungqvist M.
AU - Nordander O.
AU - Mildner A.
AU - Liu T.
AU - Nugues P.
PY - 2022
SP - 48
EP - 59
DO - 10.5220/0010778200003124
PB - SciTePress