On the Transferability of Winning Tickets in Non-natural Image Datasets

Matthia Sabatelli, Mike Kestemont, Pierre Geurts

2021

Abstract

We study the generalization properties of pruned models that are the winners of the lottery ticket hypothesis on photorealistic datasets. We analyse their potential under conditions in which training data is scarce and comes from a not-photorealistic domain. More specifically, we investigate whether pruned models that are found on the popular CIFAR-10/100 and Fashion-MNIST datasets, generalize to seven different datasets coming from the fields of digital pathology and digital heritage. Our results show that there are significant benefits in training sparse architectures over larger parametrized models, since in all of our experiments pruned networks significantly outperform their larger unpruned counterparts. These results suggest that winning initializations do contain inductive biases that are generic to neural networks, although, as reported by our experiments on the biomedical datasets, their generalization properties can be more limiting than what has so far been observed in the literature.

Paper Citation

in Harvard Style

Sabatelli M., Kestemont M. and Geurts P. (2021). On the Transferability of Winning Tickets in Non-natural Image Datasets. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 59-69. DOI: 10.5220/0010196300590069

in Bibtex Style

@conference{visapp21,
author={Matthia Sabatelli and Mike Kestemont and Pierre Geurts},
title={On the Transferability of Winning Tickets in Non-natural Image Datasets},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={59-69},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010196300590069},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - On the Transferability of Winning Tickets in Non-natural Image Datasets
SN - 978-989-758-488-6
AU - Sabatelli M.
AU - Kestemont M.
AU - Geurts P.
PY - 2021
SP - 59
EP - 69
DO - 10.5220/0010196300590069
PB - SciTePress