Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?

Daniel Lehmann, Marc Ebner

2021

Abstract

Before a neural network can be trained the network weights have to be initialized somehow. If a model is trained from scratch, current approaches for weight initialization are based on random values. In this work we examine another approach to initialize the weights of convolutional neural network models for image classification. Our approach relies on presetting the weights of convolutional layers based on information given in the training images. To initialize the weights of convolutional layers we use small patches extracted from the training images to preset the filters of the convolutional layers. Experiments conducted on the MNIST, CIFAR-10 and CIFAR-100 dataset show that using image patches for the network initialization performs similar to state-of-the-art initialization approaches. The advantage is that our approach is more robust with respect to the learning rate. When a suboptimal value for the learning rate is used for training, our approach performs slightly better than current approaches. As a result, information given in the training images seems to be useful for network initialization resulting in a more robust training process.

Paper Citation

in Harvard Style

Lehmann D. and Ebner M. (2021). Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 346-353. DOI: 10.5220/0010206603460353

in Bibtex Style

@conference{visapp21,
author={Daniel Lehmann and Marc Ebner},
title={Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={346-353},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010206603460353},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?
SN - 978-989-758-488-6
AU - Lehmann D.
AU - Ebner M.
PY - 2021
SP - 346
EP - 353
DO - 10.5220/0010206603460353
PB - SciTePress