Studying Stability of Different Convolutional Neural Networks Against Additive Noise

Hamed H. Aghdam, Elnaz J. Heravi, Domenec Puig

Abstract

Understanding internal process of ConvNets is commonly done using visualization techniques. However, these techniques do not usually provide a tool for estimating stability of a ConvNet against noise. In this paper, we show how to analyze a ConvNet in the frequency domain. Using the frequency domain analysis, we show the reason that a ConvNet might be sensitive to a very low magnitude additive noise. Our experiments on a few ConvNets trained on different datasets reveals that convolution kernels of a trained ConvNet usually pass most of the frequencies and they are not able to effectively eliminate the effect of high frequencies.They also show that a convolution kernel with more concentrated frequency response is more stable against noise. Finally, we illustrate that augmenting a dataset with noisy images can compress the frequency response of convolution kernels.

References

  1. Ciresan, D., Meier, U., and Schmidhuber, J. (2012). Multicolumn deep neural networks for image classification. In IEEE Conference on Computer Vision and Pattern Recognition, number February, pages 3642- 3649. IEEE.
  2. Dosovitskiy, A. and Brox, T. (2015). Inverting Convolutional Networks with Convolutional Networks. pages 1-15.
  3. Fergus, R. and Perona, P. (2004). Learning Generative Visual Models from Few Training Examples :. In Computer Vision and Pattern Recognition (CVPR), Workshop on Generative-Model Based Vision.
  4. Girshick, R., Donahue, J., Darrell, T., Berkeley, U. C., and Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Cvpr'14, pages 2-9.
  5. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T., and Eecs, U. C. B. (2014). Caffe : Convolutional Architecture for Fast Feature Embedding. ACM Conference on Multimedia.
  6. Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images. pages 1-60.
  7. Krizhevsky, a., Sutskever, I., and Hinton, G. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, pages 1097-1105.
  8. Mahendran, A. and Vedaldi, A. (2014). Understanding Deep Image Representations by Inverting Them.
  9. Nguyen, a., Yosinski, J., and Clune, J. (2015). Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. Cvpr 2015.
  10. Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv preprint arXiv:1312.6034, pages 1-8.
  11. Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (2012). Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, 32:323-332.
  12. Szegedy, C., Reed, S., Sermanet, P., Vanhoucke, V., and Rabinovich, A. (2014). Going deeper with convolutions. pages 1-12.
  13. Szegedy, C., Zaremba, W., and Sutskever, I. (2013). Intriguing properties of neural networks. arXiv preprint arXiv: . . . , pages 1-10.
  14. Zeiler, M. D. and Fergus, R. (2013). Visualizing and Understanding Convolutional Networks. arXiv preprint arXiv:1311.2901.
Download


Paper Citation


in Harvard Style

Aghdam H., J. Heravi E. and Puig D. (2017). Studying Stability of Different Convolutional Neural Networks Against Additive Noise . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-226-4, pages 362-369. DOI: 10.5220/0006200003620369


in Bibtex Style

@conference{visapp17,
author={Hamed H. Aghdam and Elnaz J. Heravi and Domenec Puig},
title={Studying Stability of Different Convolutional Neural Networks Against Additive Noise},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={362-369},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006200003620369},
isbn={978-989-758-226-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)
TI - Studying Stability of Different Convolutional Neural Networks Against Additive Noise
SN - 978-989-758-226-4
AU - Aghdam H.
AU - J. Heravi E.
AU - Puig D.
PY - 2017
SP - 362
EP - 369
DO - 10.5220/0006200003620369