Representational Capacity of Deep Neural Networks: A Computing Study

Bernhard Bermeitinger, Tomas Hrycej, Siegfried Handschuh

2019

Abstract

There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such representations with help of numerical training methods. Tests using prototypical problems with a known mean square minimum did not confirm this hypothesis. Minima found with the help of deep networks have always been worse than those found using shallow networks. This does not directly contradict the theoretical findings—it is possible that the superior representational capacity of deep networks is genuine while finding the mean square minimum of such deep networks is a substantially harder problem than with shallow ones.

Paper Citation

in Harvard Style

Bermeitinger B., Hrycej T. and Handschuh S. (2019). Representational Capacity of Deep Neural Networks: A Computing Study. In Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2019) - Volume 1: KDIR; ISBN 978-989-758-382-7, SciTePress, pages 532-538. DOI: 10.5220/0008364305320538

in Bibtex Style

@conference{kdir19,
author={Bernhard Bermeitinger and Tomas Hrycej and Siegfried Handschuh},
title={Representational Capacity of Deep Neural Networks: A Computing Study},
booktitle={Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2019) - Volume 1: KDIR},
year={2019},
pages={532-538},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008364305320538},
isbn={978-989-758-382-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2019) - Volume 1: KDIR
TI - Representational Capacity of Deep Neural Networks: A Computing Study
SN - 978-989-758-382-7
AU - Bermeitinger B.
AU - Hrycej T.
AU - Handschuh S.
PY - 2019
SP - 532
EP - 538
DO - 10.5220/0008364305320538
PB - SciTePress