Authors:
Vladimir Golovko
1
and
Aliaksandr Kroshchanka
2
Affiliations:
1
Brest State Technical University and National Research Nuclear University (MEPHI), Belarus
;
2
Brest State Technical University, Belarus
Keyword(s):
Deep Neural Networks, Deep Learning, Restricted Boltzmann Machine, Data Visualization, Machine Learning, Cross-entropy.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Learning Paradigms and Algorithms
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Theory and Methods
Abstract:
Over the last decade the deep neural networks are the powerful tool in the domain of machine learning. The important problem is training of deep neural network, because learning of such a network is much complicated compared to shallow neural networks. This is due to the vanishing gradient problem, poor local minima and unstable gradient problem. Therefore a lot of deep learning techniques were developed that permit us to overcome some limitations of conventional training approaches. In this paper we investigate the unsupervised learning in deep neural networks. We have proved that maximization of the log-likelihood input data distribution of restricted Boltzmann machine is equivalent to minimizing the cross-entropy and to special case of minimizing the mean squared error. The main contribution of this paper is a novel view and new understanding of an unsupervised learning in deep neural networks.