Authors:
Martin Becker
;
Jens Lippel
and
Thomas Zielke
Affiliation:
University of Applied Sciences Düsseldorf, Münsterstr. 156, 40476, Düsseldorf and Germany
Keyword(s):
Deep Neural Networks, Learning Process Visualization, Machine Learning, Numerical Optimization, Gradient Descent Methods.
Related
Ontology
Subjects/Areas/Topics:
Abstract Data Visualization
;
Computer Vision, Visualization and Computer Graphics
;
General Data Visualization
;
Information and Scientific Visualization
;
Visual Data Analysis and Knowledge Discovery
;
Visual Representation and Interaction
Abstract:
We present an approach to visualizing gradient descent methods and discuss its application in the context of deep neural network (DNN) training. The result is a novel type of training error curve (a) that allows for an exploration of each individual gradient descent iteration at line search level; (b) that reflects how a DNN’s training error varies along each of the descent directions considered; (c) that is consistent with the traditional training error versus training iteration view commonly used to monitor a DNN’s training progress. We show how these three levels of detail can be easily realized as the three stages of Shneiderman’s Visual Information Seeking Mantra. This suggests the design and development of a new interactive visualization tool for the exploration of DNN learning processes. We present an example that showcases a conceivable interactive workflow when working with such a tool. Moreover, we give a first impression of a possible DNN hyperparameter analysis.