Regularized Nonlinear Discriminant Analysis - An Approach to Robust Dimensionality Reduction for Data Visualization

Martin Becker, Jens Lippel, André Stuhlsatz

2017

Abstract

We present a novel approach to dimensionality reduction for data visualization that is a combination of two deep neural networks (DNNs) with different objectives. One is a nonlinear generalization of Fisher’s linear discriminant analysis (LDA). It seeks to improve the class separability in the desired feature space, which is a natural strategy to obtain well-clustered visualizations. The other DNN is a deep autoencoder. Here, an encoding and a decoding DNN are optimized simultaneously with respect to the decodability of the features obtained by encoding the data. The idea behind the combined DNN is to use the generalized discriminant analysis as an encoding DNN and to equip it with a regularizing decoding DNN. Regarding data visualization, a well-regularized DNN guarantees to learn sufficiently similar data visualizations for different sets of samples that represent the data approximately equally good. Clearly, such a robustness against fluctuations in the data is essential for real-world applications. We therefore designed two extensive experiments that involve simulated fluctuations in the data. Our results show that the combined DNN is considerably more robust than the generalized discriminant analysis alone. Moreover, we present reconstructions that reveal how the visualizable features look like back in the original data space.

References

  1. Duda, R. O., Hart, P. E., and Stork, D. G. (2000). Pattern Classification. John Wiley & Sons, Inc.
  2. Erhan, D., Bengio, Y., Courville, A., Manzagol, P.-A., and Vincent, P. (2010). Why does unsupervised pretraining help deep learning? Journal of Machine Learning Research, 11:625-660.
  3. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179-188.
  4. Hinton, G. E. and Salakhutdinov, R. R. (2006). Reducing the Dimensionality of Data with Neural Networks. Science, 313:504-507.
  5. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. of the IEEE, pages 1-46.
  6. Sips, M., Neubert, B., Lewis, J. P., and Hanrahan, P. (2009). Selecting good views of high-dimensional data using class consistency. Computer Graphics Forum, 28(3):831-838.
  7. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929-1958.
  8. Stuhlsatz, A., Lippel, J., and Zielke, T. (2012). Feature extraction with deep neural networks by a generalized discriminant analysis. IEEE Transactions on Neural Networks, 666:1-666.
  9. Stuhlsatz, A., Meyer, C., Eyben, F., ZieIke, T., Meier, G., and Schuller, B. (2011). Deep neural networks for acoustic emotion recognition: Raising the benchmarks. In Proc. IEEE Intern. Conf. on Acoustics, Speech and Signal Processing (ICASSP).
  10. Tanaka, M. (2016). Deep neural network. MATLAB Central File Exchange (# 42853). Retrieved Dec 2016.
  11. Tanaka, M. and Okutomi, M. (2014). A novel inference of a restricted boltzmann machine. In 22nd International Conference on Pattern Recognition, ICPR 2014, Stockholm, Sweden, August 24-28, 2014, pages 1526-1531.
  12. Zhai, Y., Ong, Y. S., and Tsang, I. W. (2014). The emerging ”big dimensionality”. IEEE Computational Intelligence Magazine, 9(3):14-26.
Download


Paper Citation


in Harvard Style

Becker M., Lippel J. and Stuhlsatz A. (2017). Regularized Nonlinear Discriminant Analysis - An Approach to Robust Dimensionality Reduction for Data Visualization . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, (VISIGRAPP 2017) ISBN 978-989-758-228-8, pages 116-127. DOI: 10.5220/0006167501160127


in Bibtex Style

@conference{ivapp17,
author={Martin Becker and Jens Lippel and André Stuhlsatz},
title={Regularized Nonlinear Discriminant Analysis - An Approach to Robust Dimensionality Reduction for Data Visualization},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, (VISIGRAPP 2017)},
year={2017},
pages={116-127},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006167501160127},
isbn={978-989-758-228-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, (VISIGRAPP 2017)
TI - Regularized Nonlinear Discriminant Analysis - An Approach to Robust Dimensionality Reduction for Data Visualization
SN - 978-989-758-228-8
AU - Becker M.
AU - Lippel J.
AU - Stuhlsatz A.
PY - 2017
SP - 116
EP - 127
DO - 10.5220/0006167501160127