Deep Classifier Structures with Autoencoder for Higher-level Feature Extraction

Maysa I. A. Almulla Khalaf, John Q. Gan

2018

Abstract

This paper investigates deep classifier structures with stacked autoencoder (SAE) for higher-level feature extraction, aiming to overcome difficulties in training deep neural networks with limited training data in high-dimensional feature space, such as overfitting and vanishing/exploding gradients. A three-stage learning algorithm is proposed in this paper for training deep multilayer perceptron (DMLP) as the classifier. At the first stage, unsupervised learning is adopted using SAE to obtain the initial weights of the feature extraction layers of the DMLP. At the second stage, error back-propagation is used to train the DMLP by fixing the weights obtained at the first stage for its feature extraction layers. At the third stage, all the weights of the DMLP obtained at the second stage are refined by error back-propagation. Cross-validation is adopted to determine the network structures and the values of the learning parameters, and test datasets unseen in the cross-validation are used to evaluate the performance of the DMLP trained using the three-stage learning algorithm, in comparison with support vector machines (SVM) combined with SAE. Experimental results have demonstrated the advantages and effectiveness of the proposed method.

Paper Citation

in Harvard Style

Khalaf M. and Gan J. (2018). Deep Classifier Structures with Autoencoder for Higher-level Feature Extraction. In Proceedings of the 10th International Joint Conference on Computational Intelligence (IJCCI 2018) - Volume 1: IJCCI; ISBN 978-989-758-327-8, SciTePress, pages 31-38. DOI: 10.5220/0006883000310038

in Bibtex Style

@conference{ijcci18,
author={Maysa I. A. Almulla Khalaf and John Q. Gan},
title={Deep Classifier Structures with Autoencoder for Higher-level Feature Extraction},
booktitle={Proceedings of the 10th International Joint Conference on Computational Intelligence (IJCCI 2018) - Volume 1: IJCCI},
year={2018},
pages={31-38},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006883000310038},
isbn={978-989-758-327-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Joint Conference on Computational Intelligence (IJCCI 2018) - Volume 1: IJCCI
TI - Deep Classifier Structures with Autoencoder for Higher-level Feature Extraction
SN - 978-989-758-327-8
AU - Khalaf M.
AU - Gan J.
PY - 2018
SP - 31
EP - 38
DO - 10.5220/0006883000310038
PB - SciTePress