Authors:
Xiuling Zhou
1
;
Ping Guo
2
and
C. L. Philip Chen
3
Affiliations:
1
Beijing Normal University and Beijing City University, China
;
2
Beijing Normal University, China
;
3
The Faculty of Science & Technology and University of Macau, China
Keyword(s):
Gaussian classifier, Covariance matrix estimation, Multi-regularization parameters selection, Minimum description length.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Data Manipulation
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Learning Paradigms and Algorithms
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Supervised and Unsupervised Learning
;
Theory and Methods
Abstract:
Regularization is a solution to solve the problem of unstable estimation of covariance matrix with a small sample set in Gaussian classifier. And multi-regularization parameters estimation is more difficult than single parameter estimation. In this paper, KLIM_L covariance matrix estimation is derived theoretically based on MDL (minimum description length) principle for the small sample problem with high dimension. KLIM_L is a generalization of KLIM (Kullback-Leibler information measure) which considers the local difference in each dimension. Under the framework of MDL principle, multi-regularization parameters are selected by the criterion of minimization the KL divergence and estimated simply and directly by point estimation which is approximated by two-order Taylor expansion. It costs less computation time to estimate the multi-regularization parameters in KLIM_L than in RDA (regularized discriminant analysis) and in LOOC (leave-one-out covariance matrix estimate) where cross vali
dation technique is adopted. And higher classification accuracy is achieved by the proposed KLIM_L estimator in experiment.
(More)