Authors:
Liliya Avdiyenko
1
;
Nils Bertschinger
1
and
Juergen Jost
2
Affiliations:
1
Max Planck Institute for Mathematics in the Sciences, Germany
;
2
Max Planck Institute for Mathematics in the Sciences and Santa Fe Institute, Germany
Keyword(s):
Adaptivity, Feature Selection, Mutual Information, Multivariate Density Estimation, Pattern Recognition.
Related
Ontology
Subjects/Areas/Topics:
Adaptive Architectures and Mechanisms
;
Artificial Intelligence
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Data Manipulation
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Theory and Methods
Abstract:
Feature selection helps to focus resources on relevant dimensions of input data. Usually, reducing the input dimensionality to the most informative features also simplifies subsequent tasks, such as classification. This is, for instance, important for systems operating in online mode under time constraints. However, when the training data is of limited size, it becomes difficult to define a single small subset of features sufficient for classification of all data samples. In such situations, one should select features in an adaptive manner, i.e. use different feature subsets for every testing sample. Here, we propose a sequential adaptive algorithm that for a given testing sample selects features maximizing the expected information about its class. We provide experimental evidence that especially for small data sets our algorithm outperforms two the most similar information-based static and adaptive feature selectors.