Automatic Representation and Classifier Optimization for Image-based Object Recognition

Fabian Bürger, Josef Pauli

2015

Abstract

The development of image-based object recognition systems with the desired performance is – still – a challenging task even for experts. The properties of the object feature representation have a great impact on the performance of any machine learning algorithm. Manifold learning algorithms like e.g. PCA, Isomap or Autoencoders have the potential to automatically learn lower dimensional and more useful features. However, the interplay of features, classifiers and hyperparameters is complex and needs to be carefully tuned for each learning task which is very time-consuming, if it is done manually. This paper uses a holistic optimization framework with feature selection, multiple manifold learning algorithms, multiple classifier concepts and hyperparameter optimization to automatically generate pipelines for image-based object classification. An evolutionary algorithm is used to efficiently find suitable pipeline configurations for each learning task. Experiments show the effectiveness of the proposed representation and classifier tuning on several high-dimensional object recognition datasets. The proposed system outperforms other state-of-the-art optimization frameworks.

References

  1. Bache, K. and Lichman, M. (2013). UCI machine learning repository. http://archive.ics.uci.edu/ml/.
  2. Belkin, M. and Niyogi, P. (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems (NIPS), volume 14, pages 585-591.
  3. Bengio, Y., Courville, A., and Vincent, P. (2013). Representation learning: A review and new perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(8):1798-1828.
  4. Bengio, Y., Paiement, J.-f., Vincent, P., Delalleau, O., Roux, N. L., and Ouimet, M. (2003). Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering. In Advances in Neural Information Processing Systems, page None.
  5. Beyer, H.-G. and Schwefel, H.-P. (2002). Evolution strategies - a comprehensive introduction. Natural Computing, 1(1):3-52.
  6. Bishop, C. M. and Nasrabadi, N. M. (2006). Pattern recognition and machine learning, volume 1. Springer New York.
  7. Brand, M. (2002). Charting a manifold. In Advances in neural information processing systems, pages 961-968. MIT Press.
  8. Bürger, F. and Pauli, J. (2015). Representation optimization with feature selection and manifold learning in a holistic classification framework. In International Conference on Pattern Recognition Applications and Methods (ICPRAM 2015, accepted), Lisbon, Portugal. INSTICC, SCITEPRESS.
  9. Buscema, M. (1998). Metanet*: The theory of independent judges. Substance use & misuse, 33(2):439-461.
  10. Donoho, D. L. and Grimes, C. (2003). Hessian eigenmaps: Locally linear embedding techniques for highdimensional data. Proceedings of the National Academy of Sciences, 100(10):5591-5596.
  11. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2):179- 188.
  12. Goldberger, J., Roweis, S., Hinton, G., and Salakhutdinov, R. (2004). Neighbourhood components analysis. In Advances in Neural Information Processing Systems 17.
  13. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The WEKA data mining software: An update. ACM SIGKDD Explorations Newsletter, 11(1):10-18.
  14. He, X., Cai, D., Yan, S., and Zhang, H.-J. (2005). Neighborhood preserving embedding. In Computer Vision (ICCV), 10th IEEE International Conference on, volume 2, pages 1208-1213.
  15. Hinton, G. E. and Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507.
  16. Huang, C.-L. and Wang, C.-J. (2006). A GA-based feature selection and parameters optimizationfor support vector machines. Expert Systems with Applications, 31(2):231 - 240.
  17. Huang, G.-B., Zhu, Q.-Y., and Siew, C.-K. (2006). Extreme learning machine: theory and applications. Neurocomputing, 70(1):489-501.
  18. Huang, H.-L. and Chang, F.-L. (2007). Esvm: Evolutionary support vector machine for automatic feature selection and classification of microarray data. Biosystems, 90(2):516 - 528.
  19. Jain, A. K., Duin, R. P. W., and Mao, J. (2000). Statistical pattern recognition: a review. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(1):4- 37.
  20. Lowe, D. G. (2004). Distinctive image features from scaleinvariant keypoints. International Journal of Computer Vision, 60(2):91-110.
  21. Ma, Y. and Fu, Y. (2011). Manifold Learning Theory and Applications. CRC Press.
  22. Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., Le, Q. V., and Ng, A. Y. (2011). On optimization methods for deep learning. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 265-272.
  23. Niyogi, X. (2004). Locality preserving projections. In Neural information processing systems, volume 16, page 153.
  24. Ojala, T., Pietikainen, M., and Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(7):971-987.
  25. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):559-572.
  26. Ranawana, R. and Palade, V. (2006). Multi-classifier systems: Review and a roadmap for developers. International Journal of Hybrid Intelligent Systems, 3(1):35- 61.
  27. Schölkopf, B., Smola, A., and Müller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5):1299-1319.
  28. Spearman, C. (1904). “general intelligence”, objectively determined and measured. The American Journal of Psychology, 15(2):201-292.
  29. Tenenbaum, J. B., De Silva, V., and Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323.
  30. Thornton, C., Hutter, F., Hoos, H. H., and Leyton-Brown, K. (2013). Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proc. of KDD-2013, pages 847-855.
  31. Van der Maaten, L., Postma, E., and Van Den Herik, H. (2009). Dimensionality reduction: A comparative review. Journal of Machine Learning Research, 10:1- 41.
  32. Van der Maaten, LJP, L. (2009). Learning a parametric embedding by preserving local structure. In International Conference on Artificial Intelligence and Statistics, pages 384-391.
  33. Weinberger, K. Q. and Saul, L. K. (2009). Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research, 10:207- 244.
  34. Zhang, T., Yang, J., Zhao, D., and Ge, X. (2007). Linear local tangent space alignment and application to face recognition. Neurocomputing, 70(7):1547-1553.
  35. Kernel-PCA with polynomial and Gaussian kernel (Schö lkopf et al., 1998), Denoising Autoencoder (Hinton and Salakhutdinov, 2006), Local Linear Embedding (LLE) (Donoho and Grimes, 2003), Isomap (Tenenbaum et al., 2000), Manifold Charting (Brand, 2002), Laplacian Eigenmaps (Belkin and Niyogi, 2001), parametric t-distributed Stochastic Neighborhood Embedding (t-SNE) (Van der Maaten, 2009)
Download


Paper Citation


in Harvard Style

Bürger F. and Pauli J. (2015). Automatic Representation and Classifier Optimization for Image-based Object Recognition . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-090-1, pages 542-550. DOI: 10.5220/0005359005420550


in Bibtex Style

@conference{visapp15,
author={Fabian Bürger and Josef Pauli},
title={Automatic Representation and Classifier Optimization for Image-based Object Recognition},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={542-550},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005359005420550},
isbn={978-989-758-090-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)
TI - Automatic Representation and Classifier Optimization for Image-based Object Recognition
SN - 978-989-758-090-1
AU - Bürger F.
AU - Pauli J.
PY - 2015
SP - 542
EP - 550
DO - 10.5220/0005359005420550