Fuzzy Base Predictor Outputs as Conditional Selectors for Evolved Combined Prediction System

Athanasios Tsakonas, Bogdan Gabrys

2012

Abstract

In this paper, we attempt to incorporate trained base learners outputs as inputs to the antecedent parts in fuzzy rule-based construction of hybrid ensembles. To accomplish this we adopt a versatile framework for the production of ensemble systems that uses a grammar driven genetic programming to evolve combinations of multilayer perceptrons and support vector machines. We evaluate the proposed architecture using three realworld regression tasks and compare it with multi-level, hierarchical ensembles. The conducted preliminary experiments showed very interesting results indicating that given a large pool of base predictors to choose from, the outputs of some of them, when applied to fuzzy sets, can be used as selectors for building accurate ensembles from other more accurate and complementary members of the same base predictor pool.

References

  1. Alba, E., Cotta, C., and Troya, J. (1996). Evolutionary design of fuzzy logic controllers using strongly-typed gp. In Proc. 1996 IEEE Int'l Symposium on Intelligent Control. New York, NY.
  2. Brown, G., Wyatt, J., Harris, R., and Yao, X. (2005). Diversity creation methods: a survey and categorisation. Inf. Fusion, 6(1):5-20.
  3. Chandra, A. and Yao, X. (2004). Divace: Diverse and accurate ensemble learning algorithm. LNCS 3177, IDEAL 2004, 17(4):619-625.
  4. Duin, R. (2002). The combining classifier: to train or not to train? In Proc. of the 16th Int'l Conf. on Pattern Recognition, pages 765-770.
  5. Ein-Dor, P. and Feldmesser, J. (1984). Attributes of the performance of central processing units: A relative performance prediction model. Commun. ACM, 30(30):308-317.
  6. Evangelista, P., Bonissone, P., Embrechts, M., and Szymanski, B. (2005). Unsupervised fuzzy ensembles and their use in intrusion detection. In European Symposium on Artificial Neural Networks (ESANN'05). Bruges, Belgium.
  7. Fernandez, F., Tommassini, M., and Vanneschi, L. (2003). An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines, 4(1).
  8. Folino, G., Pizzuti, C., and Spezzano, G. (2003). Ensemble techniques for parallel genetic programming based classifiers. In C.Ryan, T.Soule, M.Keijzer, et al.(Eds.), Proc. of the European Conf. Gen. Prog.(EuroGP 03), LNCS 2610, pages 59-69. Springer.
  9. Frank, A. and Asuncion, A. (2010). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. CA: University of California, School of Information and Computer Science, Irvine.
  10. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. (2009). The weka data mining software: An update. SIGKDD Explorations, 11(1).
  11. Hong, J. and Cho, S. (2006). The classification of cancer based on dna microarray data that uses diverse ensemble genetic programming. Artif. Intell. in Med., 36(1):43-58.
  12. Ishibuchi, H. (2007). Multiobjective genetic fuzzy systems: Review and future research directions. In IEEE Int'l Conf. on Fuzzy Systems (FUZZ-IEEE 2007), pages 59-69. Imperial College.
  13. Jacobs, R. (1997). Bias-variance analyses of mixture-ofexperts architectures. Neural Computation, 0:369- 383.
  14. Jensen, R. and Shen, Q. (2009). New approaches to fuzzyrough feature selection. IEEE Trans. on Fuzzy Systems, 17(4):824-838.
  15. Kadlec, P. and Gabrys, B. (2011). Local learning-based adaptive soft sensor for catalyst activation prediction. AIChE Journal, 57(5):1288-1301.
  16. Koza, J. (1992). Genetic programming - On the programming of computers by means of natural selection. The MIT Press, Cambridge, Massachussets, USA.
  17. Kuncheva, L. (2003). Fuzzy versus nonfuzzy in combining classifiers designed by boosting. IEEE Trans. on Fuzzy Systems, 11(6):729-741.
  18. Liaw, A. and Wiener, M. (2002). Classification and regression by randomforest. Expert Systems with Applications (Under Review).
  19. Medina-Chico, V., Suarez, A., and Lutsko, J. F. (2001). Backpropagation in decision trees for regression. In ECML 2001, LNAI 2167, pages 348-359, Springer Verlag.
  20. Quinlan, J. R. (1992). Learning with continuous classes. In AI'92, Singapore: World Scientific.
  21. Ruta, D. and Gabrys, B. (2000). An overview of classifier fusion methods. Computing and Information Systems, 7(2):1-10.
  22. Scholkopf, B. and Smola, A. (2002). Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press, Cambridge, Massachussets, USA.
  23. Sharkey, A., Sharkey, N., Gerecke, U., and Chandroth, G. (2000). The test and select approach to ensemble combination. Multiple Classifier Systems, LNCS 1857, pages 30-44.
  24. Tsakonas, A. (2006). A comparison of classification accuracy of four genetic programming evolved intelligent structures. Information Sciences, 17(1):691-724.
  25. Tsakonas, A. and Gabrys, B. (2011). Evolving takagisugeno-kang fuzzy systems using multi-population grammar guided genetic programmings. In Int'l Conf. Evol. Comp. Theory and Appl. (ECTA'11), Paris, France.
  26. Tsakonas, A. and Gabrys, B. (2012). Gradient: Grammar-driven genetic programming framework for building multi-component, hierarchical predictive systems. Expert Systems with Applications, DOI:10.1016/j.eswa.2012.05.076.
  27. Yeh, I.-C. (2008). Modeling slump of concrete with fly ash and superplasticizer. Computers and Concrete, 5(6):559-572.
  28. Zhang, Y. and Bhattacharyya, S. (2004). Genetic programming in classifying large-scale data: an ensemble method. Information Sciences, 163(1):85-101.
  29. Zhou, Z., Wu, J., Jiang, Y., and Chen, R. (2001). Genetic algorithm based selective neural network ensemble. In 17th Int'l Joint Conf. Artif. Intell., pages 797-802, USA, Morgan Kaufmann.
Download


Paper Citation


in Harvard Style

Tsakonas A. and Gabrys B. (2012). Fuzzy Base Predictor Outputs as Conditional Selectors for Evolved Combined Prediction System . In Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: ECTA, (IJCCI 2012) ISBN 978-989-8565-33-4, pages 34-41. DOI: 10.5220/0004147600340041


in Bibtex Style

@conference{ecta12,
author={Athanasios Tsakonas and Bogdan Gabrys},
title={Fuzzy Base Predictor Outputs as Conditional Selectors for Evolved Combined Prediction System},
booktitle={Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: ECTA, (IJCCI 2012)},
year={2012},
pages={34-41},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004147600340041},
isbn={978-989-8565-33-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: ECTA, (IJCCI 2012)
TI - Fuzzy Base Predictor Outputs as Conditional Selectors for Evolved Combined Prediction System
SN - 978-989-8565-33-4
AU - Tsakonas A.
AU - Gabrys B.
PY - 2012
SP - 34
EP - 41
DO - 10.5220/0004147600340041