A META-LEARNING METHOD FOR CONCEPT DRIFT

Runxin Wang, Lei Shi, Mícheál Ó. Foghlú, Eric Robson

Abstract

The knowledge hidden in evolving data may change with time, this issue is known as concept drift. It often causes a learning system to decrease its prediction accuracy. Most existing techniques apply ensemble methods to improve learning performance on concept drift. In this paper, we propose a novel meta learning approach for this issue and develop a method: Multi-Step Learning (MSL). In our method, a MSL learner is structured in a recursive manner, which contains all the base learners maintained in a hierarchy, ensuring the learned concepts are traceable. We evaluated MSL and two ensemble techniques on three synthetic datasets, which contain a number of drastic concept drifts. The experimental results show that the proposed method generally performs better than the ensemble techniques in terms of prediction accuracy.

References

  1. Bach, S. H. and Maloof, M. A. (2008). Paired learners for concept drift. In 2008 Eighth IEEE International Conference on Data Mining, pages 23-32.
  2. Bifet, A., Holmes, G., Pfahringer, B., Kirkby, R., and Gavalda, R. (2009). New ensemble methods for evolving data streams. In KDD09, June 28, 2009, Paris, France. ACM Press.
  3. Breiman, L. (1996). Bagging predictors. In Machine Learning, pages 123-140.
  4. Dzeroski, S. and Zenko, B. (2004). Is combining classifiers better than selecting the best one? Machine Learning, 54:255-273.
  5. Freund, Y. and Schapire, R. E. (1997). A DecisionTheoretic Generalization of On-Line Learning and an Application to Boosting.
  6. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The weka data mining software: An update. SIGKDD Explorations, 11.
  7. Hulten, G., Spencer, L., and Domingos, P. (2001). Mining time-changing data streams. In KDD'01, San Francisco, CA. ACM Press.
  8. Kittler, J., Hatef, M., Duin, R. P. W., and Matas, J. (1998). On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:226-239.
  9. Kolter, J. and Maloof, M. A. (2007). Dynamic weighted majority: An ensemble method for drifting concepts. Journal of Machine Learning Research, 8:2755-2790.
  10. Polikar, R., Udpa, L., Udpa, S. S., and Honavar, V. (2001). Learn++: An Incremental Learning Algorithm for Supervised Neural Networks. IEEE Trans. on Systems, Man and Cybernetics. Part C, 31:497-508.
  11. Schlimmer, J. C. and Granger, J. (1986). Incremental learning from noisy data. Machine Learning, 1:317-354.
  12. Street, W. N. and Kim, Y. (2001). A streaming ensemble algorithm (sea) for large-scale classification. In Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 377- 382. ACM Press.
  13. Tsymbal, A. (2004). The problem of concept drift: Definitions and related work. Computer Science Department, Trinity College Dublin, Technical Report.
  14. Widmer, G. (1997). Tracking context changes through meta-learning. Machine Learning, 27:259-286.
  15. Widmer, G. and Kubat, M. (1996). Learning in the presence of concept drift and hidden contexts. Machine Learning, 23:69-101.
Download


Paper Citation


in Harvard Style

Wang R., Shi L., Foghlú M. and Robson E. (2010). A META-LEARNING METHOD FOR CONCEPT DRIFT . In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2010) ISBN 978-989-8425-28-7, pages 257-262. DOI: 10.5220/0003095502570262


in Bibtex Style

@conference{kdir10,
author={Runxin Wang and Lei Shi and Mícheál Ó. Foghlú and Eric Robson},
title={A META-LEARNING METHOD FOR CONCEPT DRIFT},
booktitle={Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2010)},
year={2010},
pages={257-262},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003095502570262},
isbn={978-989-8425-28-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2010)
TI - A META-LEARNING METHOD FOR CONCEPT DRIFT
SN - 978-989-8425-28-7
AU - Wang R.
AU - Shi L.
AU - Foghlú M.
AU - Robson E.
PY - 2010
SP - 257
EP - 262
DO - 10.5220/0003095502570262