CONVEX COMBINATIONS OF MAXIMUM MARGIN BAYESIAN NETWORK CLASSIFIERS

Sebastian Tschiatschek, Franz Pernkopf

Abstract

Maximum margin Bayesian networks (MMBN) can be trained by solving a convex optimization problem using, for example, interior point (IP) methods (Guo et al., 2005). However, for large datasets this training is computationally expensive (in terms of runtime and memory requirements). Therefore, we propose a less resource intensive batch method to approximately learn a MMBN classifier: we train a set of (weak) MMBN classifiers on subsets of the training data, and then exploit the convexity of the original optimization problem to obtain an approximate solution, i.e., we determine a convex combination of the weak classifiers. In experiments on different datasets we obtain similar results as for optimal MMBN determined on all training samples. However, in terms of computational efficiency (runtime) we are faster and the memory requirements are much lower. Further, the proposed method facilitates parallel implementation.

References

  1. Acid, S., Campos, L. M., and Castellano, J. G. (2005). Learning Bayesian network classifiers: Searching in a space of partially directed acyclic graphs. Machine Learning, 59:213-235.
  2. Bishop, C. M. (2007). Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, 1st ed. 2006. corr. 2nd printing edition.
  3. Boyd, S. and Lieven, V. (2004). Convex Optimization. Cambridge University Press.
  4. Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2):123-140.
  5. Freund, Y. and Schapire, R. E. (1995). A decision-theoretic generalization of on-line learning and an application to boosting.
  6. Greiner, R., Su, X., Shen, B., and Zhou, W. (2005). Structural extension to logistic regression: Discriminative parameter learning of belief net classifiers. Machine Learning, 59(3):297-322.
  7. Guo, Y., Wilkinson, D., and Schuurmans, D. (2005). Maximum margin Bayesian networks. In Proceedings of the 21th Annual Conference on Uncertainty in Artificial Intelligence, pages 233-242. AUAI Press.
  8. Karypis, G. and Kumar, V. (2006). A fast and high quality multilevel scheme for partitioning irregular graphs,. SIAM Journal on Scientific Computing, 20(1):359- 392.
  9. Koller, D. and Friedman, N. (2009). Probabilistic Graphical Models: Principles and Techniques. MIT Press.
  10. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324.
  11. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
  12. Pernkopf, F., Van Pham, T., and Bilmes, J. A. (2009). Broad phonetic classification using discriminative Bayesian networks. Speech Communication, 51(2):151-166.
  13. Pernkopf, F., Wohlmayr, M., and Tschiatschek, S. (2011). Maximum margin bayesian network classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, (accepted).
  14. Platt, J. (1999). Sequential minimal optimization: A fast algorithm for training support vector machines. Advances in Kernel Methods-Support Vector Learning, pages 1-21.
  15. Rish, I. (2001). An empirical study of the naive Bayes classifier. In IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, pages 41-46.
  16. Roos, T., Wettig, H., Grünwald, P., Myllymäki, P., and Tirri, H. (2005). On Discriminative Bayesian Network Classifiers and Logistic Regression. Machine Learning, 59(3):267-296.
  17. Schenk, O. and Gärtner, K. (2004). Solving unsymmetric sparse systems of linear equations with PARDISO. Future Generation Computer Systems, 20(3):475- 487.
  18. Schenk, O. and Gärtner, K. (2006). On fast factorization pivoting methods for symmetric indefinite sys-
Download


Paper Citation


in Harvard Style

Tschiatschek S. and Pernkopf F. (2012). CONVEX COMBINATIONS OF MAXIMUM MARGIN BAYESIAN NETWORK CLASSIFIERS . In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8425-98-0, pages 69-77. DOI: 10.5220/0003770300690077


in Bibtex Style

@conference{icpram12,
author={Sebastian Tschiatschek and Franz Pernkopf},
title={CONVEX COMBINATIONS OF MAXIMUM MARGIN BAYESIAN NETWORK CLASSIFIERS},
booktitle={Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2012},
pages={69-77},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003770300690077},
isbn={978-989-8425-98-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - CONVEX COMBINATIONS OF MAXIMUM MARGIN BAYESIAN NETWORK CLASSIFIERS
SN - 978-989-8425-98-0
AU - Tschiatschek S.
AU - Pernkopf F.
PY - 2012
SP - 69
EP - 77
DO - 10.5220/0003770300690077