A BAYESIAN NETWORKS STRUCTURAL LEARNING ALGORITHM BASED ON A MULTIEXPERT APPROACH

Francesco Colace, Massimo De Santo, Mario Vento, Pasquale Foggia

2005

Abstract

The determination of Bayesian network structure, especially in the case of large domains, can be complex, time consuming and imprecise. Therefore, in the last years, the interest of the scientific community in learning Bayesian network structure from data is increasing. This interest is motivated by the fact that many techniques or disciplines, as data mining, text categorization, ontology building, can take advantage from structural learning. In literature we can find many structural learning algorithms but none of them provides good results in every case or dataset. In this paper we introduce a method for structural learning of Bayesian networks based on a multiexpert approach. Our method combines the outputs of five structural learning algorithms according to a majority vote combining rule. The combined approach shows a performance that is better than any single algorithm. We present an experimental validation of our algorithm on a set of “de facto” standard networks, measuring performance both in terms of the network topological reconstruction and of the correct orientation of the obtained arcs.

References

  1. Singh, M., Valtorta, M., Construction of Bayesian Network Structures from Data: a Brief Survey and an Efficient Algorithm. International Journal of Approximate Reasoning, 1995, 12:111-131
  2. Glymour, C., Scheines, R., Spirtes P. and Kelly, K., Discovering Casual Structure, Academic Press, 1987
  3. Fung R. M., Crawford S. L., Constructor: a System For The Induction of Probabilistic Models, Proceedings of AAAI-90, 1990, 762-769
  4. Pearl J., Verma T., A Theory of Inferred Causation, Principles of Knowledge Representation and Reasoning, 1991, 441-452, Morgan Kaufmann
  5. Cooper G. F., E. Herskovits, A Bayesian Method For The Induction of Probabilistic Networks From Data, Machine Learning. 1992, 9, 309-347
  6. Lauritzen S., Thiesson B., Spiegelhalter D., Diagnostic Systems Created by Model Selection Methods: A Case Study., AI and Statistics IV, Volume Lecture Notes in Statistics, 143-152. Springer Verlag, New York, 1989
  7. Suzuki J., Learning Bayesian Belief Networks Based on the MDL Principle: an Efficient Algorithm Using the Branch and Bound Technique, IEICE Trans. Inf. & Syst., Vol. E82, No. 2 February, 1999
  8. Cheng J., Greiner R., Learning Bayesian Belief Network Classifiers: Algorithms and System, Lecture Notes in Computer Science 2056, 141-160, 2001
  9. D. M. Chickering, Learning Bayesian NP-Complete, Learning from Data: AI and Statistics, Springer and Verlag, 1996
  10. D. Heckermann, Bayesian Networks for Data Mining, Journal of Knowledge Discovery and Data Mining 1(1), pag. 79-119, Kluwer Academic Publishers, 1997
  11. Bouckaert R., Probabilistic Network Construction Using the Minimum Description Length Principle, Lecture Notes in Computer Science, Vol. 747, 1993
  12. Rissanen J., Modeling by shortest data description, Automatica, Vol. 14, pp. 465-471, 1978
  13. Spirtes, P., Glymour, C., Scheines, R, Causation, Prediction and Search, MIT press, 2001
  14. Cheng , J., Bell, D., Liu, W., Learning belief networks from data: an information theory based approach, Proceedings of the Sixth ACM International Conference on Information and Knowledge Management, 1997
  15. Heckermann, D., Geiger, D., and Chickering, D.. Learning Bayesian Networks. The Combination of Knowledge and Statistical Data. Machine Learning, 1995 20(3):197-243
  16. Cheng , J., Bell, D., Liu, W., Learning Bayesian networks from data: an efficient approach based on information theory, Conference on Information and Knowledge Management, 1997
  17. Bell, D., Cheng , J., Liu, W., An Algorithm for Bayesian Belief Network Construction from Data, Proceedings of AI&STAT'97, Ft. Lauderdale, Florida, 1997
  18. Chow, C.K., Liu, C.N., Approximating Discrete Probability Distribution with Dependence Trees, IEEE Trans. Information Theory, vol.14, 1968
  19. Geiger, D., An Entropy Based Learning Algorithm of Bayesian Conditional Trees, Dubois et al., pp. 92-97
  20. Lam, W., Bacchus, F., Learning Bayesian Belief Networks: an Approach Based on the MDL principle, Computational Intelligence, Vol. 10-4, 1994
  21. Colace, F., De Santo, M., Foggia, P., Vento, M., Bayesian Network Structural Learning from Data: an Algorithms Comparison, Proceedings of International Conference on Enterprise Information Systems, Porto, 2004
  22. Ho TK, Hull JJ, Srihari SN, Decision Combination in Multiple Classifiers, IEEE Trans. On PAMI, vol. 16, 1994
  23. Kittler J., Hatef D., Matas J., On Combining Classifiers, IEEE Trans. On PAMI, vol. 20 n. 3, 1998
Download


Paper Citation


in Harvard Style

Colace F., De Santo M., Vento M. and Foggia P. (2005). A BAYESIAN NETWORKS STRUCTURAL LEARNING ALGORITHM BASED ON A MULTIEXPERT APPROACH . In Proceedings of the Seventh International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 972-8865-19-8, pages 194-200. DOI: 10.5220/0002521401940200


in Bibtex Style

@conference{iceis05,
author={Francesco Colace and Massimo De Santo and Mario Vento and Pasquale Foggia},
title={A BAYESIAN NETWORKS STRUCTURAL LEARNING ALGORITHM BASED ON A MULTIEXPERT APPROACH},
booktitle={Proceedings of the Seventh International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2005},
pages={194-200},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002521401940200},
isbn={972-8865-19-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Seventh International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - A BAYESIAN NETWORKS STRUCTURAL LEARNING ALGORITHM BASED ON A MULTIEXPERT APPROACH
SN - 972-8865-19-8
AU - Colace F.
AU - De Santo M.
AU - Vento M.
AU - Foggia P.
PY - 2005
SP - 194
EP - 200
DO - 10.5220/0002521401940200