MULTI-OUTPUT RANKING FOR AUTOMATED REASONING

Daniel Kühlwein, Josef Urban, Evgeni Tsivtsivadze, Herman Geuvers, Tom Heskes

2011

Abstract

Premise selection and ranking is a pressing problem for applications of automated reasoning to large formal theories and knowledge bases. Smart selection of premises has a significant impact on the efficiency of automated proof assistant systems in large theories. Despite this, machine-learning methods for this domain are underdeveloped. In this paper we propose a general learning algorithm to address the premise selection problem. Our approach consists of simultaneous training of multiple predictors that learn to rank a set of premises in order to estimate their expected usefulness when proving a new conjecture. The proposed algorithm efficiently constructs prediction functions and can take correlations among multiple tasks into account. The experiments demonstrate that the proposed method significantly outperforms algorithms previously applied to the task.

References

  1. Agarwal, S. (2005). A study of the bipartite ranking problem in machine learning. PhD thesis, University of Illinois at Urbana-Champaign, Champaign, IL, USA. AAI3182211.
  2. Bancerek, G., Endou, N., and Sakai, Y. (2001). On the characterizations of compactness. Journal of Formalized Mathematics, 13(4):733-738.
  3. Carlson, A., Cumby, C., Rizzolo, N., and Rosen, J. (1999). SNoW user manual. http://scholar.google.com/scholar?hl=en&btnG=Searc h&q=intitle:SNoW+User+Manual#1.
  4. Cortez, C. and Mori, M. (2004). AUC optimization vs. error rate minimization. Advances in Neural Information Processing Systems, 16:313.
  5. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391-407.
  6. Dems?ar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1-30.
  7. D'Silva, V., Kroening, D., and Weissenbacher, G. (2008). A survey of automated techniques for formal software verification. Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on, 27(7):1165-1178.
  8. Fuernkranz, J. and Huellermeier, E. (2011). Preference Learning. Springer Berlin Heidelberg, Berlin, Heidelberg.
  9. Gonthier, G. (2008). Formal Proof - The Four-Color Theorem. Notices of the American Mathematical Society, 55(11):1382-1393.
  10. Hales, T. (2005). A proof of the Kepler conjecture. Annals of Mathematics, 162(3):1065-1185.
  11. Hales, T. (2008). A Special Issue on Formal Proof. Notices of the American Mathematical Society, 55(11).
  12. Hurd, J. (2003). First-order proof tactics in higher-order logic theorem provers. Technical report, Design and Application of Strategies/Tactics in Higher Order Logics, number NASA/CP-2003-212448 in NASA Technical Reports.
  13. Klein, G., Elphinstone, K., Heiser, G., Andronick, J., Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., Kolanski, R., Norrish, M., Sewell, T., Tuch, H., and Winwood, S. (2009). seL4: Formal Verification of an OS Kernel. In Anderson, T., editor, Proceedings of the CICM Workshop on Empirically Successful Automated Reasoning in Mathematics, pages 207-220, Big Sky, USA. ACM Press.
  14. McCune, W. (1997). Solution of the Robbins Problem. Journal of Automated Reasoning, 19(3):263-276.
  15. Meng, J. and Paulson, L. C. (2008). Translating Higherorder Problems to First-order Clauses. Journal of Automated Reasoning, 40(1):35-60.
  16. Phillips, J. and Stanovsky, D. (2008). Automated Theorem Proving in Loop Theory. In Sutcliffe, G., Colton, S., and Schulz, S., editors, Proceedings of the CICM Workshop on Empirically Successful Automated Reasoning in Mathematics, pages 42-54, Birmingham, United Kingdom.
  17. Poggio, T. and Smale, S. (2003). The Mathematics of Learning : Dealing with Data. Notices of the American Mathematical Society, 50(5):537-544.
  18. Rifkin, R., Yeo, G., Poggio, T., Rifkin, R., Yeo, G., and Poggio, T. (2003). Regularized Least-Squares Classification, chapter 131-154. IOS Press.
  19. Roederer, A., Puzis, Y., and Sutcliffe, G. (2009). Divvy: An ATP Meta-system Based on Axiom Relevance Ordering. Automated Deduction - CADE-22, pages 157- 162.
  20. Schoelkopf, B., Herbrich, R., Williamson, R., and Smola, A. J. (2001). A Generalized Representer Theorem. In Helmbold, D. and Williamson, R., editors, Proceedings of the 14th Annual Conference on Computational Learning Theory, pages 416-426, Berlin, Germany.
  21. Shawe-Taylor, J. and Cristianini, N. (2004). Kernel Methods for Pattern Analysis, volume 47. Cambridge University Press.
  22. Sutcliffe, G. and Suttner, C. (2006). The state of CASC. AI Communications, 19:35-48.
  23. Tsivtsivadze, E., Pahikkala, T., Boberg, J., Salakoski, T., and Heskes, T. (2010). Co-Regularized LeastSquares for Label Ranking, chapter 5, pages 107-123. Springer.
  24. Tsivtsivadze, E., Urban, J., Geuvers, H., and Heskes, T. (2011). Semantic Graph Kernels for Automated Reasoning. In SIAM Conference on Data Mining.
  25. Tsoumakas, G., Katakis, I., and Vlahavas, I. P. (2010). Mining multi-label data. In Data Mining and Knowledge Discovery Handbook, pages 667-685. editor.
  26. Urban, J. (2006). MPTP 0.2: Design, Implementation, and Initial Experiments. Journal of Automated Reasoning, 37(1-2):21-43.
  27. Urban, J., Hoder, K., and Voronkov, A. (2010). Evaluation of Automated Theorem Proving on the Mizar Mathematical Library. Mathematical Software - ICMS 2010, pages 155-166.
  28. Urban, J. and Sutcliffe, G. (2008). ATP-based CrossVerification of Mizar Proofs: Method, Systems, and First Experiments. Mathematics in Computer Science, 2(2):231-251.
  29. Urban, J., Sutcliffe, G., and Pudlák, P. (2008). MaLARea SG1-Machine Learner for Automated Reasoning with Semantic Guidance. Automated Reasoning, pages 441-456.
  30. van Gestel, T., a.K. Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., de Moor, B., and Vandewalle, J. (2004). Benchmarking Least Squares Support Vector Machine Classifiers. Machine Learning, 54(1):5-32.
  31. Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80-83.
  32. Zhang, P. and Peng, J. (2004). SVM vs Regularized Least Squares Classification. In Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, volume 1, pages 176-179. IEEE.
Download


Paper Citation


in Harvard Style

Kühlwein D., Urban J., Tsivtsivadze E., Geuvers H. and Heskes T. (2011). MULTI-OUTPUT RANKING FOR AUTOMATED REASONING . In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2011) ISBN 978-989-8425-79-9, pages 42-51. DOI: 10.5220/0003650400420051


in Bibtex Style

@conference{kdir11,
author={Daniel Kühlwein and Josef Urban and Evgeni Tsivtsivadze and Herman Geuvers and Tom Heskes},
title={MULTI-OUTPUT RANKING FOR AUTOMATED REASONING},
booktitle={Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2011)},
year={2011},
pages={42-51},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003650400420051},
isbn={978-989-8425-79-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2011)
TI - MULTI-OUTPUT RANKING FOR AUTOMATED REASONING
SN - 978-989-8425-79-9
AU - Kühlwein D.
AU - Urban J.
AU - Tsivtsivadze E.
AU - Geuvers H.
AU - Heskes T.
PY - 2011
SP - 42
EP - 51
DO - 10.5220/0003650400420051