Predicting the Empirical Robustness of the Ontology Reasoners based on Machine Learning Techniques

Nourhène Alaya, Sadok Ben Yahia, Myriam Lamolle

2015

Abstract

Reasoning with ontologies is one of the core tasks of research in Description Logics. A variety of reasoners with highly optimized algorithms have been developed to allow inference tasks on expressive ontology languages such as OWL (DL). However, unexpected behaviours of reasoner engines is often observed in practice. Both reasoner time efficiency and result correctness would vary across input ontologies, which is hardly predictable even for experienced reasoner designers. Seeking for better understanding of reasoner empirical behaviours, we propose to use supervised machine learning techniques to automatically predict reasoner robustness from its previous running. For this purpose, we introduced a set of comprehensive ontology features. We conducted huge body of experiments for 6 well known reasoners and using over 1000 ontologies from the ORE’2014 corpus. Our learning results show that we could build highly accuracy reasoner robustness predictive models. Moreover, by interpreting these models, it would be possible to gain insights about particular ontology features likely to be reasoner robustness degrading factors.

References

  1. Abburu, S. (2012). A survey on ontology reasoners and comparison. International Journal of Computer Applications, 57(17):33-39.
  2. Alaya, N., Lamolle, M., and Yahia, S. B. (2015). Towards unveiling the ontology key features altering reasoner performances. Technical report, IUT of Montreuil, http://arxiv.org/abs/1509.08717, France.
  3. Baader, F., Calvanese, D., McGuinness, D. L., Nardi, D., and Patel-Schneider, P. F., editors (2003). The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, USA.
  4. Baader, F. and Sattler, U. (2000). Tableau algorithms for description logics. In Proceedings of the International Conference on Automated Reasoning with Analytic Tableaux and Related Methods (TABLEAUX).
  5. Bail, S., Glimm, B., Jimnez-Ruiz, E., Matentzoglu, N., Parsia, B., and Steigmiller, A. (2014). Summary ore 2014 competition. In the 3rd Int. Workshop on OWL Reasoner Evaluation (ORE 2014), Vienna, Austria.
  6. Faezeh, E. and Weichang, D. (2010). Canadian semantic web. chapter A Modular Approach to Scalable Ontology Development, pages 79-103.
  7. Gangemi, A., Catenacci, C., Ciaramita, M., and Lehmann, J. (2006). Modelling ontology evaluation and validation. In Proceedings of the 3rd European Semantic Web Conference.
  8. Garcia, S., Luengo, J., Saez, J., Lopez, V., and Herrera, F. (2013). A survey of discretization techniques: Taxonomy and empirical analysis in supervised learning. IEEE Transactions on Knowledge and Data Engineering, 25:734-750.
  9. Gardiner, T., Tsarkov, D., and Horrocks, I. (2006). Framework for an automated comparison of description logic reasoners. In Proceedings of the International Semantic Web Conference, USA, pages 654-667.
  10. Glimm, B., Horrocks, I., Motik, B., Shearer, R., and Stoilos, G. (2012). A novel approach to ontology classification. Web Semant., 14:84-101.
  11. Gonc¸alves, R. S., Matentzoglu, N., Parsia, B., and Sattler, U. (2013). The empirical robustness of description logic classification. In Informal Proceedings of the 26th International Workshop on Description Logics, Ulm, Germany, pages 197-208.
  12. Gonc¸alves, R. S., Bail, S., Jim énez-Ruiz, E., Matentzoglu, N., Parsia, B., Glimm, B., and Kazakov, Y. (2013). Owl reasoner evaluation (ore) workshop 2013 results: Short report. In ORE, pages 1-18.
  13. Gonc¸alves, R. S., Parsia, B., and Sattler, U. (2012). Performance heterogeneity and approximate reasoning in description logic ontologies. In Proceedings of the 11th International Conference on The Semantic Web, pages 82-98.
  14. Hall, M., Frank, E., Holmes, G., Pfahringer, B., and Reutemann, P. (2009). The weka data mining software: An update. SIGKDD Explor. Newsl., 11:10-18.
  15. Horrocks, I. (2003). Implementation and optimisation techniques. In The Description Logic Handbook: Theory, Implementation, and Applications, chapter 9, pages 306-346. Cambridge University Press.
  16. Horrocks, I., Kutz, O., and Sattler, U. (2006). The even more irresistible S R OI Q . In Proceedings of the 23rd Benelux Conference on Artificial Intelligence , pages 57-67.
  17. Hu, B. and Dong, W. (2014). A study on cost behaviors of binary classification measures in class-imbalanced problems. CoRR, abs/1403.7100.
  18. Kang, Y.-B., Li, Y.-F., and Krishnaswamy, S. (2012). Predicting reasoning performance using ontology metrics. In Proceedings of the 11th International Conference on The Semantic Web, pages 198-214.
  19. Kang, Y.-B., Li, Y.-F., and Krishnaswamy, S. (2014). How long will it take? accurate prediction of ontology reasoning performance. In Proceedings of the 28th AAAI Conference on Artificial Intelligence , pages 80-86.
  20. Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. In Proceedings of the Emerging Artificial Intelligence Applications in Computer Engineering Conference., pages 3-24, The Netherlands. IOS Press.
  21. LePendu, P., Noy, N., Jonquet, C., Alexander, P., Shah, N., and Musen, M. (2010). Optimize first, buy later: Analyzing metrics to ramp-up very large knowledge bases. In Proceedings of The International Semantic Web Conference, pages 486-501.
  22. Mikolàs˜ek, V. (2009). Dependability and robustness: State of the art and challenges. In Software Technologies for Future Dependable Distributed Systems, pages 25-31.
  23. Motik, B., Shearer, R., and Horrocks, I. (2009). Hypertableau reasoning for description logics. Journal of Artificial Intelligence Research , 36:165-228.
  24. OWL Working Group, W. (27 October 2009). OWL 2 Web Ontology Language: Document Overview. W3C Recommendation. Available at http://www.w3.org/TR/ owl2-overview/.
  25. Sazonau, V., Sattler, U., and Brown, G. (2014). Predicting performance of owl reasoners: Locally or globally? In Proceedings of the Fourteenth International Conference on Principles of Knowledge Representation and Reasoning.
  26. Sirin, E., Parsia, B., Grau, B. C., Kalyanpur, A., and Katz, Y. (2007). Pellet: A practical owl-dl reasoner. Web Semant., 5:51-53.
  27. Steigmiller, A., Liebig, T., and Glimm, B. (2014). Konclude: System description. Web Semantics: Science, Services and Agents on the World Wide Web, 27(1).
  28. Tartir, S., Arpinar, I. B., Moore, M., Sheth, A. P., and Aleman-Meza, B. (2005). OntoQA: Metric-based ontology quality analysis. In Proceedings of IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources.
  29. Tsarkov, D., Horrocks, I., and Patel-Schneider, P. F. (2007). Optimizing terminological reasoning for expressive description logics. J. of Automated Reasoning, 39(3):277-316.
  30. Weithöner, T., Liebig, T., Luther, M., B öhm, S., Henke, F., and Noppens, O. (2007). Real-world reasoning with owl. In Proceedings of the 4th European Conference on The Semantic Web, pages 296-310.
  31. Zhang, H., Li, Y.-F., and Tan, H. B. K. (2010). Measuring design complexity of semantic web ontologies. J. Syst. Softw., 83(5):803-814.
Download


Paper Citation


in Harvard Style

Alaya N., Ben Yahia S. and Lamolle M. (2015). Predicting the Empirical Robustness of the Ontology Reasoners based on Machine Learning Techniques . In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KEOD, (IC3K 2015) ISBN 978-989-758-158-8, pages 61-73. DOI: 10.5220/0005599800610073


in Bibtex Style

@conference{keod15,
author={Nourhène Alaya and Sadok Ben Yahia and Myriam Lamolle},
title={Predicting the Empirical Robustness of the Ontology Reasoners based on Machine Learning Techniques},
booktitle={Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KEOD, (IC3K 2015)},
year={2015},
pages={61-73},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005599800610073},
isbn={978-989-758-158-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KEOD, (IC3K 2015)
TI - Predicting the Empirical Robustness of the Ontology Reasoners based on Machine Learning Techniques
SN - 978-989-758-158-8
AU - Alaya N.
AU - Ben Yahia S.
AU - Lamolle M.
PY - 2015
SP - 61
EP - 73
DO - 10.5220/0005599800610073