A Survey on Ontology Evaluation Methods

Joe Raad, Christophe Cruz

2015

Abstract

Ontologies nowadays have become widely used for knowledge representation, and are considered as foundation for Semantic Web. However with their wide spread usage, a question of their evaluation increased even more. This paper addresses the issue of finding an efficient ontology evaluation method by presenting the existing ontology evaluation techniques, while discussing their advantages and drawbacks. The presented ontology evaluation techniques can be grouped into four categories: gold standard-based, corpus-based, task-based and criteria based approaches.

References

  1. Alani, H., & Brewster, C. (2006). Metrics for ranking ontologies.
  2. d'Aquin, M., Schlicht, A., Stuckenschmidt, H., & Sabou, M. (2009). Criteria and evaluation for ontology modularization techniques. In Modular ontologies (pp. 67-89). Springer Berlin Heidelberg.
  3. Brewster, C., Alani, H., Dasmahapatra, S., & Wilks, Y. (2004). Data driven ontology evaluation.
  4. Dellschaft, K., & Staab, S. (2008, June). Strategies for the evaluation of ontology learning. In Proceedings of the 2008 Conference on Ontology Learning and Population: Bridging the Gap between Text and Knowledge, Frontiers in Artificial Intelligence and Applications (Vol. 167, pp. 253-272).
  5. Elias Zavitsanos, George Paliouras, and George A. Vouros. (2011, November) Gold standard evaluation of ontology learning methods through ontology transformation and alignment. IEEE Trans. on Knowl. and Data Eng., 23(11):1635-1648.
  6. Fernández, M., Overbeeke, C., Sabou, M., & Motta, E. (2009). What makes a good ontology? A case-study in fine-grained knowledge reuse. In The semantic web (pp. 61-75). Springer Berlin Heidelberg.
  7. Gangemi, A., Catenacci, C., Ciaramita, M., & Lehmann, J. (2005). Ontology evaluation and validation: an integrated formal model for the quality diagnostic task. On-line: http://www.loa-cnr.it/Files/OntoEval4Onto Dev_Final. pdf.
  8. Gangemi, A., Catenacci, C., Ciaramita, M., & Lehmann, J. (2006). Modelling ontology evaluation and validation (pp. 140-154). Springer Berlin Heidelberg.
  9. Gómez-Pérez, A. (2004). Ontology evaluation. In Handbook on ontologies (pp. 251-273). Springer Berlin Heidelberg.
  10. Gruber, T. R. (1993). A translation approach to portable ontology specifications.Knowledge acquisition, 5(2), 199-220.
  11. Gruber, T. R. (1995). Toward principles for the design of ontologies used for knowledge sharing. International journal of human-computer studies, 43(5), 907-928.
  12. Guarino, N., & Welty, C. (2004). An Overview of OntoClean in S. Staab, R. Studer (eds.), Handbook on Ontologies.
  13. Haase, P., & Sure, Y. (2005). D3. 2.1 Usage Tracking for Ontology Evolution.
  14. Hlomani, H., & Stacey, D. (2014). Approaches, methods, metrics, measures, and subjectivity in ontology evaluation: A survey. Semantic Web Journal, na (na), 1-5.
  15. Jones, M., & Alani, H. (2006). Content-based ontology ranking.
  16. Kashyap, V., Ramakrishnan, C., Thomas, C., & Sheth, A. (2005). TaxaMiner: an experimentation framework for automated taxonomy bootstrapping. International Journal of Web and Grid Services, 1(2), 240-266.
  17. Leacock, C., & Chodorow, M. (1998). Combining local context and WordNet similarity for word sense identification. WordNet: An electronic lexical database, 49(2), 265-283.
  18. Lozano-Tello, A., & Gómez-Pérez, A. (2004). Ontometric: A method to choose the appropriate ontology. Journal of Database Management, 2(15), 1-18.
  19. Maedche, A., & Staab, S. (2002). Measuring similarity between ontologies. In Knowledge engineering and knowledge management: Ontologies and the semantic web (pp. 251-263). Springer Berlin Heidelberg.
  20. Obrst, L., Ceusters, W., Mani, I., Ray, S., & Smith, B. (2007). The evaluation of ontologies. In Semantic Web (pp. 139-158). Springer US.
  21. Patel, C., Supekar, K., Lee, Y., & Park, E. K. (2003, November). OntoKhoj: a semantic web portal for ontology searching, ranking and classification. InProceedings of the 5th ACM international workshop on Web information and data management (pp. 58- 61). ACM.
  22. Ponzetto, S. P., & Strube, M. (2007, July). Deriving a large scale taxonomy from Wikipedia. In AAAI (Vol. 7, pp. 1440-1445).
  23. Porzel, R., & Malaka, R. (2004, August). A task-based approach for ontology evaluation. In ECAI Workshop on Ontology Learning and Population, Valencia, Spain.
  24. Raad, J., Bertaux, A., & Cruz, C. (2015, July). A survey on how to cross-reference web information sources. In Science and Information Conference (SAI), 2015 (pp. 609-618). IEEE.
  25. Rada, R., Mili, H., Bicknell, E., & Blettner, M. (1989). Development and application of a metric on semantic nets. Systems, Man and Cybernetics, IEEE Transactions on, 19(1), 17-30.
  26. Resnik, Philip. "Using information content to evaluate semantic similarity in a taxonomy." arXiv preprint cmp-lg/9511007 (1995).
  27. Treeratpituk, P., Khabsa, M., & Giles, C. L. (2013). Graph-based Approach to Automatic Taxonomy Generation (GraBTax). arXiv preprint arXiv:1307.1718.
  28. Ulanov, A., Shevlyakov, G., Lyubomishchenko, N., Mehra, P., & Polutin, V. (2010, August). Monte Carlo Study of Taxonomy Evaluation. In Database and Expert Systems Applications (DEXA), 2010 Workshop on (pp. 164-168). IEEE.
  29. Vrandecic, D. (2009). Ontology evaluation (pp. 293-313). Springer Berlin Heidelberg.
  30. Welty, C. A., Mahindru, R., & Chu-Carroll, J. (2003, October). Evaluating ontological analysis. In Semantic Integration Workshop (SI-2003) (p. 92).
  31. Wu, Z., & Palmer, M. (1994, June). Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on Association for Computational Linguistics (pp. 133-138). Association for Computational Linguistics.
Download


Paper Citation


in Harvard Style

Raad J. and Cruz C. (2015). A Survey on Ontology Evaluation Methods . In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 2: KEOD, (IC3K 2015) ISBN 978-989-758-158-8, pages 179-186. DOI: 10.5220/0005591001790186


in Bibtex Style

@conference{keod15,
author={Joe Raad and Christophe Cruz},
title={A Survey on Ontology Evaluation Methods},
booktitle={Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 2: KEOD, (IC3K 2015)},
year={2015},
pages={179-186},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005591001790186},
isbn={978-989-758-158-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 2: KEOD, (IC3K 2015)
TI - A Survey on Ontology Evaluation Methods
SN - 978-989-758-158-8
AU - Raad J.
AU - Cruz C.
PY - 2015
SP - 179
EP - 186
DO - 10.5220/0005591001790186