Data-driven Diachronic and Categorical Evaluation of Ontologies - Framework, Measure, and Metrics

Hlomani Hlomani, Deborah A. Stacey

2014

Abstract

Ontologies are a very important technology in the semantic web. They are an approximate representation and formalization of a domain of discourse in a manner that is both machine and human interpretable. Ontology evaluation therefore, concerns itself with measuring the degree to which the ontology approximates the domain. In data-driven ontology evaluation, the correctness of an ontology is measured agains a corpus of documents about the domain. This domain knowledge is dynamic and evolves over several dimensions such as the temporal and categorical. Current research makes an assumption that is contrary to this notion and hence does not account for the existence of bias in ontology evaluation. This work addresses this gap and proposes two metrics as well as a theoretical framework. It also presents a statistical evaluation of the framework and the associated metrics.

References

  1. Alani, H., Sanghee, K., Millard, E. D., Weal, J. M., Hall, W., Lewis, H. P., and Shadbolt, R. N. (2003). Automatic ontology-based knowledge extraction from web documents. Intelligent Systems, IEEE, 18(1):14-21.
  2. Brank, J., Grobelnik, M., and Mladenic, D. (2005). A survey of ontology evaluation techniques. In Proceedings of the Conference on Data Mining and Data Warehouses (SiKDD 2005), pages 166-170.
  3. Brewster, C., Alani, H., Dasmahapatra, S., and Wilks, Y. (2004). Data-driven ontology evaluation. In Proceedings of the 4th International Conference on Language Resources and Evaluation, Lisbon, Portugal.
  4. Burton-Jones, A., Storey, C. V., Sugumaran, V., and Ahluwalia, P. (2005). A semiotic metrics suite for assessing the quality of ontologies. Data & Knowledge Engineering, 55(1):84 - 102.
  5. Hlomani, H. and Stacey, A. D. (2013). Contributing evidence to data-driven ontology evaluation: Workflow ontologies perspective. In Proceedings of the 5th International Conference on Knowledge Engineering and Ontology Development, Vilamoura, Portugal.
  6. Lusk, S., Paley, S., and Spanyi, A. (2005). The evolution of business process management as a professional discipline. In Evolution of BPM as a Professional Discipline. BPTrends.
  7. Nonaka, I. and Toyama, R. (2005). The theory of the knowledge-creating firm: subjectivity, objectivity and synthesis. Industrial and Corporate Change, 14(3):419-436.
  8. Ouyang, L., Zou, B., Qu, M., and Zhang, C. (2011). A method of ontology evaluation based on coverage, cohesion and coupling. In Fuzzy Systems and Knowledge Discovery (FSKD), 2011 Eighth International Conference on, volume 4, pages 2451 -2455.
  9. Patel, C., Supekar, K., Lee, Y., and Park, E. K. (2003). Ontokhoj: A semantic web portal for ontology searching, ranking and classification. In In Proc. 5th ACM Int. Workshop on Web Information and Data Management, pages 58-61.
  10. Sebastian, A., Noy, N., Tudorache, T., and Musen, M. (2008). A generic ontology for collaborative ontology-development workflows. In Gangemi, A. and Euzenat, J., editors, Knowledge Engineering: Practice and Patterns, volume 5268 of Lecture Notes in Computer Science, pages 318-328. Springer Berlin / Heidelberg.
  11. Spyns, P. (2005). EvaLexon: Assessing triples mined from texts. Technical Report 09, Star Lab, Brussels, Belgium.
  12. Thalhammer, A., Toma, I., Hasan, R., Simperl, E., and Vrandecic, D. (2011). How to represent knowledge diversity. Poster at 10th International Semantic Web Conference.
  13. Van Der Aalst, W. M. P., Ter Hofstede, A. H. M., Kiepuszewski, B., and Barros, A. P. (2003). Workflow patterns. Distrib. Parallel Databases, 14(1):5-51.
  14. Villalon, J. and Calvo, R. A. (2013). A decoupled architecture for scalability in text mining applications. Journal of Universal Computer Science, 19(3):406-427.
  15. Vrandecic, D. (2010). Ontology Evaluation. PhD thesis, Karlsruhe Institute of Technology, Karlsruhe, Germany.
Download


Paper Citation


in Harvard Style

Hlomani H. and A. Stacey D. (2014). Data-driven Diachronic and Categorical Evaluation of Ontologies - Framework, Measure, and Metrics . In Proceedings of the International Conference on Knowledge Engineering and Ontology Development - Volume 1: KEOD, (IC3K 2014) ISBN 978-989-758-049-9, pages 56-66. DOI: 10.5220/0005072700560066


in Bibtex Style

@conference{keod14,
author={Hlomani Hlomani and Deborah A. Stacey},
title={Data-driven Diachronic and Categorical Evaluation of Ontologies - Framework, Measure, and Metrics},
booktitle={Proceedings of the International Conference on Knowledge Engineering and Ontology Development - Volume 1: KEOD, (IC3K 2014)},
year={2014},
pages={56-66},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005072700560066},
isbn={978-989-758-049-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Engineering and Ontology Development - Volume 1: KEOD, (IC3K 2014)
TI - Data-driven Diachronic and Categorical Evaluation of Ontologies - Framework, Measure, and Metrics
SN - 978-989-758-049-9
AU - Hlomani H.
AU - A. Stacey D.
PY - 2014
SP - 56
EP - 66
DO - 10.5220/0005072700560066