A Methodology for Ontology Evaluation Using Topic Models

Abstract

The purpose of this paper is to describe a methodology for objectively evaluating ontologies. Our approach involves randomly partitioning the elements of an ontology into disjoints training and test set respectively, generating topic models on the training set, and evaluating how well the model fits the test set. We have tested our methodology on the Translational Medicine Ontology and collected extensive experimental results. The results include the average perplexity score for the entire ontology as well as those for individual elements. Since our methodology provides a numeric score for an ontology it can be used to compare ontologies. Furthermore, elements with high perplexity scores might indicate that either these do not fit well with the rest of the ontology, or that the descriptions for these elements are inadequate. Different perplexity scores among sibling elements indicate the need to revise the structure of the ontology.