Knowledge-based vector space model for text clustering

Liping Jing*, Kwok Po NG, Joshua Z. Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

62 Citations (Scopus)

Abstract

This paper presents a new knowledge-based vector space model (VSM) for text clustering. In the new model, semantic relationships between terms (e.g., words or concepts) are included in representing text documents as a set of vectors. The idea is to calculate the dissimilarity between two documents more effectively so that text clustering results can be enhanced. In this paper, the semantic relationship between two terms is defined by the similarity of the two terms. Such similarity is used to re-weight term frequency in the VSM. We consider and study two different similarity measures for computing the semantic relationship between two terms based on two different approaches. The first approach is based on the existing ontologies like WordNet and MeSH. We define a new similarity measure that combines the edge-counting technique, the average distance and the position weighting method to compute the similarity of two terms from an ontology hierarchy. The second approach is to make use of text corpora to construct the relationships between terms and then calculate their semantic similarities. Three clustering algorithms, bisecting k-means, feature weighting k-means and a hierarchical clustering algorithm, have been used to cluster real-world text data represented in the new knowledge-based VSM. The experimental results show that the clustering performance based on the new model was much better than that based on the traditional term-based VSM.

Original languageEnglish
Pages (from-to)35-55
Number of pages21
JournalKnowledge and Information Systems
Volume25
Issue number1
DOIs
Publication statusPublished - 2010

Scopus Subject Areas

  • Software
  • Information Systems
  • Human-Computer Interaction
  • Hardware and Architecture
  • Artificial Intelligence

User-Defined Keywords

  • Knowledge-based VSM
  • Semantic relationship
  • Term similarity
  • Text clustering

Fingerprint

Dive into the research topics of 'Knowledge-based vector space model for text clustering'. Together they form a unique fingerprint.

Cite this