Subspace clustering of text documents with feature weighting k-means algorithm

Liping Jing; Michael K. Ng; Jun Xu; Joshua Zhexue Huang

doi:10.1007/11430919_94

Subspace clustering of text documents with feature weighting k-means algorithm

Liping Jing^*, Michael K. Ng, Jun Xu, Joshua Zhexue Huang

^*Corresponding author for this work

Department of Mathematics

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

87 Citations (Scopus)

Abstract

This paper presents a new method to solve the problem of clustering large and complex text data. The method is based on a new subspace clustering algorithm that automatically calculates the feature weights in the k-means clustering process. In clustering sparse text data the feature weights are used to discover clusters from subspaces of the document vector space and identify key words that represent the semantics of the clusters. We present a modification of the published algorithm to solve the sparsity problem that occurs in text clustering. Experimental results on real-world text data have shown that the new method outper-formed the Standard K Means and Bisection-KMeans algorithms, while still maintaining efficiency of the k-means clustering process.

Original language	English
Title of host publication	Advances in Knowledge Discovery and Data Mining
Subtitle of host publication	9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings
Editors	Tu Bao Ho, David Cheung, Huan Liu
Publisher	Springer Berlin Heidelberg
Pages	802-812
Number of pages	11
Edition	1st
ISBN (Electronic)	9783540319351
ISBN (Print)	3540260765, 9783540260769
DOIs	https://doi.org/10.1007/11430919_94
Publication status	Published - 10 May 2005
Event	9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2005 - Melia Hanoi Hotel, Hanoi, Viet Nam Duration: 18 May 2005 → 20 May 2005 https://www.jaist.ac.jp/PAKDD-05/ (conference website) https://www.jaist.ac.jp/PAKDD-05/ (conference program)

Publication series

Name	Lecture Notes in Computer Science
Volume	3518
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Name	Lecture Notes in Artificial Intelligence
Publisher	Springer
ISSN (Print)	2945-9133
ISSN (Electronic)	2945-9141

Name	PAKDD: Pacific-Asia Conference on Knowledge Discovery and Data Mining

Conference

Conference	9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2005
Country/Territory	Viet Nam
City	Hanoi
Period	18/05/05 → 20/05/05
Internet address	https://www.jaist.ac.jp/PAKDD-05/ (conference website) https://www.jaist.ac.jp/PAKDD-05/ (conference program)

Scopus Subject Areas

Theoretical Computer Science
General Computer Science

User-Defined Keywords

Cluster Interpretation
Feature Weighting
High Dimensional Data
Subspace Clustering
Text Mining

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/11430919_94

Cite this

Jing, L., Ng, M. K., Xu, J., & Huang, J. Z. (2005). Subspace clustering of text documents with feature weighting k-means algorithm. In T. B. Ho, D. Cheung, & H. Liu (Eds.), Advances in Knowledge Discovery and Data Mining: 9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings (1st ed., pp. 802-812). (Lecture Notes in Computer Science; Vol. 3518), (Lecture Notes in Artificial Intelligence), (PAKDD: Pacific-Asia Conference on Knowledge Discovery and Data Mining). Springer Berlin Heidelberg. https://doi.org/10.1007/11430919_94

Jing, Liping ; Ng, Michael K. ; Xu, Jun et al. / Subspace clustering of text documents with feature weighting k-means algorithm. Advances in Knowledge Discovery and Data Mining: 9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings. editor / Tu Bao Ho ; David Cheung ; Huan Liu. 1st. ed. Springer Berlin Heidelberg, 2005. pp. 802-812 (Lecture Notes in Computer Science). (Lecture Notes in Artificial Intelligence). (PAKDD: Pacific-Asia Conference on Knowledge Discovery and Data Mining).

@inproceedings{03ad6907a84d43a19698e6d41bb95723,

title = "Subspace clustering of text documents with feature weighting k-means algorithm",

abstract = "This paper presents a new method to solve the problem of clustering large and complex text data. The method is based on a new subspace clustering algorithm that automatically calculates the feature weights in the k-means clustering process. In clustering sparse text data the feature weights are used to discover clusters from subspaces of the document vector space and identify key words that represent the semantics of the clusters. We present a modification of the published algorithm to solve the sparsity problem that occurs in text clustering. Experimental results on real-world text data have shown that the new method outper-formed the Standard K Means and Bisection-KMeans algorithms, while still maintaining efficiency of the k-means clustering process.",

keywords = "Cluster Interpretation, Feature Weighting, High Dimensional Data, Subspace Clustering, Text Mining",

author = "Liping Jing and Ng, {Michael K.} and Jun Xu and Huang, {Joshua Zhexue}",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2005; 9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2005 ; Conference date: 18-05-2005 Through 20-05-2005",

year = "2005",

month = may,

day = "10",

doi = "10.1007/11430919_94",

language = "English",

isbn = "3540260765",

series = "Lecture Notes in Computer Science",

publisher = "Springer Berlin Heidelberg",

pages = "802--812",

editor = "Ho, {Tu Bao} and David Cheung and Huan Liu",

booktitle = "Advances in Knowledge Discovery and Data Mining",

edition = "1st",

url = "https://www.jaist.ac.jp/PAKDD-05/, https://www.jaist.ac.jp/PAKDD-05/",

}

Jing, L, Ng, MK, Xu, J & Huang, JZ 2005, Subspace clustering of text documents with feature weighting k-means algorithm. in TB Ho, D Cheung & H Liu (eds), Advances in Knowledge Discovery and Data Mining: 9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings. 1st edn, Lecture Notes in Computer Science, vol. 3518, Lecture Notes in Artificial Intelligence, PAKDD: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer Berlin Heidelberg, pp. 802-812, 9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2005, Hanoi, Viet Nam, 18/05/05. https://doi.org/10.1007/11430919_94

Subspace clustering of text documents with feature weighting k-means algorithm. / Jing, Liping; Ng, Michael K.; Xu, Jun et al.
Advances in Knowledge Discovery and Data Mining: 9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings. ed. / Tu Bao Ho; David Cheung; Huan Liu. 1st. ed. Springer Berlin Heidelberg, 2005. p. 802-812 (Lecture Notes in Computer Science; Vol. 3518), (Lecture Notes in Artificial Intelligence), (PAKDD: Pacific-Asia Conference on Knowledge Discovery and Data Mining).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Subspace clustering of text documents with feature weighting k-means algorithm

AU - Jing, Liping

AU - Ng, Michael K.

AU - Xu, Jun

AU - Huang, Joshua Zhexue

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2005

PY - 2005/5/10

Y1 - 2005/5/10

N2 - This paper presents a new method to solve the problem of clustering large and complex text data. The method is based on a new subspace clustering algorithm that automatically calculates the feature weights in the k-means clustering process. In clustering sparse text data the feature weights are used to discover clusters from subspaces of the document vector space and identify key words that represent the semantics of the clusters. We present a modification of the published algorithm to solve the sparsity problem that occurs in text clustering. Experimental results on real-world text data have shown that the new method outper-formed the Standard K Means and Bisection-KMeans algorithms, while still maintaining efficiency of the k-means clustering process.

AB - This paper presents a new method to solve the problem of clustering large and complex text data. The method is based on a new subspace clustering algorithm that automatically calculates the feature weights in the k-means clustering process. In clustering sparse text data the feature weights are used to discover clusters from subspaces of the document vector space and identify key words that represent the semantics of the clusters. We present a modification of the published algorithm to solve the sparsity problem that occurs in text clustering. Experimental results on real-world text data have shown that the new method outper-formed the Standard K Means and Bisection-KMeans algorithms, while still maintaining efficiency of the k-means clustering process.

KW - Cluster Interpretation

KW - Feature Weighting

KW - High Dimensional Data

KW - Subspace Clustering

KW - Text Mining

UR - http://www.scopus.com/inward/record.url?scp=26944481948&partnerID=8YFLogxK

UR - https://link.springer.com/chapter/10.1007/11430919_94#Abs1

U2 - 10.1007/11430919_94

DO - 10.1007/11430919_94

M3 - Conference proceeding

AN - SCOPUS:26944481948

SN - 3540260765

SN - 9783540260769

T3 - Lecture Notes in Computer Science

SP - 802

EP - 812

BT - Advances in Knowledge Discovery and Data Mining

A2 - Ho, Tu Bao

A2 - Cheung, David

A2 - Liu, Huan

PB - Springer Berlin Heidelberg

T2 - 9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2005

Y2 - 18 May 2005 through 20 May 2005

ER -

Jing L, Ng MK, Xu J, Huang JZ. Subspace clustering of text documents with feature weighting k-means algorithm. In Ho TB, Cheung D, Liu H, editors, Advances in Knowledge Discovery and Data Mining: 9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings. 1st ed. Springer Berlin Heidelberg. 2005. p. 802-812. (Lecture Notes in Computer Science). (Lecture Notes in Artificial Intelligence). (PAKDD: Pacific-Asia Conference on Knowledge Discovery and Data Mining). doi: 10.1007/11430919_94

Subspace clustering of text documents with feature weighting k-means algorithm

Abstract

Publication series

Conference

Scopus Subject Areas

User-Defined Keywords

UN SDGs

Access to Document

Other files and links

Cite this