Distance-based k-nearest neighbors outlier detection method in large-scale traffic data

Taurus T. Dang; Henry Y.T. Ngan; Wei Liu

doi:10.1109/ICDSP.2015.7251924

Distance-based k-nearest neighbors outlier detection method in large-scale traffic data

Taurus T. Dang
, Henry Y.T. Ngan
, Wei Liu

Department of Mathematics

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

108 Citations (Scopus)

Abstract

This paper presents a k-nearest neighbors (kNN) method to detect outliers in large-scale traffic data collected daily in every modern city. Outliers include hardware and data errors as well as abnormal traffic behaviors. The proposed kNN method detects outliers by exploiting the relationship among neighborhoods in data points. The farther a data point is beyond its neighbors, the more possible the data is an outlier. Traffic data here was recorded in a video format, and converted to spatial-temporal (ST) traffic signals by statistics. The ST signals are then transformed to a two-dimensional (2D) (x, y) -coordinate plane by Principal Component Analysis (PCA) for dimension reduction. The distance-based kNN method is evaluated by unsupervised and semi-supervised approaches. The semi-supervised approach reaches 96.19% accuracy.

Original language	English
Title of host publication	2015 IEEE International Conference on Digital Signal Processing, DSP 2015
Publisher	IEEE
Pages	507-510
Number of pages	4
ISBN (Electronic)	9781479980581, 9781479980581
DOIs	https://doi.org/10.1109/ICDSP.2015.7251924
Publication status	Published - 9 Sept 2015
Event	IEEE International Conference on Digital Signal Processing, DSP 2015 - Singapore, Singapore Duration: 21 Jul 2015 → 24 Jul 2015

Publication series

Name	International Conference on Digital Signal Processing, DSP
Volume	2015-September

Conference

Conference	IEEE International Conference on Digital Signal Processing, DSP 2015
Country/Territory	Singapore
City	Singapore
Period	21/07/15 → 24/07/15

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 11 Sustainable Cities and Communities

User-Defined Keywords

distance-based
kNN
large-scale
Outlier detection
traffic data

Access to Document

10.1109/ICDSP.2015.7251924

Cite this

Dang, T. T., Ngan, H. Y. T., & Liu, W. (2015). Distance-based k-nearest neighbors outlier detection method in large-scale traffic data. In 2015 IEEE International Conference on Digital Signal Processing, DSP 2015 (pp. 507-510). Article 7251924 (International Conference on Digital Signal Processing, DSP; Vol. 2015-September). IEEE. https://doi.org/10.1109/ICDSP.2015.7251924

@inproceedings{2c0512c854d04485b034fdd4cc7b1656,

title = "Distance-based k-nearest neighbors outlier detection method in large-scale traffic data",

abstract = "This paper presents a k-nearest neighbors (kNN) method to detect outliers in large-scale traffic data collected daily in every modern city. Outliers include hardware and data errors as well as abnormal traffic behaviors. The proposed kNN method detects outliers by exploiting the relationship among neighborhoods in data points. The farther a data point is beyond its neighbors, the more possible the data is an outlier. Traffic data here was recorded in a video format, and converted to spatial-temporal (ST) traffic signals by statistics. The ST signals are then transformed to a two-dimensional (2D) (x, y) -coordinate plane by Principal Component Analysis (PCA) for dimension reduction. The distance-based kNN method is evaluated by unsupervised and semi-supervised approaches. The semi-supervised approach reaches 96.19\% accuracy.",

keywords = "distance-based, kNN, large-scale, Outlier detection, traffic data",

author = "Dang, \{Taurus T.\} and Ngan, \{Henry Y.T.\} and Wei Liu",

year = "2015",

month = sep,

day = "9",

doi = "10.1109/ICDSP.2015.7251924",

language = "English",

series = "International Conference on Digital Signal Processing, DSP",

publisher = "IEEE",

pages = "507--510",

booktitle = "2015 IEEE International Conference on Digital Signal Processing, DSP 2015",

address = "United States",

note = "IEEE International Conference on Digital Signal Processing, DSP 2015 ; Conference date: 21-07-2015 Through 24-07-2015",

}

Dang, TT, Ngan, HYT & Liu, W 2015, Distance-based k-nearest neighbors outlier detection method in large-scale traffic data. in 2015 IEEE International Conference on Digital Signal Processing, DSP 2015., 7251924, International Conference on Digital Signal Processing, DSP, vol. 2015-September, IEEE, pp. 507-510, IEEE International Conference on Digital Signal Processing, DSP 2015, Singapore, Singapore, 21/07/15. https://doi.org/10.1109/ICDSP.2015.7251924

Distance-based k-nearest neighbors outlier detection method in large-scale traffic data. / Dang, Taurus T.; Ngan, Henry Y.T.; Liu, Wei.
2015 IEEE International Conference on Digital Signal Processing, DSP 2015. IEEE, 2015. p. 507-510 7251924 (International Conference on Digital Signal Processing, DSP; Vol. 2015-September).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Distance-based k-nearest neighbors outlier detection method in large-scale traffic data

AU - Dang, Taurus T.

AU - Ngan, Henry Y.T.

AU - Liu, Wei

PY - 2015/9/9

Y1 - 2015/9/9

N2 - This paper presents a k-nearest neighbors (kNN) method to detect outliers in large-scale traffic data collected daily in every modern city. Outliers include hardware and data errors as well as abnormal traffic behaviors. The proposed kNN method detects outliers by exploiting the relationship among neighborhoods in data points. The farther a data point is beyond its neighbors, the more possible the data is an outlier. Traffic data here was recorded in a video format, and converted to spatial-temporal (ST) traffic signals by statistics. The ST signals are then transformed to a two-dimensional (2D) (x, y) -coordinate plane by Principal Component Analysis (PCA) for dimension reduction. The distance-based kNN method is evaluated by unsupervised and semi-supervised approaches. The semi-supervised approach reaches 96.19% accuracy.

AB - This paper presents a k-nearest neighbors (kNN) method to detect outliers in large-scale traffic data collected daily in every modern city. Outliers include hardware and data errors as well as abnormal traffic behaviors. The proposed kNN method detects outliers by exploiting the relationship among neighborhoods in data points. The farther a data point is beyond its neighbors, the more possible the data is an outlier. Traffic data here was recorded in a video format, and converted to spatial-temporal (ST) traffic signals by statistics. The ST signals are then transformed to a two-dimensional (2D) (x, y) -coordinate plane by Principal Component Analysis (PCA) for dimension reduction. The distance-based kNN method is evaluated by unsupervised and semi-supervised approaches. The semi-supervised approach reaches 96.19% accuracy.

KW - distance-based

KW - kNN

KW - large-scale

KW - Outlier detection

KW - traffic data

UR - https://www.scopus.com/pages/publications/84961298270

U2 - 10.1109/ICDSP.2015.7251924

DO - 10.1109/ICDSP.2015.7251924

M3 - Conference proceeding

AN - SCOPUS:84961298270

T3 - International Conference on Digital Signal Processing, DSP

SP - 507

EP - 510

BT - 2015 IEEE International Conference on Digital Signal Processing, DSP 2015

PB - IEEE

T2 - IEEE International Conference on Digital Signal Processing, DSP 2015

Y2 - 21 July 2015 through 24 July 2015

ER -

Distance-based k-nearest neighbors outlier detection method in large-scale traffic data

Abstract

Publication series

Conference

UN SDGs

User-Defined Keywords

Access to Document

Other files and links

Fingerprint

Cite this