Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing

Xin Lin; Jianliang Xu; Haibo Hu; Fan Zhe

doi:10.1109/ICDE.2018.00236

Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing

Xin Lin, Jianliang Xu, Haibo Hu, Fan Zhe

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

2 Citations (Scopus)

Abstract

In this paper, we propose a novel pairwise crowd-sourcing model to reduce the uncertainty of top-k ranking using a crowd of domain experts. Given a crowdsourcing task of limited budget, we propose efficient algorithms to select the best object pairs for crowdsourcing that will bring in the highest quality improvement. Extensive experiments show that our proposed solutions outperform a random selection method by up to 30 times in terms of quality improvement of probabilistic top-k ranking queries. In terms of efficiency, our proposed solutions can reduce the elapsed time of a brute-force algorithm from several days to one minute.

Original language	English
Title of host publication	Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018
Publisher	IEEE
Pages	1757-1758
Number of pages	2
ISBN (Electronic)	9781538655207
ISBN (Print)	9781538655214
DOIs	https://doi.org/10.1109/ICDE.2018.00236
Publication status	Published - Apr 2018
Event	34th IEEE International Conference on Data Engineering, ICDE 2018 - Paris, France Duration: 16 Apr 2018 → 19 Apr 2018 https://ieeexplore.ieee.org/xpl/conhome/8476188/proceeding

Publication series

Name	Proceedings - IEEE International Conference on Data Engineering, ICDE

Conference

Conference	34th IEEE International Conference on Data Engineering, ICDE 2018
Country/Territory	France
City	Paris
Period	16/04/18 → 19/04/18
Internet address	https://ieeexplore.ieee.org/xpl/conhome/8476188/proceeding

Scopus Subject Areas

Hardware and Architecture
Information Systems and Management
Information Systems

User-Defined Keywords

Crowdsourcing
Top k
Uncertain query

Access to Document

10.1109/ICDE.2018.00236

Cite this

@inproceedings{7763720a96f44f028cb84206fab1f554,

title = "Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing",

abstract = "In this paper, we propose a novel pairwise crowd-sourcing model to reduce the uncertainty of top-k ranking using a crowd of domain experts. Given a crowdsourcing task of limited budget, we propose efficient algorithms to select the best object pairs for crowdsourcing that will bring in the highest quality improvement. Extensive experiments show that our proposed solutions outperform a random selection method by up to 30 times in terms of quality improvement of probabilistic top-k ranking queries. In terms of efficiency, our proposed solutions can reduce the elapsed time of a brute-force algorithm from several days to one minute.",

keywords = "Crowdsourcing, Top k, Uncertain query",

author = "Xin Lin and Jianliang Xu and Haibo Hu and Fan Zhe",

note = "Funding Information: This work was supported by National Natural Science Foundation of China (Grants: 61773167, 61572413, 61572193, 61502258 and U1636205), NSF of Shanghai (No.: 17ZR1444900), and Research Grants Council, Hong Kong SAR, China, under projects 12200817, 12201615, 12200914, 15238116, and C1008-16G. Publisher Copyright: {\textcopyright} 2018 IEEE.; 34th IEEE International Conference on Data Engineering, ICDE 2018 ; Conference date: 16-04-2018 Through 19-04-2018",

year = "2018",

month = apr,

doi = "10.1109/ICDE.2018.00236",

language = "English",

isbn = "9781538655214",

series = "Proceedings - IEEE International Conference on Data Engineering, ICDE",

publisher = "IEEE",

pages = "1757--1758",

booktitle = "Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/8476188/proceeding",

}

Lin, X, Xu, J, Hu, H & Zhe, F 2018, Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing. in Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. Proceedings - IEEE International Conference on Data Engineering, ICDE, IEEE, pp. 1757-1758, 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, 16/04/18. https://doi.org/10.1109/ICDE.2018.00236

Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing. / Lin, Xin; Xu, Jianliang; Hu, Haibo et al.
Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018. IEEE, 2018. p. 1757-1758 (Proceedings - IEEE International Conference on Data Engineering, ICDE).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing

AU - Lin, Xin

AU - Xu, Jianliang

AU - Hu, Haibo

AU - Zhe, Fan

N1 - Funding Information: This work was supported by National Natural Science Foundation of China (Grants: 61773167, 61572413, 61572193, 61502258 and U1636205), NSF of Shanghai (No.: 17ZR1444900), and Research Grants Council, Hong Kong SAR, China, under projects 12200817, 12201615, 12200914, 15238116, and C1008-16G. Publisher Copyright: © 2018 IEEE.

PY - 2018/4

Y1 - 2018/4

N2 - In this paper, we propose a novel pairwise crowd-sourcing model to reduce the uncertainty of top-k ranking using a crowd of domain experts. Given a crowdsourcing task of limited budget, we propose efficient algorithms to select the best object pairs for crowdsourcing that will bring in the highest quality improvement. Extensive experiments show that our proposed solutions outperform a random selection method by up to 30 times in terms of quality improvement of probabilistic top-k ranking queries. In terms of efficiency, our proposed solutions can reduce the elapsed time of a brute-force algorithm from several days to one minute.

AB - In this paper, we propose a novel pairwise crowd-sourcing model to reduce the uncertainty of top-k ranking using a crowd of domain experts. Given a crowdsourcing task of limited budget, we propose efficient algorithms to select the best object pairs for crowdsourcing that will bring in the highest quality improvement. Extensive experiments show that our proposed solutions outperform a random selection method by up to 30 times in terms of quality improvement of probabilistic top-k ranking queries. In terms of efficiency, our proposed solutions can reduce the elapsed time of a brute-force algorithm from several days to one minute.

KW - Crowdsourcing

KW - Top k

KW - Uncertain query

UR - http://www.scopus.com/inward/record.url?scp=85057116065&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2018.00236

DO - 10.1109/ICDE.2018.00236

M3 - Conference proceeding

AN - SCOPUS:85057116065

SN - 9781538655214

T3 - Proceedings - IEEE International Conference on Data Engineering, ICDE

SP - 1757

EP - 1758

BT - Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018

PB - IEEE

T2 - 34th IEEE International Conference on Data Engineering, ICDE 2018

Y2 - 16 April 2018 through 19 April 2018

ER -

Reducing uncertainty of probabilistic Top-K ranking via Pairwise crowdsourcing

Abstract

Publication series

Conference

Scopus Subject Areas

User-Defined Keywords

Access to Document

Other files and links

Fingerprint

Cite this