BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures

Shuhao Zhang; Jiong He; Amelie Chi Zhou; Bingsheng He

doi:10.1145/3299869.3300067

BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures

Shuhao Zhang
, Jiong He
, Amelie Chi Zhou
, Bingsheng He

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

42 Citations (Scopus)

Abstract

We introduce BriskStream, an in-memory data stream processing system (DSPSs) specifically designed for modern shared-memory multicore architectures. BriskStream's key contribution is an execution plan optimization paradigm, namely RLAS, which takes relative-location (i.e., NUMA distance) of each pair of producer-consumer operators into consideration. We propose a branch and bound based approach with three heuristics to resolve the resulting nontrivial optimization problem. The experimental evaluations demonstrate that BriskStream yields much higher throughput and better scalability than existing DSPSs on multi-core architectures when processing different types of workloads.

Original language	English
Title of host publication	SIGMOD '19
Subtitle of host publication	Proceedings of the 2019 International Conference on Management of Data
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	705-722
Number of pages	18
ISBN (Electronic)	9781450356435
DOIs	https://doi.org/10.1145/3299869.3300067
Publication status	Published - 25 Jun 2019
Event	2019 International Conference on Management of Data, SIGMOD 2019 - Amsterdam, Netherlands Duration: 30 Jun 2019 → 5 Jul 2019 https://dl.acm.org/doi/proceedings/10.1145/3299869 (Conference Proceedings)

Publication series

Name	Proceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)	0730-8078

Conference

Conference	2019 International Conference on Management of Data, SIGMOD 2019
Country/Territory	Netherlands
City	Amsterdam
Period	30/06/19 → 5/07/19
Internet address	https://dl.acm.org/doi/proceedings/10.1145/3299869 (Conference Proceedings)

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 9 Industry, Innovation, and Infrastructure

Access to Document

10.1145/3299869.3300067

Cite this

Zhang, S., He, J., Zhou, A. C., & He, B. (2019). BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures. In SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data (pp. 705-722). (Proceedings of the ACM SIGMOD International Conference on Management of Data). Association for Computing Machinery (ACM). https://doi.org/10.1145/3299869.3300067

Zhang, Shuhao ; He, Jiong ; Zhou, Amelie Chi et al. / BriskStream : Scaling Data Stream Processing on Shared-Memory Multicore Architectures. SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data. New York : Association for Computing Machinery (ACM), 2019. pp. 705-722 (Proceedings of the ACM SIGMOD International Conference on Management of Data).

@inproceedings{841ace195e8f4ef0bbdc65000c1a3356,

title = "BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures",

abstract = "We introduce BriskStream, an in-memory data stream processing system (DSPSs) specifically designed for modern shared-memory multicore architectures. BriskStream's key contribution is an execution plan optimization paradigm, namely RLAS, which takes relative-location (i.e., NUMA distance) of each pair of producer-consumer operators into consideration. We propose a branch and bound based approach with three heuristics to resolve the resulting nontrivial optimization problem. The experimental evaluations demonstrate that BriskStream yields much higher throughput and better scalability than existing DSPSs on multi-core architectures when processing different types of workloads.",

author = "Shuhao Zhang and Jiong He and Zhou, \{Amelie Chi\} and Bingsheng He",

note = "Publisher Copyright: {\textcopyright} 2019 Association for Computing Machinery. Funding Information: The authors would like to thank the anonymous reviewers for their valuable comments. This work is supported by a MoE AcRF Tier 2 grant (MOE2017-T2-1-122) and an NUS startup grant in Singapore. Jiong He is supported by the National Research Foundation, Prime Ministers Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme. Chi Zhou{\textquoteright}s work is partially supported by the National Natural Science Foundation of China under Grant 61802260 and the Guangdong Natural Science Foundation under Grant 2018A030310440.; 2019 International Conference on Management of Data, SIGMOD 2019 ; Conference date: 30-06-2019 Through 05-07-2019",

year = "2019",

month = jun,

day = "25",

doi = "10.1145/3299869.3300067",

language = "English",

series = "Proceedings of the ACM SIGMOD International Conference on Management of Data",

publisher = "Association for Computing Machinery (ACM)",

pages = "705--722",

booktitle = "SIGMOD '19",

address = "United States",

url = "https://dl.acm.org/doi/proceedings/10.1145/3299869",

}

Zhang, S, He, J, Zhou, AC & He, B 2019, BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures. in SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data. Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery (ACM), New York, pp. 705-722, 2019 International Conference on Management of Data, SIGMOD 2019, Amsterdam, Netherlands, 30/06/19. https://doi.org/10.1145/3299869.3300067

BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures. / Zhang, Shuhao; He, Jiong; Zhou, Amelie Chi et al.
SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data. New York: Association for Computing Machinery (ACM), 2019. p. 705-722 (Proceedings of the ACM SIGMOD International Conference on Management of Data).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - BriskStream

T2 - 2019 International Conference on Management of Data, SIGMOD 2019

AU - Zhang, Shuhao

AU - He, Jiong

AU - Zhou, Amelie Chi

AU - He, Bingsheng

N1 - Publisher Copyright: © 2019 Association for Computing Machinery. Funding Information: The authors would like to thank the anonymous reviewers for their valuable comments. This work is supported by a MoE AcRF Tier 2 grant (MOE2017-T2-1-122) and an NUS startup grant in Singapore. Jiong He is supported by the National Research Foundation, Prime Ministers Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme. Chi Zhou’s work is partially supported by the National Natural Science Foundation of China under Grant 61802260 and the Guangdong Natural Science Foundation under Grant 2018A030310440.

PY - 2019/6/25

Y1 - 2019/6/25

N2 - We introduce BriskStream, an in-memory data stream processing system (DSPSs) specifically designed for modern shared-memory multicore architectures. BriskStream's key contribution is an execution plan optimization paradigm, namely RLAS, which takes relative-location (i.e., NUMA distance) of each pair of producer-consumer operators into consideration. We propose a branch and bound based approach with three heuristics to resolve the resulting nontrivial optimization problem. The experimental evaluations demonstrate that BriskStream yields much higher throughput and better scalability than existing DSPSs on multi-core architectures when processing different types of workloads.

AB - We introduce BriskStream, an in-memory data stream processing system (DSPSs) specifically designed for modern shared-memory multicore architectures. BriskStream's key contribution is an execution plan optimization paradigm, namely RLAS, which takes relative-location (i.e., NUMA distance) of each pair of producer-consumer operators into consideration. We propose a branch and bound based approach with three heuristics to resolve the resulting nontrivial optimization problem. The experimental evaluations demonstrate that BriskStream yields much higher throughput and better scalability than existing DSPSs on multi-core architectures when processing different types of workloads.

UR - https://www.scopus.com/pages/publications/85069524668

U2 - 10.1145/3299869.3300067

DO - 10.1145/3299869.3300067

M3 - Conference proceeding

AN - SCOPUS:85069524668

T3 - Proceedings of the ACM SIGMOD International Conference on Management of Data

SP - 705

EP - 722

BT - SIGMOD '19

PB - Association for Computing Machinery (ACM)

CY - New York

Y2 - 30 June 2019 through 5 July 2019

ER -

Zhang S, He J, Zhou AC, He B. BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures. In SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data. New York: Association for Computing Machinery (ACM). 2019. p. 705-722. (Proceedings of the ACM SIGMOD International Conference on Management of Data). doi: 10.1145/3299869.3300067

BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures

Abstract

Publication series

Conference

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this