SparkSQL+: Next-generation Query Planning over Spark

Binyang Dai; Qichen Wang; Ke Yi

doi:10.1145/3555041.3589715

SparkSQL+: Next-generation Query Planning over Spark

Binyang Dai, Qichen Wang, Ke Yi

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

Abstract

We will demonstrate SparkSQL+, a SQL processing engine built on top of Spark. Unlike the vanilla SparkSQL that uses classical query plans, SparkSQL+ adopts some of the recently developed new query plans, including generalized hypertree decompositions (GHD), worst-case optimal join (WCOJ) algorithms, and conjunctive queries with comparisons (CQC). SparkSQL+ also provides a platform for users to explore different query plans for a given query through a web-based interface, and compare their performance with classical query plans on the same Spark core.

Original language	English
Title of host publication	SIGMOD/PODS '23: Companion of the 2023 International Conference on Management of Data
Publisher	Association for Computing Machinery (ACM)
Pages	115–118
Number of pages	4
ISBN (Electronic)	9781450395076
ISBN (Print)	9781450395076
DOIs	https://doi.org/10.1145/3555041.3589715
Publication status	Published - 5 Jun 2023
Event	ACM SIGMOD International Conference on Management of Data, SIGMOD/PODS 2023 - Seattle, United States Duration: 18 Jun 2023 → 23 Jun 2023 https://2023.sigmod.org/ https://dl.acm.org/doi/proceedings/10.1145/3555041

Publication series

Name	Proceedings of the ACM SIGMOD International Conference on Management of Data
ISSN (Print)	0730-8078

Conference

Conference	ACM SIGMOD International Conference on Management of Data, SIGMOD/PODS 2023
Country/Territory	United States
City	Seattle
Period	18/06/23 → 23/06/23
Internet address	https://2023.sigmod.org/ https://dl.acm.org/doi/proceedings/10.1145/3555041

Scopus Subject Areas

Software
Information Systems

User-Defined Keywords

acyclic joins
compilation
conjunctive query
visualization

Access to Document

10.1145/3555041.3589715

Cite this

@inproceedings{307be172810c468992106bc821b3a3e7,

title = "SparkSQL+: Next-generation Query Planning over Spark",

abstract = "We will demonstrate SparkSQL+, a SQL processing engine built on top of Spark. Unlike the vanilla SparkSQL that uses classical query plans, SparkSQL+ adopts some of the recently developed new query plans, including generalized hypertree decompositions (GHD), worst-case optimal join (WCOJ) algorithms, and conjunctive queries with comparisons (CQC). SparkSQL+ also provides a platform for users to explore different query plans for a given query through a web-based interface, and compare their performance with classical query plans on the same Spark core.",

keywords = "acyclic joins, compilation, conjunctive query, visualization",

author = "Binyang Dai and Qichen Wang and Ke Yi",

note = "Funding Information: ACKNOWLEDGMENTS This work has been supported by HKRGC under grants 16201819, 16205420, 16209821, and 16205422. REFERENCES Publisher Copyright: {\textcopyright} 2023 ACM.; ACM SIGMOD International Conference on Management of Data, SIGMOD/PODS 2023 ; Conference date: 18-06-2023 Through 23-06-2023",

year = "2023",

month = jun,

day = "5",

doi = "10.1145/3555041.3589715",

language = "English",

isbn = "9781450395076",

series = "Proceedings of the ACM SIGMOD International Conference on Management of Data",

publisher = "Association for Computing Machinery (ACM)",

pages = "115–118",

booktitle = "SIGMOD/PODS '23: Companion of the 2023 International Conference on Management of Data",

address = "United States",

url = "https://2023.sigmod.org/, https://dl.acm.org/doi/proceedings/10.1145/3555041",

}

Dai, B, Wang, Q & Yi, K 2023, SparkSQL+: Next-generation Query Planning over Spark. in SIGMOD/PODS '23: Companion of the 2023 International Conference on Management of Data. Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery (ACM), pp. 115–118, ACM SIGMOD International Conference on Management of Data, SIGMOD/PODS 2023, Seattle, Washington, United States, 18/06/23. https://doi.org/10.1145/3555041.3589715

SparkSQL+: Next-generation Query Planning over Spark. / Dai, Binyang; Wang, Qichen; Yi, Ke.
SIGMOD/PODS '23: Companion of the 2023 International Conference on Management of Data. Association for Computing Machinery (ACM), 2023. p. 115–118 (Proceedings of the ACM SIGMOD International Conference on Management of Data).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - SparkSQL+: Next-generation Query Planning over Spark

AU - Dai, Binyang

AU - Wang, Qichen

AU - Yi, Ke

PY - 2023/6/5

Y1 - 2023/6/5

N2 - We will demonstrate SparkSQL+, a SQL processing engine built on top of Spark. Unlike the vanilla SparkSQL that uses classical query plans, SparkSQL+ adopts some of the recently developed new query plans, including generalized hypertree decompositions (GHD), worst-case optimal join (WCOJ) algorithms, and conjunctive queries with comparisons (CQC). SparkSQL+ also provides a platform for users to explore different query plans for a given query through a web-based interface, and compare their performance with classical query plans on the same Spark core.

AB - We will demonstrate SparkSQL+, a SQL processing engine built on top of Spark. Unlike the vanilla SparkSQL that uses classical query plans, SparkSQL+ adopts some of the recently developed new query plans, including generalized hypertree decompositions (GHD), worst-case optimal join (WCOJ) algorithms, and conjunctive queries with comparisons (CQC). SparkSQL+ also provides a platform for users to explore different query plans for a given query through a web-based interface, and compare their performance with classical query plans on the same Spark core.

KW - acyclic joins

KW - compilation

KW - conjunctive query

KW - visualization

UR - http://www.scopus.com/inward/record.url?scp=85162921292&partnerID=8YFLogxK

U2 - 10.1145/3555041.3589715

DO - 10.1145/3555041.3589715

M3 - Conference proceeding

SN - 9781450395076

T3 - Proceedings of the ACM SIGMOD International Conference on Management of Data

SP - 115

EP - 118

BT - SIGMOD/PODS '23: Companion of the 2023 International Conference on Management of Data

PB - Association for Computing Machinery (ACM)

T2 - ACM SIGMOD International Conference on Management of Data, SIGMOD/PODS 2023

Y2 - 18 June 2023 through 23 June 2023

ER -

SparkSQL+: Next-generation Query Planning over Spark

Abstract

Publication series

Conference

Scopus Subject Areas

User-Defined Keywords

Access to Document

Other files and links

Fingerprint

Cite this