DyVer: Dynamic Version Handling for Array Databases

Amelie Chi Zhou, Zhoubin Ke, Jianming Lao

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

Abstract

Array databases are important data management systems for scientific applications. In array databases, version handling is an important problem due to the no-overwrite feature of scientific data. Existing studies for optimizing data versioning in array databases are relatively simple, which either focus on minimizing storage sizes or improving simple version chains. In this paper, we focus on two challenges: (1) how to balance the tradeoff between storage size and query time for numerous version data, which may have derivative relationships with each other; (2) how to dynamically maintain this balance with continuously added new versions. To address the above challenges, this paper presents DyVer, a versioning framework for SciDB which is one of the most well-known array databases. DyVer includes two techniques, including an efficient storage layout optimizer to quickly reduce data query time under storage capacity constraint and a version segment technique to cope with dynamic version additions. We evaluate DyVer using real-world scientific datasets. Results show that DyVer can achieve up to 95% improvement on the average query time compared to state-of-the-art data versioning techniques under the same storage capacity constraint.

Original languageEnglish
Title of host publicationICS 2023 - Proceedings of the 37th International Conference on Supercomputing
PublisherAssociation for Computing Machinery (ACM)
Pages144-154
Number of pages11
Edition1st
ISBN (Electronic)9798400700569
DOIs
Publication statusPublished - 21 Jun 2023
Event37th ACM International Conference on Supercomputing, ICS 2023 - Orlando, United States
Duration: 21 Jun 202323 Jun 2023
https://dl.acm.org/doi/proceedings/10.1145/3577193

Publication series

NameProceedings of the International Conference on Supercomputing, ICS
PublisherAssociation for Computing Machinery

Conference

Conference37th ACM International Conference on Supercomputing, ICS 2023
Country/TerritoryUnited States
CityOrlando
Period21/06/2323/06/23
Internet address

Scopus Subject Areas

  • Computer Science(all)

User-Defined Keywords

  • scientific data management
  • array database
  • versioning

Fingerprint

Dive into the research topics of 'DyVer: Dynamic Version Handling for Array Databases'. Together they form a unique fingerprint.

Cite this