ESet: Placing data towards efficient recovery for large-scale erasure-coded storage systems

Chengjian Liu, Xiaowen CHU, Hai Liu, Yiu Wing LEUNG

Research output: Chapter in book/report/conference proceedingConference contributionpeer-review

4 Citations (Scopus)

Abstract

Erasure coding has been extensively deployed in distributed storage systems to ensure high reliability and low storage overhead. However, erasure coding requires much more disk I/O to recover a damaged data block than replication does, resulting in very long data recovery time. Data placement algorithm can be tailored to speed up data recovery process by exploiting I/O parallelism. However, existing algorithms that obtain good I/O parallelism for replication can not directly work with erasure-coded storage systems; and other algorithms for both replication based and erasure-coded storage systems overlook the importance of recovery I/O parallelism, which may jeopardize the service quality and reliability of these systems. In this paper, we present a data placement strategy named ESet which brings recovery efficiency for each host in a distributed storage system. We define a configurable parameter named overlapping factor for system administrator to easily achieve desirable recovery I/O parallelism. Our simulation results show that ESet can significantly improve the data recovery performance without violating the reliability requirement by distributing data and code blocks across different failure domains.

Original languageEnglish
Title of host publication2016 25th International Conference on Computer Communications and Networks, ICCCN 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509022793
DOIs
Publication statusPublished - 14 Sep 2016
Event25th International Conference on Computer Communications and Networks, ICCCN 2016 - Waikoloa, United States
Duration: 1 Aug 20164 Aug 2016

Publication series

Name2016 25th International Conference on Computer Communications and Networks, ICCCN 2016

Conference

Conference25th International Conference on Computer Communications and Networks, ICCCN 2016
Country/TerritoryUnited States
CityWaikoloa
Period1/08/164/08/16

Scopus Subject Areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'ESet: Placing data towards efficient recovery for large-scale erasure-coded storage systems'. Together they form a unique fingerprint.

Cite this