Abstract
In this paper we present DtCraft, a distributed execution engine that enables a new powerful programming model to streamline cluster computing. Applications are described in a set of data-parallel streams, leaving difficult execution details and concurrency controls handled by our system kernel transparently. Compared with existing systems, DtCraft is unique in (1) an efficient stream-oriented programming paradigm using modern C++17, (2) an in-context resource controller and task executor based on Linux container technology, and (3) ease of development from prototyping machines to production cloud environments. These capabilities power industry applications and create new research directions in machine learning, stream processing, and distributed multimedia systems.
Original language | English |
---|---|
Title of host publication | MM 2018 - Proceedings of the 2018 ACM Multimedia Conference |
Publisher | Association for Computing Machinery (ACM) |
Pages | 1360–1363 |
Number of pages | 4 |
ISBN (Print) | 9781450356657 |
DOIs | |
Publication status | Published - Oct 2018 |
Event | 26th ACM Multimedia conference, MM 2018 - Seoul, Korea, Republic of Duration: 22 Oct 2018 → 26 Oct 2018 https://dl.acm.org/doi/proceedings/10.1145/3240508 (Link to conference proceedings) |
Publication series
Name | Proceedings of ACM international conference on Multimedia |
---|
Conference
Conference | 26th ACM Multimedia conference, MM 2018 |
---|---|
Country/Territory | Korea, Republic of |
City | Seoul |
Period | 22/10/18 → 26/10/18 |
Internet address |
|
User-Defined Keywords
- Distributed System
- Stream Processing
- Machine Learning