CeDMA: Enhancing Memory Efficiency of Heterogeneous Accelerator Systems Through Central DMA Controlling

As heterogeneous computing systems continue to evolve, emerging workloads increasingly span multiple types of accelerators, resulting in frequent inter-accelerator data transfers. However, traditional CPU-managed memory systems often struggle to coordinate these transfers efficiently, leading to high latency, poor memory bandwidth utilization, and scalability bottlenecks. We propose CeDMA, a centralized and programmable Direct Memory Access (DMA) control architecture that enables high-performance, CPU-decoupled memory coordination across diverse accelerators. CeDMA combines a unified hardware-software co-design: a modular DMA engine with integrated address translation and dual-level arbitration logic on the hardware side, and a lightweight instruction-driven memory management model with adaptive scheduling on the software side. CeDMA enables fine-grained control over memory transfers, minimizes off-chip bandwidth consumption, and exploits memory-level parallelism through dynamic resource partitioning. Cycle-accurate simulation results across a diverse workload suite—including GEMM, Conv2D, and graph traversal kernels—demonstrate up to 75% reduction in external memory access, 60% improvement in performance, and 45% reduction in access latency. Furthermore, CeDMA maintains high throughput and predictable latency at scale, supporting up to 32 concurrent accelerators. These results position CeDMA as a scalable, general-purpose memory management substrate for future heterogeneous SoC architectures.