GPAQuant: Accelerating Personalized Recommendation with Group-Wise and PIM-Aware Quantization

Deep Learning Recommendation Models (DLRMs) have gained popularity in recommendation systems due to their effectiveness in handling large-scale recommendation tasks. However, the embedding layers of DLRMs have emerged as a significant performance bottleneck due to their high demands on memory capacity and bandwidth. Existing solutions, such as offloading to CPU memory, are hampered by communication overheads, while many proposed Processing-In-Memory (PIM) architectures are not yet commercially viable, limiting their practical impact. A promising approach, demonstrated by recent work like UpDLRM, is to accelerate these layers using commercial Processing-In-Memory (PIM) hardware. However, this approach does not address the challenge of ever-growing model sizes in industrial-scale systems. To tackle both memory bandwidth and capacity limitations, this paper proposes GPAQuant, a framework that co-designs PIM-based acceleration with model quantization. GPAQuant strategically stores quantized embedding tables (EMTs) in distributed DPU memory and offloads both the memory-intensive lookups and reductions to the DPUs. To address DPU-specific challenges, we propose four key innovations: (1) architecture-aware EMT partitioning that optimizes data placement to minimize inter-DPU communication, (2) frequency-aware workload balancing to evenly distribute access patterns across DPUs, (3) fine-grained EMT rearrangement that reduces quantization errors while preserving spatial locality, and (4) group-wise quantization to balance dequantization overhead and model accuracy. Extensive evaluations across six datasets and four DLRM architectures demonstrate GPAQuant’s superiority: it achieves 4.6× speedup on embedding layers and 3.4× end-to-end inference acceleration compared to state-of-the-art CPU-GPU implementations and PIM baselines, with negligible accuracy loss.