RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching

Large Language Models (LLMs) are transforming recommendation into a generative task, yet their industrial deployment is hindered by the prohibitive latency of processing long, personalized prompts where standard prefix caching fails due to non-contiguous reuse patterns. To address this, we propose RcLLM, a distributed inference system that accelerates generative recommendation via Beyond-Prefix KV Caching. RcLLM decomposes prompts into reusable blocks and manages the massive scale of item catalogs through a stratified distributed storage architecture: compact user histories are replicated for zero-latency retrieval, while massive item caches are sharded using a similarity-aware placement algorithm. RcLLM effectively eliminates redundant quadratic computation with two strategic techniques: 1) an affinity-based global scheduler to maximize data locality, and 2) a selective attention mechanism to correct approximation errors. Evaluations on real-world datasets demonstrate that RcLLM reduces Time-To-First-Token (TTFT) by 1.31×-9.51× compared to state-of-the-art prefix caching systems, enabling real-time serving compliance with negligible impact on recommendation accuracy.