Prompt Distillation for Efficient LLM-based Recommendation

Lei Li, Yongfeng Zhang, Li Chen

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

1 Citation (Scopus)


Large language models (LLM) have manifested unparalleled modeling capability on various tasks, e.g., multi-step reasoning, but the input to these models is mostly limited to plain text, which could be very long and contain noisy information. Long text could take long time to process, and thus may not be efficient enough for recommender systems that require immediate response. In LLM-based recommendation models, user and item IDs are usually filled in a template (i.e., discrete prompt) to allow the models to understand a given task, but the models usually need extensive fine-tuning to bridge the user/item IDs and the template words and to unleash the power of LLM for recommendation. To address the problems, we propose to distill the discrete prompt for a specific task to a set of continuous prompt vectors so as to bridge IDs and words and to reduce the inference time. We also design a training strategy with an attempt to improve the efficiency of training these models. Experimental results on three real-world datasets demonstrate the effectiveness of our PrOmpt Distillation (POD) approach on both sequential recommendation and top-N recommendation tasks. Although the training efficiency can be significantly improved, the improvement of inference efficiency is limited. This finding may inspire researchers in the community to further improve the inference efficiency of LLM-based recommendation models.

Original languageEnglish
Title of host publicationCIKM 2023 - Proceedings of the 32nd ACM International Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery (ACM)
Number of pages10
ISBN (Print)9798400701245
Publication statusPublished - 21 Oct 2023
Event32nd ACM International Conference on Information and Knowledge Management, CIKM 2023 - Birmingham, United Kingdom
Duration: 21 Oct 202325 Oct 2023

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings


Conference32nd ACM International Conference on Information and Knowledge Management, CIKM 2023
Country/TerritoryUnited Kingdom
Internet address

Scopus Subject Areas

  • Business, Management and Accounting(all)
  • Decision Sciences(all)

User-Defined Keywords

  • Explainable Recommendation
  • Generative Recommendation
  • Large Language Models
  • Prompt Distillation
  • Recommender Systems
  • Sequential Recommendation
  • Top-N Recommendation


Dive into the research topics of 'Prompt Distillation for Efficient LLM-based Recommendation'. Together they form a unique fingerprint.

Cite this