Evaluating recommender systems from the user's perspective: Survey of the state of the art

Pearl Pu*, Li CHEN, Rong Hu

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

276 Citations (Scopus)


A recommender system is a Web technology that proactively suggests items of interest to users based on their objective behavior or explicitly stated preferences. Evaluations of recommender systems (RS) have traditionally focused on the performance of algorithms. However, many researchers have recently started investigating system effectiveness and evaluation criteria from users' perspectives. In this paper, we survey the state of the art of user experience research in RS by examining how researchers have evaluated design methods that augment RS's ability to help users find the information or product that they truly prefer, interact with ease with the system, and form trust with RS through system transparency, control and privacy preserving mechanisms finally, we examine how these system design features influence users' adoption of the technology. We summarize existing work concerning three crucial interaction activities between the user and the system: the initial preference elicitation process, the preference refinement process, and the presentation of the system's recommendation results. Additionally, we will also cover recent evaluation frameworks that measure a recommender system's overall perceptive qualities and how these qualities influence users' behavioral intentions. The key results are summarized in a set of design guidelines that can provide useful suggestions to scholars and practitioners concerning the design and development of effective recommender systems. The survey also lays groundwork for researchers to pursue future topics that have not been covered by existing methods.

Original languageEnglish
Pages (from-to)317-355
Number of pages39
JournalUser Modeling and User-Adapted Interaction
Issue number4-5
Publication statusPublished - Oct 2012

Scopus Subject Areas

  • Education
  • Human-Computer Interaction
  • Computer Science Applications

User-Defined Keywords

  • Design guidelines
  • Explanation interface
  • Recommender systems
  • Research survey
  • User experience research


Dive into the research topics of 'Evaluating recommender systems from the user's perspective: Survey of the state of the art'. Together they form a unique fingerprint.

Cite this