A user-centric evaluation framework of recommender systems

Pearl Pu, Li CHEN

Research output: Contribution to journalConference articlepeer-review

32 Citations (Scopus)


User experience research is increasingly attracting researchers' attention in the recommender system community. Existing works in this area have suggested a set of criteria detailing the characteristics that constitute an effective and satisfying recommender system from the user's point of view. To combine these criteria into a more comprehensive framework which can be used to evaluate the perceived qualities of recommender systems, we have developed a model called ResQue (Recommender systems' Quality of user experience). ResQue consists of 13 constructs and a total of 60 question items, and it aims to assess the perceived qualities of recommenders such as their usability, usefulness, interface and interaction qualities, users' satisfaction of the systems, and the influence of these qualities on users' behavioral intentions, including their intention to purchase the products recommended to them, return to the system in the future, and tell their friend about the system. This model thus identifies the essential qualities of an effective and satisfying recommender system and the essential determinants that motivate users to adopt this technology. The related questionnaire can be further adapted for a custom-made user evaluation or combined with objective performance measures. We also propose a simplified version of the model with 15 questions which can be employed as a usability questionnaire for recommender systems.

Original languageEnglish
Pages (from-to)14-21
Number of pages8
JournalCEUR Workshop Proceedings
Publication statusPublished - 2010
EventACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and their Interfaces, UCERSTI 2010 - Barcelona, Spain
Duration: 30 Sep 201030 Sep 2010

Scopus Subject Areas

  • Computer Science(all)

User-Defined Keywords

  • E-commerce recommender
  • Evaluation of decision support
  • Post-study questionnaire
  • Quality measurement
  • Quality of user experience
  • Recommender systems
  • Usability evaluation


Dive into the research topics of 'A user-centric evaluation framework of recommender systems'. Together they form a unique fingerprint.

Cite this