A user-centric evaluation framework of recommender systems

Pearl Pu, Li Chen

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

39 Citations (Scopus)

Abstract

User experience research is increasingly attracting researchers' attention in the recommender system community. Existing works in this area have suggested a set of criteria detailing the characteristics that constitute an effective and satisfying recommender system from the user's point of view. To combine these criteria into a more comprehensive framework which can be used to evaluate the perceived qualities of recommender systems, we have developed a model called ResQue (Recommender systems' Quality of user experience). ResQue consists of 13 constructs and a total of 60 question items, and it aims to assess the perceived qualities of recommenders such as their usability, usefulness, interface and interaction qualities, users' satisfaction of the systems, and the influence of these qualities on users' behavioral intentions, including their intention to purchase the products recommended to them, return to the system in the future, and tell their friend about the system. This model thus identifies the essential qualities of an effective and satisfying recommender system and the essential determinants that motivate users to adopt this technology. The related questionnaire can be further adapted for a custom-made user evaluation or combined with objective performance measures. We also propose a simplified version of the model with 15 questions which can be employed as a usability questionnaire for recommender systems.

Original languageEnglish
Title of host publicationProceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces
EditorsBart P. Knijnenburg, Lars Schmidt-Thieme, Dirk Bollen
PublisherCEUR-WS
Pages14-21
Number of pages8
Publication statusPublished - Sept 2010
EventACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and their Interfaces, UCERSTI 2010 - Barcelona, Spain
Duration: 30 Sept 201030 Sept 2010
https://ceur-ws.org/Vol-612/

Publication series

NameCEUR Workshop Proceedings
PublisherCEUR-WS
Volume612
ISSN (Print)1613-0073

Conference

ConferenceACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and their Interfaces, UCERSTI 2010
Country/TerritorySpain
CityBarcelona
Period30/09/1030/09/10
Internet address

Scopus Subject Areas

  • Computer Science(all)

User-Defined Keywords

  • E-commerce recommender
  • Evaluation of decision support
  • Post-study questionnaire
  • Quality measurement
  • Quality of user experience
  • Recommender systems
  • Usability evaluation

Fingerprint

Dive into the research topics of 'A user-centric evaluation framework of recommender systems'. Together they form a unique fingerprint.

Cite this