User evaluations on sentiment-based recommendation explanations

Li Chen, Dongning Yan*, Feng Wang

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

21 Citations (Scopus)


The explanation interface has been recognized as important in recommender systems because it can allow users to better judge the relevance of recommendations to their preferences and, hence, make more informed decisions. In different product domains, the specific purpose of explanation can be different. For high-investment products (e.g., digital cameras, laptops), how to educate the typical type of new buyers about product knowledge and, consequently, improve their preference certainty and decision quality is essentially crucial. With this objective, we have developed a novel tradeoff-oriented explanation interface that particularly takes into account sentiment features as extracted from product reviews to generate recommendations and explanations in a category structure. In this manuscript, we first reported the results of an earlier user study (in both before-after and counter-balancing setups) that compared our prototype system with the traditional one that purely considers static specifications for explanations. This experiment revealed that adding sentiment-based explanations can significantly increase users' product knowledge, preference certainty, perceived information usefulness, perceived recommendation transparency and quality, and purchase intention. In order to further identify the reason behind users' perception improvements on the sentiment-based explanation interface, we performed a follow-up lab controlled eye-tracking experiment that investigated how users viewed information and compared products on the interface. This study shows that incorporating sentiment features into the tradeoff-oriented explanations can significantly affect users' eye-gaze pattern. They were stimulated to not only notice bottom categories of products, but also, more frequently, to compare products across categories. The results also disclose users' inherent information needs for sentiment-based explanations, as they allow users to better understand the recommended products and gain more knowledge about static specifications.

Original languageEnglish
Article number20
Number of pages38
JournalACM Transactions on Interactive Intelligent Systems
Issue number4
Early online date9 Aug 2019
Publication statusPublished - Dec 2019

Scopus Subject Areas

  • Human-Computer Interaction
  • Artificial Intelligence

User-Defined Keywords

  • Explanation interfaces
  • Eye-tracking experiment
  • Product reviews
  • Recommender systems
  • Sentiment analysis
  • User perceptions
  • User study


Dive into the research topics of 'User evaluations on sentiment-based recommendation explanations'. Together they form a unique fingerprint.

Cite this