TY - JOUR
T1 - User evaluations on sentiment-based recommendation explanations
AU - Chen, Li
AU - Yan, Dongning
AU - Wang, Feng
N1 - Funding Information:
This research work was supported by Hong Kong Research Grants Council (under project RGC/HKBU12200415) and the Fundamental Research Funds of Shandong University, China. We thank all participants who took part in our experiments. We also thank Lin Zhang for assisting in eye-tracking data preprocessing and KingFar International Inc for providing the eye tracker. Authors’ addresses: L. Chen and F. Wang, Department of Computer Science, Hong Kong Baptist University, 224 Waterloo Road, Kowloon Tong, Hong Kong, China; emails: {lichen, fwang}@comp.hkbu.edu.hk; D. Yan (corresponding author), School of Mechanical Engineering, Shandong University, 17923 Jingshi Road, Jinan, Shandong, China; email: yandongning@sdu.edu.cn. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2019 Association for Computing Machinery. 2160-6455/2019/08-ART20 $15.00 https://doi.org/10.1145/3282878
PY - 2019/12
Y1 - 2019/12
N2 - The explanation interface has been recognized as important in recommender systems because it can allow users to better judge the relevance of recommendations to their preferences and, hence, make more informed decisions. In different product domains, the specific purpose of explanation can be different. For high-investment products (e.g., digital cameras, laptops), how to educate the typical type of new buyers about product knowledge and, consequently, improve their preference certainty and decision quality is essentially crucial. With this objective, we have developed a novel tradeoff-oriented explanation interface that particularly takes into account sentiment features as extracted from product reviews to generate recommendations and explanations in a category structure. In this manuscript, we first reported the results of an earlier user study (in both before-after and counter-balancing setups) that compared our prototype system with the traditional one that purely considers static specifications for explanations. This experiment revealed that adding sentiment-based explanations can significantly increase users' product knowledge, preference certainty, perceived information usefulness, perceived recommendation transparency and quality, and purchase intention. In order to further identify the reason behind users' perception improvements on the sentiment-based explanation interface, we performed a follow-up lab controlled eye-tracking experiment that investigated how users viewed information and compared products on the interface. This study shows that incorporating sentiment features into the tradeoff-oriented explanations can significantly affect users' eye-gaze pattern. They were stimulated to not only notice bottom categories of products, but also, more frequently, to compare products across categories. The results also disclose users' inherent information needs for sentiment-based explanations, as they allow users to better understand the recommended products and gain more knowledge about static specifications.
AB - The explanation interface has been recognized as important in recommender systems because it can allow users to better judge the relevance of recommendations to their preferences and, hence, make more informed decisions. In different product domains, the specific purpose of explanation can be different. For high-investment products (e.g., digital cameras, laptops), how to educate the typical type of new buyers about product knowledge and, consequently, improve their preference certainty and decision quality is essentially crucial. With this objective, we have developed a novel tradeoff-oriented explanation interface that particularly takes into account sentiment features as extracted from product reviews to generate recommendations and explanations in a category structure. In this manuscript, we first reported the results of an earlier user study (in both before-after and counter-balancing setups) that compared our prototype system with the traditional one that purely considers static specifications for explanations. This experiment revealed that adding sentiment-based explanations can significantly increase users' product knowledge, preference certainty, perceived information usefulness, perceived recommendation transparency and quality, and purchase intention. In order to further identify the reason behind users' perception improvements on the sentiment-based explanation interface, we performed a follow-up lab controlled eye-tracking experiment that investigated how users viewed information and compared products on the interface. This study shows that incorporating sentiment features into the tradeoff-oriented explanations can significantly affect users' eye-gaze pattern. They were stimulated to not only notice bottom categories of products, but also, more frequently, to compare products across categories. The results also disclose users' inherent information needs for sentiment-based explanations, as they allow users to better understand the recommended products and gain more knowledge about static specifications.
KW - Explanation interfaces
KW - Eye-tracking experiment
KW - Product reviews
KW - Recommender systems
KW - Sentiment analysis
KW - User perceptions
KW - User study
UR - http://www.scopus.com/inward/record.url?scp=85075668509&partnerID=8YFLogxK
U2 - 10.1145/3282878
DO - 10.1145/3282878
M3 - Article
AN - SCOPUS:85075668509
SN - 2160-6455
VL - 9
JO - ACM Transactions on Interactive Intelligent Systems
JF - ACM Transactions on Interactive Intelligent Systems
IS - 4
M1 - 20
ER -