IMPACT OF EXPLANATIONS ON TRUST IN ONLINE RECOMMENDATION AGENTS

1. IMPACT OF EXPLANATIONS ON TRUST IN ONLINE RECOMMENDATION AGENTS Weiquan Wang & Izak Benbasat Sauder School of Business The University of British Columbia November 2005

2. Acknowledgements: This work is supported by: Natural Sciences & Engineering Research Council Social Sciences & Humanities Research Council Canada Research Chairs Program Sauder School of Business, UBC

3. An Overview:UBC research program on human-computer interaction in e-commerce

4. Emphasis on Communication (HCI) in the E-Commerce Era • Organization’s “window to the world” • From efficiency & effectiveness to relationship-based issues • Reducing Distance & Enhancing Trust • Social presence of other customers • Product Understanding • Store (telepresence) Sauder School of Business, UBC

5. Framework of Analysis Foci of Virtual Online Communication Experience STORE (web page) AGENT (software/person) PRODUCT [understanding] PEOPLE • Adaptiveness&personalization • Trust-assuring arguments • Customer Service Life Cycle • Form vs. Function • Static, Video, • vs. VPE • Collaborative Shopping • Sales reps (Avatars) • Communities • Needs-based (personalization) • Explanations • Handheld devices • Strategy choice • Similarity Sauder School of Business, UBC

6. APPLICATION DOMAINS Foci of Virtual Online Communication Experience STORE AGENT PRODUCT PEOPLE E-government Mobile computing Sauder School of Business, UBC

7. Agenda • eCommerce Recommendation Agents • Objectives • Motivation • Literature Review • Research Model and Hypotheses • Method • Results • Discussion Sauder School of Business, UBC

8. Definition Motivation Agents Literature Hypotheses Method Results Discussion Content Filtering Product Recommendation Agents Software entities that provide shopping advice on what to buy for consumers based on their own specified needs and/or preferences. • Provider Examples Examples at: www.activebuyersguide.com (mortgages, notebook computers, dog food, cameras) Sauder School of Business, UBC

9. Objectives Motivation Agents Literature Hypotheses Method Results Discussion Product Brokering (what to buy) Collaborative Filtering (Segmentation) Recommendation Agents Merchant Brokering (where to buy) Content Filtering (Conversation) (Maes et al. 1999) (Ansari et al. 2000) Sauder School of Business, UBC

10. Objectives Motivation Agents Literature Hypotheses Method Results Discussion • Identify the types of explanations that should be embedded in online recommendation agents. • Empirically investigate the influence of these explanations on users’ initial trust in and intentions to adopt online recommendation agents. • Identify the trust building/reducing processes in online recommendation agents. Sauder School of Business, UBC

11. Part III Perceived Usefulness Use of How Explanations + + + Competence Benevolence Integrity Perceived Ease of Use Trust in Agent Use of Why Explanations + + + + + + Intention to Adopt Use of Guidance Objectives Motivation Agents Literature Hypotheses Method Results Discussion Overall Project Part I Part II Sauder School of Business, UBC

12. Objectives Motivation Agents Literature Hypotheses Method Results Discussion • Why Agents? • Information overload (Maes 1994) • No salesperson available to consult with (Kim & Yoo 2000) • Improve decision quality (Haubl & Trifts 2000) • Increase user trust in the website/e-store (Urban et al. 1999) • Why Trust? • Key concern in eCommerce for both researchers and practitioners (McKnight and Chervany 2001) • Trust in intelligent agents is one of the main issues in the agent applications (Maes 1994) and is an under-investigated area • Why Explanations? • One of the critical components of intelligent and Knowledge-based Systems (KBS) since their inception (Gregor and Benbasat 1999) What Agents? What Trust? What Explanations? Sauder School of Business, UBC

13. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Two issues affect of the types of explanations that should be provided by recommendation agents. • An agency relationship between an agent and its users • An Internet-based agent is not owned exclusively by one user or company • Does an agent benefit the user only, or also for its provider, i.e., a particular merchant or manufacturer? • Information asymmetry and opportunism (Bergen et al. 1992) • A high amount of discretion is granted to users by the agent in specifying product attributes Sauder School of Business, UBC

14. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Sauder School of Business, UBC

15. Objectives Motivations Agents Literature Hypotheses Method Results Discussions • KBS Explanation Studies • Related to explanation variables Sauder School of Business, UBC

16. Explanations Users’ actions:Guidance(what to do, what input values to use, etc.) System’s actions: KBS Explanations (what it does, how it works, why its actions are appropriate, etc .) Objectives Motivations Agents Literature Hypotheses Method Results Discussions KBS Explanations Studies (Silver 1991, Wilson and Zigurs 1999, Limayem and DeSanctis 2000, Barkhi 2002 ) (Ye and Johnson 1995, Dhaliwal and Benbasat 1996, Gregor and Benbasat 1999, Mao and Benbasat 2000) Sauder School of Business, UBC

17. Objectives Motivations Agents Literature Hypotheses Method Results Discussions KBS Explanations • Classification by the nature of the explanation queries (What, Why, How, When..) Sauder School of Business, UBC

18. Objectives Motivations Agents Literature Hypotheses Method Results Discussions KBS Explanations (cont’d) • Outcomes, especially, users’ perceptions • Gregor and Benbasat (1999) propose that explanations, by virtue of making the performance of a system transparent to its users, are influential for user acceptance of intelligent systems and for improving user’s trust in the advice provided. • Ye and Johnson (1995) suggest that explanation facilities can make the advice generated by an expert system more acceptable: post-explanation beliefs were higher than pre-explanation belief scores. • Mao and Benbasat, and Nah and Benbasat observed that explanations lead to knowledge transfer from KBS (source experts) to users. • Since no reliable measures are available (e.g., trust) until recently, most previous empirical studies used surrogate variables and the results seem mixed (Lerch et al. 1990, Dhaliwal 1993). Sauder School of Business, UBC

19. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Guidance • Defn: how a decision support system influences its users as they structure and execute their decision-making processes (Silver 1991). • Guidance can be both about structuring the decision making process, how to use different features of a system, as well as recommendations about parameters and other input values (Silver 1991) Sauder School of Business, UBC

20. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Trust • Trust in Online Recommendation Agents General Approaches in recent literature • A belief(s); • Emotional feelings; • An intention; • A combination of these elements. Sauder School of Business, UBC

21. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Trust (cont’d) • Belief approach • Competence refers to trustor’s perception that the agent has the knowledge and expertise that enables it to perform effectively. • Benevolence refers to trustor’s perception that the agent behaves in a way that is for the good of the trustor (e.g., customers), aside from an egocentric profit motive. • Integrity refers to the trustor’s perception that the information provided by the RA is true and objective. Sauder School of Business, UBC

22. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Trust (cont’d) • Trust in Humans vs. Technological Artifacts • Opponents: only 21% of participants consistently held computers morally responsible for error  “technological artifacts have not yet produced in substance and structure that warrant in any stringent sense the attribution of consciousness or agency”(Friedman et al. 2000, p.36). Sauder School of Business, UBC

23. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Trust (cont’d) • Proponents: • Muir and his collaborators defined trust in machines and automation to include a morality dimension (e.g., responsibility). • Cassell and Bickmore (2000) have defined trust in computer agents as a composite of benevolence and credibility. • Sztompka (1999) and Searle (1981) argue that trust in a person and trust in technology are not fundamentally different: “Behind all human-made technologies are people (designers)”. • Theory of social responses to computers (Reeves and Nass 1996): “People treat computers like real people”. • Jian et al. (2000) have empirically examined different types of trust: human-human trust, human-machine trust, and trust in general, and found them to be similar. Sauder School of Business, UBC

24. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Trust (cont’d) • Trust in Humans vs. Technological Artifacts • Summary: • As if response: People respond socially to technological artifacts and do perceive that they have human properties (e.g., motivation, integrity, and personalities). • Given the perceptual nature of trust, such trustor’s perceptions are important in trust formation. • Interpersonal trust applies to trust in technological artifacts and components of trust in humans and in technological artifacts are not significantly different. • The agents and their explanation facilities investigated in this study are designed to bolster perceptions of benevolence and integrity. Sauder School of Business, UBC

25. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Sauder School of Business, UBC

26. Perceived Usefulness Use of How Explanations H1a Competence Benevolence Integrity H6a H6b H5b Perceived Ease of Use H2a Use of Why Explanations Trust in Agent H7 H3a H5a H5c Use of Guidance Intention to Adopt Objectives Motivations Agents Literature Hypotheses Method Results Discussions Research Model Sauder School of Business, UBC

27. Objectives Motivations Agents Literature Hypotheses Method Results Discussions PLS Results Trust Components -Competence -Benevolence -Integrity Perceived Usefulness (r2=.62) .51** .35** .90** Trust in Agent (r2=.42) .87** Perceived Ease of Use .65** .65** .43** .20* n.s. Intention to Adopt (r2=.35) ** significance at .01 level; * denotes significance at .05 level n.s. denotes non-significance at .05 level. Trust in agent is treated as a second-order reflective construct. In PLS, all measures for the first order trusting beliefs were repeated for the second order trust construct (Chin 2000). Sauder School of Business, UBC

28. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Explanation Defn and Operationalization Our Experimental Platform • Input /Dialogue • Why Ask? (Users’ needed considered) • Guidance for Determining Needs • HOW • WHY • Decision Making • How? (Reasoning) • GUIDANCE • Results/Recommendations • Why Recommend? (Users’ needs satisfied) • How Reached? (Formula and Deductions) • Guidance (If no recommendations, suggestions) Sauder School of Business, UBC

29. Objectives Motivations Agents Literature Hypotheses Method Results Discussions How Explanation Defn and Operationalization Q: How far away are your subjects that will be focused on most often by the digital camera? 1). I don't need my camera to focus on anything other than the subject in the immediate vicinity 2). I want a camera that'll focus on subjects at a moderate distance 3). I want a camera that'll focus on subjects from far away 4). I don't have an opinion on this How explanations reveal the line of reasoning used by the agent based on users’ needs and preferences, and detail the logical process to reach the final recommendations. How far away the subjects you want to focus on are will determine the suitable zoom level of a digital camera. If you want a camera that will focus on subjects more far away, the camera with stronger optical zoom level will have higher priority in my recommendations. Specifically, the four options will determine the following zoom levels: 1). 2X optical zoom and below. 2). Between 2X and 5X optical zoom 3). 4X optical zoom and above. 4). No minimum requirement in zoom capability. Sauder School of Business, UBC

30. Objectives Motivations Agents Literature Hypotheses Method Results Discussions How Explanation & Hypothesis • Use of How Explanations  Competence belief • How explanations link what buyers need to product attributes that fit their needs. • How explanations demonstrate the competencies that enable the agent to make recommendations. • The more skills and expertise that an trustee demonstrates, the more credible it becomes in the eyes of the audience/trustor, (Hovland et al. 1953) Sauder School of Business, UBC

31. Objectives Motivations Agents Literature Hypotheses Method Results Discussions How Explanation & Hypothesis (cont’d) • Use of How Explanations  Benevolence belief (H1b) • Use of How Explanations  Integrity belief • No direct theoretical or empirical evidence. • Exploratory. Sauder School of Business, UBC

32. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Why Explanation Defn and Operationalization Q: How far away are your subjects that will be focused on most often by the digital camera? 1). I don't need my camera to focus on anything other than the subject in the immediate vicinity 2). I want a camera that'll focus on subjects at a moderate distance 3). I want a camera that'll focus on subjects from far away 4). I don't have an opinion on this Why explanations: 1) justify the importance and purpose of an agent’s questions to users to gather their inputs, and 2) provide justifications (i.e., what are user’s most important needs are satisfied by the recommended product) for the recommendations provided after the consultation is complete]. The purpose of asking this question is to know what kinds of photos you will take often. It is quite useful to take photos at different distances. For example, for portraits of family and friends, subjects are close to a camera, but for many scenery or artistic photos, subjects may be far away from your camera. Sauder School of Business, UBC

33. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Why Explanation & Hypothesis • Why explanations • The existence of agency relationship between an agent and its users leads to information asymmetry and opportunism (Bergen, et al. 1992). • Strategy: The agent can engage in actions aimed at signaling the user that it is the type of the agent the user desires. Sauder School of Business, UBC

34. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Why Explanation & Hypothesis (cont’d) • Use of Why Explanations  Benevolence belief • Why explanations in this study are used to show the agent’s goal of meeting the needs of the users and satisfying their preferences. • Motives and intentions are important factors in building a benevolent image (Cook and Wall 1980). • Trust emerges when one party identifies and understand the other party’s goals and intentions better. (Brashear et al. 2003 and Doney and Cannon 1997) Sauder School of Business, UBC

35. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Why Explanation & Hypothesis (cont’d) • Use of Why Explanations  Competence belief (H2b) • Use of Why Explanations  Integrity belief • No theoretical or empirical evidence. • Exploratory. Sauder School of Business, UBC

36. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Guidance Defn and Operationalization Q: How far away are the subjects that will be focused on most often by the digital camera? 1). I don't need my camera to focus on anything other than the subject in the immediate vicinity 2). I want a camera that'll focus on subjects at a moderate distance 3). I want a camera that'll focus on subjects from far away 4). I don't have an opinion on this Guidance refers to the knowledge about the potential constraints brought by different choices for each question in the user-agent dialogue, and about how to adjust a user’s needs and preferences accordingly. Most digital cameras can take pictures beyond the immediate vicinity. However, cameras capable of taking pictures from very far away will be more expensive. As well, your choices will be more limited (only about 20%). Hence, be careful not to over-estimate your needs. Sauder School of Business, UBC

37. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Guidance & Hypothesis • Use of Decisional Guidance  Integrity belief • Nielsen et al. (2000) report that balanced product information is among the information that online shoppers want most. • Using the decisional guidance, users will not only notice the usefulness of different product feature choices, but also be exposed to the potential costs of different choices. • The objectivity and honesty of the agent can be released through the decisional guidance. • The trustee is deemed to exhibit high integrity by the trustor when the trustee is believed to have a strong sense of justice, honesty, and objectivity(Mayer et al. 1995). Sauder School of Business, UBC

38. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Guidance & Hypothesis (cont’d) • Use of Decisional Guidance  Competence belief Use of Decisional Guidance  Benevolence belief • No theoretical or empirical evidence. • Exploratory. Sauder School of Business, UBC

39. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Control Variables • Trust propensity • A personality trait that affect the likelihood an entity will exhibit trust (Lee and Turban 2001, Mayer et al. 1995). • McKnight et al. (2002) suggest that disposition to trust should influence trusting beliefs. • Users’ product expertise • Expertise influences the use of explanations and effects (e.g., performance and perception) of explanation use (Ye and Johnson 1995, Dhaliwal and Benbasat 1996, Gregor and Benbasat 1999). • Users’ product expertise will provide a foundation for their understanding of the explanations. • Users’ preference for effort saving vs. decision quality • Use and assimilation of the explanations requires users’ cognitive effort. Sauder School of Business, UBC

40. Objectives Motivations Agents Literature Hypotheses Method Results Discussions • A 2 x 2 x 2 factorial experimental design • How explanations (with, without), why explanations (with, without), and decisional guidance (with, without). • All three factors were manipulated between subjects. • A simulated recommendation agent for digital cameras was developed. • Digital cameras? • Content-filtering-based agent technology works best for relatively complex products (Russo 2002). • Many participants did not have digital cameras though they are interested in them. Sauder School of Business, UBC

41. Objectives Motivations Agents Literature Hypotheses Method Results Discussions • Explanation Validation • Face validity and definitional accuracy of the explanations • Definitional accuracy refers to how faithfully an explanation represents an operationalization of the definition of its class. • 8 graduate students. Sauder School of Business, UBC

42. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Sauder School of Business, UBC

43. Objectives Motivations Agents Literature Hypotheses Method Results Discussions • Subjects, Incentives, and Experimental Tasks • A total of 120 student participants, 15 per group • Only those who do not have digital cameras were invited to participate • $15 for their participation, the top 25% performers would get $25 each and the best one $200 • Two tasks, choosing one digital camera once for a best friend and once for a close family member • The agent was evaluated followed by the completion of two tasks • Measures • Adapted from Xiao and Benbasat (2002), Davis (1989). Sauder School of Business, UBC

44. Objectives Motivations Agents Literature Hypotheses Method Results Discussions • Manipulation Check • Measurement Validation • ANCOVA Sauder School of Business, UBC

45. Objectives Motivations Agents Literature Hypotheses Method Results Discussions Manipulation Check • Manipulation Check. • The average usages are quite high comparing with other empirical studies. Sauder School of Business, UBC

46. Note *: For each type of explanations that are provided to participants in the recommendation agent, the total number of explanations ranged from 26 to 38. Explanations were provided for each question in the agent-user dialogues as well as for each recommendation, if any, after the dialogues. The total number of explanations varied because participants got different numbers of recommendations after the agent-user dialogues. Sauder School of Business, UBC

47. Objectives Motivations Agents Literature Hypotheses Method Results Discussions ANCOVA • ANCOVA • Of the three covariates, only trust propensity influenced the dependent scores (competence) significantly at the .05 level • Only trust propensity was included as covariate Sauder School of Business, UBC

48. Objectives Motivations Agents Literature Hypotheses Method Results Discussions ANCOVA (cont’d) • Effects on Competence Note: All interactions (i.e., Guidance * How, How * Why, Guidance * Why, and Guidance * Why * How) are insignificant at .1 level. Sauder School of Business, UBC

49. Objectives Motivations Agents Literature Hypotheses Method Results Discussions ANCOVA (cont’d) • Effects on Benevolence Note: All interactions (i.e., Guidance * How, How * Why, Guidance * Why, and Guidance * Why * How) are insignificant at .1 level. Sauder School of Business, UBC

50. Objectives Motivations Agents Literature Hypotheses Method Results Discussions ANCOVA (cont’d) • Effects on Integrity Note: All interactions (i.e., Guidance * How, How * Why, Guidance * Why, and Guidance * Why * How) are insignificant at .1 level. Sauder School of Business, UBC