Preference Elicitation: An Overview

1. Preference Elicitation: An Overview Ronen Brafman Computer Science Department Ben-Gurion University, ISRAEL

2. Intelligence Readiness Objective conditions Political situation Strategic and tactical intentions and preferences Budget limitations Beliefs Experience Outcome evaluation for alternative decisions Campaign-level goals Monitor and filter information Adapt presentation Beliefs and preference refinement through incremental questioning Better decision Proposals of preferable courses of actions

3. Preference Elicitation: Statement of Problem To make good decisions on behalf of a user we must understand and represent his/her objectives Preference elicitation problem: obtaining a good and useful description of the agent’s objectives Preference elicitation is a well known bottle-neck for decision support and automation systems

4. The Task • Use: • Natural statements users make • Answers to questions users find intuitive • To generate • good (informative, faithful) and • useful (convenient to reason with) • representation of the user’s preference for the task at hand

5. CLASSICAL APPROACHES

6. Utility Functions • Real valued function on the space of possible outcomes • U(o) > U(o’)  o is a better outcome than o’ • Much more – allows evaluating actions • Classical means of expressing preferences • Rich, quantitative representation • Difficult to elicit from users • Over-kill when task essentially deterministic • E.g., search, filtering, display, content

7. Multi-Attribute Utility Theory - I • Describe a utility function for each aspect of an outcome, independently • e.g., utility of different arrival times, utility of safety of each means of transportation, etc. • Compare outcomes by comparing utilities component-wise • Use dominance relations (Pareto optimality) • Other techniques for qualitative/semi-quantitatve comparisons

8. Multi-Attribute Utility Theory - II • Additive representation: where a goes over all attributes • Estimation task is much simpler now • Reason: very strong independence assumption – attributes are mutually independent • Classical sources: • Decisions with Multiple Objectives, Keeney & Raiffa (1976) • Foundations of Measurement (Vol. 1), Krantz, Luce, Suppers, & Tversky (1971). • Utility Theory for Decision Making, Fishburn (1969)

9. Current and New Techniques

10. Capitalizing on Structure • Bacchus & Grove `96: Generalized additive independence (GAI) Here z goes over sets of (not necessarily disjoint) attributes. Some interesting connection with probabilistic independence, graphical models • Shoham `97: A Bayesian network like notion of utilities. Based on a new concept of utility “factors”. Allows one to benefit from all advantages of BN technology Problematic issue – counterpart of causality for utilities (mental causality?), thus in practice, it is not clear that users can actually describe this. • La Mura & Shoham `01: Expected utility networks Based on multiplicative notion of conditional independence. Interesting, but still has to be studied from the preference elicitation point of view. • Boutilier, Bacchus, and Brafman `01: UCP-nets Special case of GAI. Interesting and useful properties (quantitative CP-nets)

11. Qualitative Approaches • People are more comfortable making qualitative preference statements (Doyle’s presentation): • “I like this more than that” • “Safety is more important than performance” • Qualitative information can be used to rank options/outcomes • Can be used as a starting point for obtaining a quantitative description (i.e., utilities) • Sufficient for many tasks (e.g., filtering, monitoring, and displaying information)

12. Conditional Logic of Preference Identifying and exploiting interrelation (and mutual irrelevance) between various parameters of the problem. • Doyle and Wellman (1991):Logic of relative desire • Reason about statements of the form “a is preferred to b” • Adopts ceteris paribus (“all else being equal”) semantics. • Extends the “logic of preference” of von Wright (mainly propositional) • Boutilier (1994) – The Logic CO • Reason about statements of the form “If p then it is better to have q” • Semantics: Any p-world with q is better than any p-world without q, unless over-ruled by a more specific statement. • Reasoning based on propositional non-monotonic logic (various options can be used, no clear “winner”…).

13. Capitalizing on Structure: CP-nets [Boutilier, Brafman, Hoos, Poole] An intuitive, qualitative, graphical model of preferences, that captures statements of (conditional) preferential independence (Try to get Bayes-nets like benefits for preference reasoning) • Core part of the model is a directed graph: • Each node represents a domain variable. • The immediate parents Parents(X) of a variable Xin the network are those variables that affect user’s preference over the values of X. • Parents(exterior-color) = { vehicle-category } • TCP-nets [Brafman & Domshlak]: a more expressive variant • Associated optimization algorithms

14. Pants Jacket Shirt

15. worst best

16. Day of the flight Departure Time Airline Stop-overs Class

17. Other Techniqes - I • Minmax Regret (Boutilier et. al.) • Get partial information about utility function • Structure, Bounds, etc. • Find decision minimizing “regret” w.r.t. all utility functions consistent with information • Incremental Elicitation • Try to solve with partial information • Identify specific information required and ask • Examples: • Combinatorial auctions [Sandholm et. al.] • Product search [Blythe et. al., Brafman & Domshlak]

18. Other Techniques - II • Mixed Qualitative and Quantitative • Get qualitative information • Interpret this as constraints on utility function • Work with some consistent utility function • Can be used in conjunction with incremental elicitation (refining the utility function estimate) • Examples: [Doyle et al.; Brafman & Domshlak] • Preference elicitation as a decision/learning problem (e.g., a POMDP [Chajewska & Koller; Boutilier] ) • Using “problematic” preference information [Domshlak & Brafman, Brafman & Dimopoulos]

19. Recent Applications • Content display and adaptation [Domshlak et. al., Brafman & Friedman] • Flight selection [Blythe; Brafman & Domshlak, Boutilier et. al.] • Autonomic Computing (resource allocation) [Boutilier et. al.] • Combinatorial Auctions [Sandholm et. al.]

20. Future • A number of promising directions • Need more multi-disciplinary interaction • Preference workshop AAAI’02 • Upcoming special issue of Comp. Intell. • A Dagstuhl workshop in June 2004 with representatives from Database, AI, Economics, and Philosophy communities • More cooperation, applications, needed

21. Improvements • Examples, less formulas • Don’t mention less important directions – just the idea • Slide on approaches – very short • Slides on select interesting approaches with examples