1 / 38

Decision Making

Decision Making. ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10. Outline. Maximum Expected Utility (MEU) Decision network Making decisions Russell & Norvig, chapter 16. Acting Under Uncertainty. With no uncertainty, rational decision is to pick action with “best” outcome

matteo
Télécharger la présentation

Decision Making

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Decision Making ECE457 Applied Artificial Intelligence Spring 2007 Lecture #10

  2. Outline • Maximum Expected Utility (MEU) • Decision network • Making decisions • Russell & Norvig, chapter 16 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 2

  3. Acting Under Uncertainty • With no uncertainty, rational decision is to pick action with “best” outcome • Two actions • #1 leads to great outcome • #2 leads to good outcome • It’s only rational to pick #1 • Assumes outcome is 100% certain • With uncertainty, it’s a little harder • Two actions • #1 has 1% probability to lead to great outcome • #2 has 90% probability to lead to good outcome • What is the rational decision? ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 3

  4. Acting Under Uncertainty • Maximum Expected Utility (MEU) • Pick action that leads to best outcome averaged over all possible outcomes of the action • How do we compute the MEU? • Easy once we know the probability of each outcome and their utility ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 4

  5. Utility • Value of a state or outcome • Computed by utility function • U(S) = utility of state S • U(S)  [0,1] if normalized ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 5

  6. Expected Utility • Sum of utility of all possible outcomes times probability of that outcome • Known evidence E about the world • Action A has i possible outcomes, with probability P(Resulti(A)|Do(A),E) • Utility of each outcome is U(Resulti(A)) • Evaluation function of the state of the world given Resulti(A) • EU(A|E)=i P(Resulti(A)|Do(A),E) U(Resulti(A)) ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 6

  7. Maximum Expected Utility • List all possible actions Aj • For each action, list all possible outcomes Resulti(Aj) • Compute EU(Aj|E) • Pick action that maximises EU ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 7

  8. Utility of Money • Use money as measure of utility? • Example • A1 = 100% chance of $1M • A2 = 50% change of $3M or nothing • EU(A2) = $1.5M > $1M = EU(A1) • Is that rational? ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 8

  9. Utility of Money • Utility/Money relationship is logarithmic, not linear • Example • EU(A2) = 3.1 < 6 = EU(A1) • Insurance • EU(paying) = –U(value of premium) • EU(not paying) = U(value of premium) – U(value of house) * P(losing house) ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 9

  10. Axioms • Given three states A, B, C • A  B • The agent prefers A to B • A ~ B • The agent is indifferent between A and B • A  B • The agent prefers A to B or is indifferent between A and B • [p1, A; p2, B; p3, C] • A can occur with probability p1, B can occur with probability p2, C can occur with probability p3 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 10

  11. Axioms • Orderability • (A  B)  (B  A)  (A ~ B) • Transitivity • (A  B)  (B  C)  (A  C) • Continuity • A  B  C  p [p, A; 1-p, C] ~ B • Substituability • A ~ B  [p, A; 1-p, C] ~ [p, B; 1-p, C] • Monotonicity • A  B  ( p  q  [p, A; 1-p, B]  [q, A; 1-q, B] ) • Decomposability • [p, A; 1-p, [q, B; 1-q, C]] ~ [p, A; (1-p)q, B; (1-p)(1-q), C] ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 11

  12. Axioms • Utility principle • U(A) > U(B)  A  B • U(A) = U(B) A ~ B • Maximum utility principle • U([p1, A1; … ; pn, An]) = i piU(Ai) • Given these axioms, MEU is rational! ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 12

  13. Decision Network • Our agent makes decisions given evidence • Observed variables and conditional probability tables of hidden variables • Similar to conditional probability • Probability of variables given other variables • Relationships represented graphically in Bayesian network • Could we make a similar graph here? ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 13

  14. Decision Network • Sometimes called influence diagram • Like a Bayesian Network for decision making • Start with variables of problem • Add decision variables that the agent controls • Add utility variable that specify how good each state is ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 14

  15. Decision Network • Chance node (oval) • Uncertain variable • Like in Bayesian network • Decision node (rectangle) • Choice of action • Parents: variables affecting decision, evidence • Children: variables affected by decision • Utility node (diamond) • Utility function • Parents: variables affecting utility • Typically only one in network ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 15

  16. Study Lucky Happiness PassExam Win Decision Network Example P(L) = 0.75 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 16

  17. Run into friends Bomber Patio Join your friends Sunny Have $ U Decision Network Example ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 17

  18. Making a Rational Decision • At a decision node • Given a combination of values of evidence variables, and each possible action given this evidence • Compute the EU of each action you can decide to do • Decide to do the action with the maximum EU • Policy: choice of action (not necessarily the best) for each possible combination of values of evidence variables ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 18

  19. Policy • Decision node Di • Can take values in domain dom(Di) • Has set of parents Pi that take values in domain dom(Pi) • Policy  is a set of mappings i of dom(Pi) to dom(Di) • iassociates a decision to each state the parents of Di can be in •  associates a series of decisions to each state the network can be in ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 19

  20. Bomber Patio Sunny Have $ Policy • Policy on going to Bomber patio • bp($,S) = BP • bp(¬$,S) = BP • bp($,¬S) = ¬BP • bp(¬$,¬S) = ¬BP ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 20

  21. Value of a Policy • Expected utility if decisions are taken according to the policy • EU() = x P(x) U(x,(x)) • EU(bp) = $,s P($,S) U($,S,bp($,S)) • Optimal policy * is the one with the highest expected utility • EU(*)  EU() for all policies  ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 21

  22. Computing the Optimal Policy • Start from last decision node before utility • For each combination of values of a node’s parents • Compute the expected utility of each decision • Set policy as decision that maximises utility • Work backward to the first decision in the network ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 22

  23. Run into friends Bomber Patio Join your friends Sunny Have $ U Computing the Optimal Policy • Compute the optimal policy for JF • For each combination of BP, RF and $, make a decision JF and compute U(JF,$) • Set the policy as the max utility decision for each combination of BP and RF ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 23

  24. Run into friends Bomber Patio Join your friends Sunny Have $ U Computing the Optimal Policy • Compute the optimal policy for BP given JF(BP,RF,$) • For each combination of S and $, make a decision BP, which will affect RF and JF • JF is decided by optimal policy • So we can compute U(JF,$) ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 24

  25. Decision Network Example • Bob wants to buy a used car. Unfortunately, the car he’s considering has a 50% chance of being a lemon. Before buying, he can decide to take the car to a mechanic to have it inspected. The mechanic will report if the car is good or bad, but he can make mistakes, and the inspection is expensive. Bob prefers owning a good car to not owning a car, and prefers that to owning a lemon. • Should Bob have the car inspected first or not? ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 25

  26. Inspect Buy Lemon Report U Decision Network Example ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 26

  27. Inspect Buy Lemon Report U Decision Network Example Utility cost of inspection = -50 P(L) = 0.5 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 27

  28. Decision Network Example • Compute EU of Buy and Not Buy given all combinations of evidence • Select action with MEU given each case • Compute EU of Inspect and Not Inspect given all combinations of evidence and then select Buy/Not Buy action • Decide on Inspect or Not Inspect, depending on MEU ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 28

  29. Decision Network Example • Compute the expected utility of buying and not buying the car given the evidence • The evidence is whether or not Bob got the car inspected, and what the result of the inspection is • EU(b|i,r) = l P(l|b,i,r)U(b,i,l) ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 29

  30. Decision Network Example • EU(B|¬I,N) = l P(l|B,¬I,N)U(B,¬I,l)EU(B|¬I,N) = P(L)U(B,L,¬I) + P(¬L)U(B,¬L,¬I)EU(B|¬I,N) = 0.5 * -600 + 0.5 * 1000EU(B|¬I,N) = 200 • EU(¬B|¬I,N) = l P(l|¬B,¬I,N)U(¬B,¬I,l)EU(¬B|¬I,N) = P(L)U(¬B,L,¬I) + P(¬L)U(¬B,¬L,¬I)EU(¬B|¬I,N) = 0.5 * -300 + 0.5 * -300EU(¬B|¬I,N) = -300 • Rational decision, if Bob doesn’t get the car inspected, is to buy it ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 30

  31. Decision Network Example • EU(B|I,G) = l P(l|B,I,G)U(B,I,l) • We’re missing some information! • From the network, we know P(L) and P(G|L), but not P(L|G) nor P(G) • Compute P(G) using marginalization • P(G) = P(G|L)P(L) + P(G|¬L)P(¬L) = 0.55 • Compute P(L|G) using Bayes’ Theorem • P(L|G) = P(G|L)P(L)/P(G) = 0.18 • P(¬L|G) = P(G|¬L)P(¬L)/P(G) = 0.82 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 31

  32. Decision Network Example • EU(B|I,G) = l P(l|B,I,G)U(B,I,l)EU(B|I,G) = P(L|G)U(B,L,I) + P(¬L|G)U(B,¬L,I)EU(B|I,G) = 0.18 * -650 + 0.82 * 950EU(B|I,G) = 662 • EU(¬B|I,G) = l P(l|¬B,I,G)U(¬B,I,l)EU(¬B|I,G) = P(L|G)U(¬B,L,I) + P(¬L|G)U(¬B,¬L,I)EU(¬B|I,G) = 0.18 * -350 + 0.82 * -350EU(¬B|I,G) = -350 • Rational decision, if Bob gets the car inspected and the report says it’s good, is to buy it ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 32

  33. Decision Network Example • EU(B|I,¬G) = l P(l|B,I,G)U(B,I,l)EU(B|I,¬G) = P(L|G)U(B,L,I) + P(¬L|G)U(B,¬L,I)EU(B|I,¬G) = 0.89 * -650 + 0.11 * 950EU(B|I,¬G) = -474 • EU(¬B|I,¬G) = l P(l|¬B,I,¬G)U(¬B,I,l)EU(¬B|I,¬G) = P(L|¬G)U(¬B,L,I) + P(¬L|¬G)U(¬B,¬L,I)EU(¬B|I,¬G) = 0.89 * -350 + 0.11 * -350EU(¬B|I,¬G) = -350 • Rational decision, if Bob gets the car inspected and the report says it’s not good, is to not buy it ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 33

  34. Decision Network Example • Should Bob get the car inspected? • EU(i) = l,r P(l,r|i)U(l,i,b) P(l,r|i) = P(r|l,i)P(l|i) = P(r|l,i)P(l) EU(i) = l,r P(r|l,i)P(l)U(l,i,b) • EU(¬I) = P(N|L,¬I)P(L)U(L,¬I,B) + P(N|¬L,¬I)P(¬L)U(¬L,¬I,B)EU(¬I) = 1 * 0.5 * -600 + 1 * 0.5 * 1000EU(¬I) = 200 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 34

  35. Decision Network Example • EU(I) = P(G|L,I)P(L)U(L,I,B) + P(G|¬L,I)P(¬L)U(¬L,I,B) + P(¬G|L,I)P(L)U(L,I,¬B) + P(¬G|¬L,I)P(¬L)U(¬L,I,¬B)EU(I) = 0.2 * 0.5 * -650 + 0.9 * 0.5 * 950 + 0.8 * 0.5 * -350 + 0.1 * 0.5 * -350EU(I) = 205 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 35

  36. Decision Network Example • EU(I) = 205 > EU(¬I) = 200 • Therefore, Bob should get the car inspected ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 36

  37. Value of Information • Utility of decision without inspectionis 200 • Utility of decision with inspectionis 205, utility of the decision minus the utility cost of the inspection • Utility of decision is 255 • At what point is the utility cost of the inspection too high? • 255 – Utility Cost < 200 • Value of the information gained from the inspection is 55 ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 37

  38. Value of Information • Information has value if • It causes a change in the decision • The new decision has higher utility than the old one • The value is • Non-negative • Zero for irrelevant facts • Zero for information already known ECE457 Applied Artificial Intelligence R. Khoury (2007) Page 38

More Related