State Space 4 Chapter 4

1. State Space 4Chapter 4 Adversarial Games

2. Games of Perfect Information • Each player knows everything that can be known • Chess, Othello • Games of Imperfect Information • Player’s have partial knowledge • Poker: dispute is settled by revealing the contents of one’s hand Two Flavors

3. Use simple heuristics and search many nodes • Use sophisticated heuristics and search few nodes • Cost of calculating the heuristics might outweigh the cost of opening many nodes • The closer h is to h*, the better informed it is. But information can be expensive Two Approaches to Perfect Information Games

4. Two Players • min: tries to achieve an outcome of 0 • max: tries to achieve an outcome of 1 A Model: MiniMax on exhaustively searchable graphs

5. MAX A 0 C D Min B 1 Max G E F Min K 1 0 J H I You are max at node A Max O N 1 1 L M 0 0 R 0 P Q Min 1

6. Max would like to go to F,J,N,Q,L • Min would like to go to D,H,P,R,M Propagating Scores: A first pass • Min’s Turn a Node I • Go to M to win • So assign I a 0 • Max’s turn at node O • Go to Q to win • So assign 1 to node O Wins

7. If faced with two labeled choices, you would choose 0 (if min) or 1 (if max) • Assume you’re opponent will play the same way Conclusion

8. For each unlabeled node in the tree • If it’s max’s turn, give it the max score of its children • If it’s min’s turn, give it the min score of its children Now label the tree Conclusion: Max must choose C at the first move or lose the game Propagating Scores

9. 7 coins • 2 players • Players divide coins into two piles at each move, such that • Piles have an unequal number of coins • No pile is empty • Play ends when a player can no longer move Nim

10. min 7 4,3 5,2 6,1 Start of Game 5,1,1 4,2,1 Complete the game to see that min wins only if max makes a mistake at 6,1 or 5,2

11. Problem • For games of any complexity, you can’t search the whole tree Solution • Look ahead a fixed number of plys (levels) • Evaluate according to some heuristic estimate • Develop a criterion for pruning subtrees that don’t require examination αβ Pruning

12. Instead of representing wins, numbers represent the relative goodness of nodes. At any junction • Max chooses highest • Min chooses lowest Higher means better for max Lower means better for min Recast

13. What should Max do? Max 8 g 4 8 j k 9 t beta n m z alpha 4 8 12 r p q Min 7 3 9

14. Max’s turn at node g • Left subtree of g has been explored • If max chooses j, min will choose m • So the best max can do by going left is 8. Call this α Now Examine K and its left subtree n with a value of 4 • If max chooses k, the worst min can do is 4. • Why? T may be < 4. If it is min will choose it. If not, min will choose 4 • So the worst min can do, if max goes right is 4. Call this β Situation

15. Must max expand the rst(K)? • No. min is guaranteed 4. • But if max chose j, min is guaranteed 8 • So max is better off by going left More formally: Val(k) = min(4, val(t)) <= 4 Val(g) = max(8,val(k)) = max(8, min(4,val(t)) = 8 Question

16. If you’re Max: • Search can be stopped below any min node where β <= α of its max ancestor Max Principle

17. What should Min do? Min 4 k 9 4 n t d e p q r Beta 4 3 7 9 alpha 3

18. Min’s turn at node k • Left subtree of k has been explored • If min chooses n, max will choose d • So the best min can do by going left is 4. Call this β Now examine T and its left subtree P with a value of 7 • If min chooses T, the worst max can do is 7. • Why? • Q or R may be > 7. If either is Max will choose one of them. If not, max will choose 7. • So the worst max can do, if min goes right is 7. Call this α Situation

19. Should min explore Q and R • No • Max is guaranteed 7 if Min chooses T • But if min chooses N, max gets only 4 • Val(T) = max(7,val(Q), val(R)) >= 7 • Val(k) = min(4,val(T)) = 4 Question

20. If you’re min: • Search can be stopped below any max node where α >= β of its min ancestor Min Principle

21. Max’s turn • β is min’s guaranteed score (the worst min can do) • α is best max can do • Max principle • Search can be stopped below any min node where β <= α of its max ancestor • Min’s turn • α is max’s guaranteed score (the worst max can do) • β is best min can do • Min principle • Search can be stopped below any max node where α >= β of its min ancestor To Summarize

22. On White Board Examples 3 - 6

24. Similarity between • 2 and 2’ • Arguments for min/min principles (Slides 15, 19) AB Prune (Nilsson, p. 205)

25. Call • D the depth of the search space • ND the number of terminal nodes • B the branching factor • Best case AB performance: ND = 2BD/2 – 1 for even D ND = 2B(D+1)/2 + B(D-1)/2 for odd D Best Case Performance

26. Suppose B = 5, D = 6 w/out alpha/beta ND= 56 = 15625 w/alpha/beta ND= 2* 53 - 1 = 249 Approx. 1.6% of terminal nodes without AB prune Example

27. AB prune reduces branching factor B to B3/4 Suppose B = 5, Bab = 3.34 Suppose D = 6 Then ND = 56 = 15625 NDAB = 3.346 = 1388 ≈ 8.8% without AB prune Average Performance

28. Clear relationship between branching factor and the size of the search space Let T = number of nodes in a full binary tree T = 20 + 21 + 22 + … + 2D-1= 2D – 1 Easily proved through induction Binary Trees

29. T = B0 +B1 + B2 + … + BD = B(BD - 1)/(B-1) + 1 Also provable through induction Extend to arbitrary B

30. D is the depth of the search space • B is the average number of descendants at each level • Size of search space = B(BD -1)/(B-1) + 1 • Grows very fast • As branching factor increases • As depth increases • Combinatorial Explosion: search space grows too fast to be exhaustively searched • But we want to search deeply (large D) • Conclusion: reduce B through AB pruning To Sum Up