Games in the real world

1. Many real-world situations and problems can be viewed as games of a sort: Players have a choice of actions, Players have conflicting goals, Players may move sequentially or simultaneously, Alliances may prosper, Treachery may occur, Understanding of the goals of others may be useful in predicting their actions and planning actions of one’s own. Parlour games offer environments in which various kinds of simplification can be made in order to focus attention on particular AI issues: perception, representation, reasoning, learning, opponent modelling, and risk assessment. Stock Market War Passing legislation Hustling (“Bangneki” in Go) Cartels Fight for market share Biological evolution Industrial relations Democratic elections Takeover negotiations Games in the real world http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

2. Some Game Theory • Game theory is the formal study of conflict and cooperation, applicable when several “players” may act independently. • Much of the terminology in use today was introduced in 1944 in the book • Theory of Games and Economic Behaviour, by von Neumann & Morgenstern. • It is an important analytic tool for economics, and is also widely applied in politics, war, sociology, psychology, evolutionary biology. • It allows one to reason about the strategies that rational players may adopt, whether acting competitively, cooperatively, or in isolation of others, in deciding upon courses of action intended to yield some payoff: even when the payoff is merely the winning of a parlour game. http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

3. Adversarial Games Of Strategy • Players (decision maker) • Choices (feasible actions) • Payoffs (benefits - or penalties -that might accrue) • Combining whatever factors are needed to make players risk neutral • Objectives (preferences among payoffs) • When a player must make a choice this should be done • Not by blind chance • But by reasoning about which action is best • Though curiously this may involve chance after all: see mixed strategies • Using knowledge of what is good for the player • Also using knowledge of what is good for other players, esp. adversaries http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

4. Player 1 p p p r r r s s s paper scissors rock Player 2 (0, 0) (+1, -1) (-1, +1) (-1, +1) (0, 0) (+1, -1) (+1, -1) (-1, +1) (0, 0) Static vs. Dynamic games • In dynamic games the order of moves (performance of choices by players) is important:eg shopkeeper sets a price, customer then decides whether to buy. • In static games, choices are unordered (perhaps simultaneous): like Roshambo, Diplomacy. • Games may be drawn in extensive form: Consider (dynamic version) Roshambo: http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

5. Player 1 p p p r r r s s s paper scissors rock Player 2 (0, 0) (+1, -1) (-1, +1) (-1, +1) (0, 0) (+1, -1) (+1, -1) (-1, +1) (0, 0) Games of perfect information, games of imperfect information • If Player 2 knows Player 1’s move when making his own choice, the game is one of perfect information (Roshambo is then a PIG to play for player 1!). • When the game is static, moves are simultaneous, player 2 has imperfect information. In extensive form, ovals denote information sets - nodes in the game tree which cannot be distinguished by the player making a choice. http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

6. Strategies, and Strategic Form • A strategy is a function which determines which choice a player makes at every choice point. (Even those which do not arise on a particular occasion.) • Games may be described also in an alternative, so-called strategic form: A game is zero-sum iff at each terminal of the extensive form, the sum of the players’ payoffs is zero. http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

7. Two-Person Zero Sum Games & Normal Form • “Normal Form” is a compact notation for the payoffs of two players in a zero sum game: • Roshambo = • The cells of the matrix show the payoffs for player1; the payoff for player2 is trivially deduced. http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

8. Preferences over payoffs • Suppose now one does not just win or lose, but instead hurts the loser using the winner’s chosen item (and gets enjoyment proportional to the hurt inflicted) • Ow-ha-ha = • How should the players behave? • One thing player1 could do is ask • for each move I could make, what is for me the worst thing player2 could do? http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

9. Max Min • For each row i, player1 looks for the minimum payoff: mini Gi,j • He then looks for the move(s) i* which make this as good as possible: • Gi*,j = arg maxi minj Gi,j • In this case, i* = {2} (rock) • Similarly, player2 looks for the minimum payoff in each column j, and picks the j* that maximises the minimum payoff for player2: equivalently, minimises the maximum payoff for player1 (zero sum means p1’s gain = p2’s loss) • Gi,j* = arg minj j maxi Gi,j • In this case, j* = {2} (rock again) http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

10. Stable and Unstable Solutions • In this game, and in any zero-sum game, trying as hard as possible to be sure to hurt the other is equivalent to trying to profit as much as one can. • Each player may rationally do this by selecting the move with the least potential for loss. • But if one player knows that the other is doing this, they have a better choice: player 2 for example will do better with move 1 (paper). • But if player 1 knows that, he will prefer move 3 (scissors). And so on … • This is an unstable solution: a solution is stable if no player wants to move away unilaterally from the solution. At such a stable solution, a game is in equilibrium. http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

11. Gain Floor, Loss Ceiling, Saddle Point, Value of Game • The Gain Floor of a two-player zero-sum game A, written (A), is the worst outcome for player 1 if he picks his best strategy i*: (A) =min j A i*,j • The Loss Ceiling (likewise) is the best outcome for player 1 (hence worst for player 2) if player2 picks his best strategy j*: (A) = max i A i,j* • When Gain Floor equals Loss Ceiling, the game is said to have a saddle point. • In this case, both players will do no better by unilaterally picking a different strategy than i*, j*. This pair of strategies is a stable solution to the game. • A kind of cooperation emerges, even though the game is adversarial. • The Value of game A is the payoff of its stable solution, (A) = (A) = Value(A) http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

12. Games without saddle points • Zero-Sum games do not always have saddle points: eg Ow-ha-ha: • Here i* = 2, so (Ow-ha-ha) = -1, • while j* = 2, (Ow-ha-ha) = +1 • The pair of strategies {i=2, j=2} is not stable. A player who can find out what the other will do can get a better result from a different strategy. http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

13. Mixed strategies • In a game with a saddle point, each player has a best strategy (or a set of them). • The strategies remain best even if the other players knows what they are. • In a game without a saddle point, and hence without a stable solution, • each pure strategy one player might adopt, if known to an opponent, can be refuted • A mixed strategy involves a random choice among pure strategies • Each pure strategy is picked with its own particular probability • Mixed strategies may be found which, even if known, cannot be refuted http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

14. Selecting the mixed strategy • A player P will pick among the possible pure strategies S1…Sn using probabilities X1…Xn, such that  Xi = 1 • P should pick these probabilities to give the highest expected payoff. • For the very simple game G = • For player 1: 4 If p2 picks column 1 2 0 If p2 picks column 2 -2 If X1= 0 & X2= 1 If X1=1 & X2=0 X1=1/7, X2=6/7 http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

15. Selecting the mixed strategy - 2 • With pure strategies mixed in the right proportion, player 2 gains no systematic advantage from choosing either pure strategy even when he knows the mixed strategy of player 1. • Player 2 can pick a mixed strategy of probabilities Y1, Y2 according to the same principles: 4 If p1 picks row 1 2 0 If p1 picks row 2 -2 If Y1= 0 & Y2= 1 If Y1=1 & Y2=0 Y1=4/7, Y2=3/7 http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

16. Stability with mixed strategies • For each player, there is an optimal mixed strategy. • One player’s choice of strategy does not depend either • on knowing the other player’s strategy • Keeping the strategy secret • The expected payoff for one player does not depend on the other’s choice of strategy. The Value of the game is fixed. There is a kind of stability. • These observations generalise to the case of multiple pure strategies per player. • Von Neumann’s Minimax Theorem shows that a pair of mixed strategies can always be found for which the players have the same expected value for the game (which is not to say they can both expect to gain!) http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles

17. Some game-theory-reading • [3] http://www.sfb504.uni-mannheim.de/glossary/game.htm • [39] http://www.cdam.lse.ac.uk/Reports/Files/cdam-2001-09.pdf http://csiweb.ucd.ie/Staff/acater/comp4031.htmlArtificial Intelligence for Games and Puzzles