Introduction to Game Theory application to networks

1. Introduction to Game Theory application to networks Joy Ghosh CSE 716, CSE@UB 25th April, 2003

2. What is Game Theory ? • Study of problems of conflict and cooperation amongst independent decision makers • Formal way of analyzing interactions among a group of rational agents who behave strategically • Games of Strategy rather than Games of Chance! • Ingredients: • Players / decision makers • Choices / feasible actions / pure strategies • Payoffs / benefits / utilities • Preferences to payoffs

3. Some basic concepts • Group – Any game consisting of more than one player • with single player the game becomes a decision problem! • Interaction – Actions of one affects the other • else it would become simple sequence of independent decisions • Strategic – Players account for interdependence • Rationality – Players consistently opt for best choices • Common Knowledge • All players know that all players are rational • Equilibrium – a point of best shared interest for all

4. Classification of Problems • Static vs. Dynamic • In Dynamic problems the sequence of choices are relevant • Cooperative vs. Non-cooperative • In non-cooperative games players watch out for their own interests. In cooperative games, players form coalitions with shared objectives.

5. Decision Theory under Certainty • Decision problem (A, ≤) • Finite set of outcomes A = {a1, a2, …. an} • Preference relation ‘≤’ : a ≤ b => ‘b’ is at least as good as ‘a’ • Completeness – for all a, b in A, either a≤b, or b≤a • Transitivity – if a≤b and b≤c, then a≤c • Utility function u: A  R (consist with preference relation) • For all a,b in A, u(a) <= u(b) iff a ≤ b • Rational decision maker tries to maximize utility • Choose outcome a* in A s.t. for all a in A, a ≤ a*

6. Decision Theory under Uncertainty • Lottery L = {(a1, p1), (a2, p2), …… (an, pn)} • Σiεn pi = 1, 0 ≤ pi≤ 1 • Outcome ai occurs with probability pi • Infinitely many different possible lotteries • Large number of lottery comparisons • Preference relation unobservable • “Under additional restrictions on preferences over lotteries there exists a utility function over outcomes such that the expected utility of a lottery provides a consistent ranking of all lotteries” - John von Neumann and Oscar Morgenstern • u(L) = Σiεn u(ai).pi

7. Allais Paradox • Lottery A • A1 (sure win of 3000) vs. A2 (80% chance to win 4000) • A1 strictly preferred to A2 • Lottery B • B1 (90% chance to win 3000) vs. B2 (70% chance to win 4000) • B1 might still be preferred to B2 • Lottery C • C1 (25% chance to win 3000) vs. C2 (20% chance to win 4000) • Most people start preferring C2 over C1 even though these two lotteries are variations of the 1st pair in Lottery A

8. Game Theory – multi agent decision problem • A normal (strategic) form game G consists of: • A finite set of agents D = {1, 2, ….. N} • Strategy sets S1, S2, ... SN = set of feasible actions for agents • Strategy profile S = S1 x S2 x ... x SN • Payoff function ui : S  R (i = 1, 2, …. N) • NOTE: The preference an agent has is to the outcome and not to the individual action

9. Some standard games in normal form Matching Pennies Tough vs. Chicken Row: gains if pennies matchCol: gains if there is no match A game of head-on collision

10. Iterated Deletion of Dominated Strategies • Common Knowledge • assumptions about other people’s rational behavior • Some more definitions: • S-i = S1 x S2 ....x S(i-1) x S(i+1) ....x SN (strategy sets of others) • utility function of player i for a given pure strategy: • ui (s  S) = ui (si, s-i) • Belief i of agent i = probability distribution over S-i • for pure strategies the probability distribution is a point distribution • Player i is rational with beliefs i if: • si arg max s-iS-i ui (s’i, s-i). i(s-i) for all s’i  Si • Note: as igets fixed, player i faces a simple decision problem

11. Dominated Strategies • Strongly Dominated • si  Si is strictly dominated if: •  s’i  Si s.t. ui(s’i, s-i) > ui(si, s-i) for all s-i  S-i • Weakly Dominated • if the inequality is weak () for all s-i  S-i, and strong (>) for at least one • Rational players do not play strongly dominated strategy

12. Iterated Dominance (deletion) • M is strictly dominated by L. Rational column player ignores M • If row player knows column player is rational, he will ignore D • If column player knows the above, then he will choose L • With common knowledge about rationality of players U,L is the outcome

13. Iterated Dominance – Formal Definition The game is solvable by pure strategy iterated strict dominance only if S contains a single strategy profile

14. Does the order of elimination matter? • In games that are solvable by iterated dominance, the speed and order or elimination doesn’t matter. • This is however not true for weakly dominated strategies. Deletion Sequence #1: T, L - (2,1) is the playoff Deletion Sequence #2: B, R - (1,1) is the playoff

15. Nash Equilibrium for pure strategy • No incentive for a player to deviate from his best response to his/her belief about other player’s strategy • U,L was the NE in the example of strongly dominated strategies • Definitions: • A strategy profile s* is a pure strategy Nash equilibrium of G iff • ui (si*, s-i*) ≥ ui (si, s-i*) for all players i and for all si Si • A pure strategy NE is strict if the inequality is strict • There can be multiple Nash equilibria for a particular G • Two people trying to meet at one out of 2 places (NY Game!)

16. Do pure strategies always work? • Most games are not solvable by dominance • Coordination game, zero-sum game • Penny matching Game • Whatever pure strategy one player chooses, the other can win by choosing a better strategy • Players have to consider mixed strategies

17. Mixed strategies - definitions • Mixed strategy i for player i is a probability distribution over his strategy space Si • i : Si R+ s.t. siSii(si) = 1 • i is the set of probability distributions on Si •  = 1 x 2 x … x N • Player i’s expected payoff with mixed strategies • ui (i, -i) = si, s-i ui(si, s-i) i(si) -i(s-i)

18. Mixed strategies – more definitions • Mixed strategy NE of G is a *   such that: • ui (i*, -i*) ≥ ui (i, -i*) for all i and for all i i • In a finite game, support of a mixed strategy i: • supp (i) = { si Si | i (si) > 0 } • Proposition • if i* is a mixed strategy NE and si’, si’’  supp (i*), then ui (si’, -i*) = ui (si’’, -i*)

19. Proof of previous proposition

20. A mixed strategy example game • There is no pure strategy NE • Row plays U with probability  • Column plays L with probability  • Players need to be indifferent to their choice of strategies: • u1 (U, 2*) = u1 (D, 2*) •  = 2 (1 - ) • u2 (L, 1*) = u2 (R, 1*) •  + 2 (1 - ) = 4  + (1 -  ) •  = 1/4 ;  = 2/3 • Unique mixed NE • 1* = 1/4 U + 3/4 D • 2* = 2/3 L + 1/3 R

21. Two People Zero Sum Games – Pure Strategy • One player’s winnings is another player’s loss! • Each player does the following: • For each of his/her strategies, compute the maximum of losses that he could incur. • Choose the strategy with the minimum max loss

22. Example 2 people 0 sum Game • Row is player 1; Column is player 2 • If aij > 0, player 1 wins, else player 2 • Player 1: • i* = arg maxi (minj (aij)) • V(A) = minj a i*j is the gain-floor for the game A • In this case, V(A) = -2, with i*  {2, 3} • Player 2: • j* = arg minj (maxi (aij)) • (A) = maxi a ij* is the loss-ceiling for the game A • In this case, (A) = 0, with j* = 3

23. Two People Zero Sum Game – Mixed Strategy • If (A) = V(A) then A has a point of equilibrium • Else we need to develop mixed strategy • Consider the following game: • For player 1, we have V(A) = 0, with i* = 2 • For player 2, we have(A) = 1, with j* = 2 • No saddle point or equilibrium • Let players 1, 2 play strategy i with probability xi, yi

24. Best Choice Analysis

25. Optimization Problem • In a nutshell, the players are solving the following pair of dual linear programming problems • Player 1 • Player 2

26. Application to networks • Formulation for n users competing for fixed resources • Generic non-cooperative game • Each user has access control /parameter n • Each user receives certain amount n() of network resources •  (1, 2, …. n) • n [0, nmax] for some nmax > 0 • n () is a non-decreasing function of n • n (.) is continuous in n=1N [0, nmax] and is differentiable with respect to n • If n = 0, n () =0 for all  • n () maybe interpreted as the QOS received by the nth user

27. Formulation (contd.) • Let network charges be fixed at M / unit resources • Each user tries to maximize his/her net utility • Un(n()) – M.n() • Un is non-decreasing and Un(0) = 0; • U’n is non-increasing, i.e. Un is concave • Maximum net benefit of nth user • yn = arg max (Un(n()) – M.n()) = (U’n)-1.(M) • Action of nth user • Modify n to make received QOS n() equal to desired yn

28. User iterations and equilibrium • After the jth iteration/step, access parameter of user n: • nj+1 = min (G (yn, n(j), nj), nmax) • G (y, , ) •  , if  = y • > , if  < y • < , if  > y • Nash equilibria • A fixed or equilibrium point of this iteration is any *  [0, nmax] • n* = min (G (yn, n(*), n*), nmax) • By Brouwer’s fixed point theorem there exists at least one such fixed point.

29. Non-cooperative game for circuit switched network • N users compete for K circuits • nth user’s connection setup request is Poisson with intensity n and arbitrary holding time distribution with mean 1/ n • Total traffic intensity:   .1/ • Aggregate arrival rate   n=1 N n • Mean holding time over all connections 1/ = n=1N1/nn/ • Hence,  = n=1Nn/n • Per user connection blocking probability (Erlang’s form) • K()  (K/K!) / (k=0Kk/k!)

30. Formulation leading to equilibrium • Net arrival rate of nth user: • n(1 - K()) • Mean number of occupied circuits for the nth user: • n()  1/nn(1 - K(())) • Thus, n and  depend on all arrival rates  • Iteration using multiplicative increase and decrease • nj+1 = min {yn/n . n, nmax} • or, nj+1 = min {yn.n / (1 - K((j))) , nmax} • By previous formulation we can find an equilibrium!

31. References • Game Theory .NET - college lecture notes (http://www.gametheory.net) • IE675: Game Theory - Dr. Wayne Bialas, Dept. of IE, SUNY Buffalo, (http://www.acsu.buffalo.edu/~bialas/IE675.html ) • “Computational Finance: Game and Information Theoretic Approach” – Dr. B. Mishra, Dept. of CS, NYU (http://www.cs.nyu.edu/mishra/COURSES/GAME/game.html) • Introduction to Game Theory – Markus Mobius, Dept. of Economics, Harvard (http://icg.fas.harvard.edu/~ec1052/lecture/index.html) • Infocom 2003 - “Nash equilibria of a generic networking game with applications to circuit-switched networks” - Youngmi Jin and George Kesidis, Dept. of EE & CS, Pennsylvania State University.