Perfect Correlated Equilibria in Stopping Games

1. Perfect Correlated Equilibria in Stopping Games Yuval HellerTel-Aviv University (Part of my Ph.D. thesis supervised by Eilon Solan)http://www.tau.ac.il/~helleryu/ 3rd Israeli Game Theory Conference December 2008

2. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 2

3. Stopping Games(Undiscounted, Multi-player, Discrete time) • Finite set of players: I • Unknown state variable: wW (state space) • Filtration: F=(Fn) • At each stage n theplayers receive a symmetric partial information about the state : Fn(w)

4. Stopping Games(undiscounted, multi-player, discrete time) • Stage 1 - everyone is active • Stage n: • All active players simultaneously declare whether they stop or continue • A player that stops become passive for the rest of the game • Player’s payoff depends on the history of players’ actions while he has been active and on the state variable

5. Literature: 2-player zero-sum Stopping Games • Dynkin (1969) – introduction, value where simultaneous stops are not allowed • Neveu (1975) – value when each player prefers the other to stop • Rosenberg, Solan & Vieille (2001) – use of randomized strategies, value with payoffs’ integrability

6. Literature: 2-player non-zerosum Stopping Games • Existence of approximate Nash equilibrium when the payoffs have a special structure: Morimoto (86), Mamer (87), Ohtsubo (87, 91), Nowak & Szajowski (99), Neumann, Ramsey & Szajowski (02) • Recently, Shmaya & Solan (04) proved existence assuming only integrability • Multi-player stopping games: no existence results

7. Stopping Games - Applications Most applications in the literature: Payoffs: Specific assumptions, such as monotony Discount factor 2 players Multi-player variations are natural

8. Struggle of survival in a declining market • At each turn, each firm loses money • A firm can stay or exit the market for good • Partial production is inefficient • Market is more profitable with less firms • Which firms survive? What is the exit order? • Ghemawat & Nalebuff (1985)... • Steel market in 70’s and 80’s

9. Research & Development • Race for developing a patent • At each turn, continue spending money on research or leave the race • The first firm to complete the patent earns a lot • Stochastic function of spent money • Fudenberg & Tirole (1985)…

10. War of attrition • Attrition wars among animals: • Becoming the leader (alpha-male) • Territory • Maynard-Smith (1982), Nalebuff & Riley (1985)… • 2nd price auctions where all bidders pay • Krishna & Morgan (1997)…. • Political Sciences – lobbying • Bulow & Klemperer (2001)

11. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 11

12. Perfect Equilibrium • Nash equilibrium may be sustained by non-credible threats of punishment • Punisher receives a low payoff • The stronger concept of perfect equilibrium (Selten, 1965, 1975) has been studied. Examples: • Fine & Li (1989): uniqueness in discounted 2-player games with monotone payoffs • Mashiah-Yaakovi (2008) – existence of (d,e)-perfect equilibrium when simultaneous stops aren’t allowed

13. Correlated Equilibrium • Aumann (1974): An equilibrium in an extended game with a correlation device • Device D sends each player i a private signal miMi(M=Pi Mi) before the game starts according to mD(M) • The extended game G(D) • Consistent with Bayesian decision making (Aumann, 87) • Other appealing properties: computability, linear equations, closed and convex set

14. Correlated Equilibrium in Sequential Games • Two main versions: • Normal-form: signals are sent only before the game starts • Extensive-form: signals are sent at each stage • Equilibrium: normal-form  extensive-form • Correlation among players is natural in many setups: • Countries negotiate actions • Firms choose strategies based on market’s history • A manager coordinates the actions of his workers

15. Normal-Form Correlation (1) • Sometimes players may coordinate before play starts but coordination along the play is costly / impossible: • Example (1) - war of attrition in nature: • Commonly modeled as stopping games • Coordination before play starts is implemented by evolution of phenotype roles • E.g.: Shmida & Peleg, 1997

16. Normal-Form Correlation (2) Example (2) - News playing among day traders: Monthly employment report will be published at noon Several minutes elapse before market adjusts New information gradually arrives during that time Quick trading can be profitable See e.g., Christie-David, Chaudhry & Khan (2002) Traders of a firm can coordinate their actions in advance Coordination along the play is costly (time limit) Traders may have different payoffs 16

17. (d,e)-Perfect Correlated Equilibrium • d>0 – A bound for the probability of: • An event EW • Correlation device sends a signal inM’M • e>0 – A bound for the maximal profit a player can earn by deviating at any stage and after any history, conditioned on that w E and m M’ • Extending the definitions forfinite games: • Myerson (1986), Dhillon & Mertens (1996)

18. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 18

19. Main Result • For every d,e>0, a multi-player stopping game admits a normal-form uniform perfect correlated (d,e)-equilibrium with a universal correlation device • Uniform: An approximate equilibrium in any long enough finite game and in any discounted game with high enough discount factor • Universal device – doesn’t depend on game payoffs • Corollary: Uniform perfect correlated equilibrium payoff

20. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 20

21. Reductions • Terminating games: game terminates at the first stop • Tree-like games (Shmaya & Solan, 03):for every n, Fn is finite • A finite collection of matrix payoffs • Deep enough in the tree: with high probability any matrix payoff either: • Repeats infinitely often • Never occurs

22. Reductions Reductions require 2 properties from the equilibrium (d,e)-unrevealing - expected payoff of each player “almost”doesn’t change With probability of at-least 1-d, changes by less than e Universal - The correlation device D(G,e,d) depends only on |I| and e: D(G,e,d)=D(|I|,e) 22

23. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 23

24. Games on Finite Trees • Equivalent to an absorbing game: A stochastic game with a single non-absorbing state. 2 special properties: • Recursive game – Payoff in non-absorbing states is 0 • Single non-absorbing action profile

25. Games on Finite Trees • An adaptation of a result of Solan & Vohra (2002): • A game on a finite tree has one of the following: • Non-absorbing equilibrium (game never stops) • Stationary absorbing equilibrium. Adaptations: • Perfection • Limit minimal per-round terminating probability • A special distribution: allows to construct a correlated e-equilibrium. Adaptations: unrevealing, universal device

26. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 26

27. k1 Ramsey Theorem (1930) • A finite set of colors • Each 2 integers (k,n) are colored by c(k,n) • There is an infinite sequence of integers k1<k2<k3<… such that: c(k1,k2) =c(ki,kj) for all i<j 0 1 2 3 4 5 6 7 8 9 10 11 12

28. t1 Low probability Stochastic Variation of Ramsey Theorem (Shmaya & Solan, 04) • Coloring each finite sub-tree. • There is an increasing sequence of stopping times:t1<t2<t3<…, such that: Pr(c(t1,t2) =c(t2,t3) =….)>1-e

29. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 29

30. Equilibrium Construction • Each finite tree is colored according to: • Whether it has a non-absorbing perfect equilibrium, an absorbing perfect equilibrium, or a special distribution • The equilibrium payoff • The maximal payoffs when a player stops alone • If c implies that each game on finite tree has a perfect equilibrium, concatenate the equilibria to obtain an approximate perfect equilibrium of G

31. Equilibrium Construction Last case: c implies that a special distribution exists This allow to construct an approximate unrevealing perfect correlated equilibrium with a universal correlation device An adaptation of the protocol of Solan and Vohra (01) 31

32. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 32

33. Summary and Future Research • Summary: every multi-player stopping game admits an approximate normal-form uniform perfect correlated equilibrium with a universal correlation device • Future research: • Using this notion of equilibrium in the study of other dynamic games • Structure of uniform perfect correlated equilibrium payoffs in specific applications

34. Questions & Comments? • Y. Heller (2008), Perfect correlated equilibria in stopping games, mimeo. http://www.tau.ac.il/~helleryu/

35. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 35

36. Reduction to Terminating Games Proposition: Every game that stops immediately admits a (d,e)-unrevealing perfect correlated (d,e)-equilibrium with a universal correlation device Every stopping game admits the same kind of (d,e)-equilibrium 36

37. Proof Outline Induction on the number of players Given a stopping game G, we define an auxiliary terminating game G’: The payoff to I\S when a coalition S stops is the equilibrium payoff in the induced stopping game G’admits an unrevealing perfect correlated (d,e)-equilibrium with a universal correlation device Concatenation gives such an equilibrium in G 37

38. Tree-like Games Shmaya & Solan (2002) showed that any stopping game can be approximated by a tree-like stopping game, with the same set of approximate equilibria Small perturbations of the payoffs don’t change the set of approximate equilibria we can assume that the payoff process has a finite range Each set FnFn can be identified with a node in a tree 38

39. Tree-like Games – Shmaya & Solan’s Proof Outline • kth partition: • Discretization of the game: • Depth: k • Precision: e/2k • Refinement of all previous partitions • Defines the kth approximating game on a tree • The game on finite tree that begins on m and ends on l will be played on the m+l approximating game

40. Tree-like Games F1 F2 F4F4 F4 40

41. Deep Enough in the Tree G(Fn): The induced game that begins at the node Fn v1, v3, v5 occur infinitely often, all other vV do not occur at all Fn v1 v1 v1 v1 v1 v1 v1 41

42. Lemma - Induced Games • Let: G - a terminating game, t - a stopping time • Every induced game G(Fn), where Fn is in the range of t,admits an unrevealing perfect correlated (d,e)-equilibrium with a universal correlation device • Gadmits the same kind of (C·d,e)-equilibrium • Corollary: We can assume to be “deep enough” 42

43. Proof Outline • Until tthe players follow an equilibrium in a finite stopping game with absorbing states {Ft} with payoffs {xFt} - equilibria payoffs of G(Ft) • After t , players follow (d,e)-equilibrium of G(Ft) • Relying on that the equilibrium is unrevealing and with a universal correlation device • Illustration…. 43

44. Proof Outline V(|I|,e) - universal correlation device x7 x1 Ft x4 x2 x3 x5 x6 44

45. Games on Finite Trees gi: maximal payoff player i can get by stopping alone The special distribution h over (nodes · players): A stopping player i and a node with maximal payoff (Rii,n=gi) The distribution gives each player i at-least gi Each stopping player has a punisher j that stops when Rji,n<gi Allows to construct a correlated e-equilibrium 45

46. Stationary Absorbing Equilibrium: Adaptations Perfection - using a perturbed tree with e-probability to ignore players’ requests to stop Limiting the minimal per-round terminating probability p (adapting the methods of Shmaya & Solan, 2004) If there is a player i with a payoff below gi, thenp can’t be too small or player i stops when his payoff is gi Otherwise either case 3 applies, or there is a node where at-least 2 players stop with a non-negligible probability Recursive trimming of such nodes gives the needed limit 46

47. Last case: c implies that a special distribution hexists Let tik be the k-th time that player i’s maximal payoff occur with the requirement tik> tjk-1 for all i,j Using the fact that we are “deep enough” in the tree An approximate unrevealing perfect correlated equilibrium with a universal correlation device is constructed as follows… ` Equilibrium Construction: Protocol Description 47

48. Equilibrium Construction: Protocol Description A quitter i’ is secretly chosen according to the special distribution A number l’ is chosen uniformly in {1,T’} i’ receives the signal l’ A number l is chosen uniformly in {l’+1,l’+T} 1<<T<<T‘ The punisher of i’ receives the signal l Each other player receives a signal l+1 Approximate unrevealing perfect correlated equilibrium: each player stops at tl (when l is his signal) modulo 1+T+T’ 48

49. Introduction: Stopping games perfect correlated (d,e)-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction Summary 49