LOCATING MULTIPLE NASH EQUILIBRIA IN HIERARCHICAL GAMES

1. LOCATING MULTIPLE NASH EQUILIBRIA IN HIERARCHICAL GAMES WORLD CONFERENCE ON SOFT COMPUTING (WSC) 16 Andrew Koh Web version of this presentation: http://www.personal.leeds.ac.uk/~traako/wsc16/wsc16.html

2. Outline • Optimization & Equilibrium  Nash Equilibrium  Operation of NDEMO Algorithm to find NE • Applications to Transportation Networks – unique features • Specific Case Study: Competition between cities • Non cooperative Equilibrium •  Multiple Equilibria • Proposed Algorithm to detect Multiple Equilibria • Summary and Further work

3. Optimization • Fundamental principle in Economics: Rational Behaviour • leads directly to utility maximization for consumers and profit maximization for firms • Maximization is fundamental to theoretical underpinnings of economics

4. Optimization  Mutimodal Function  • Standard Differential Evolution (DE) • child vector generated by mutation and crossover • Selection: replace parent with child if it is fitter • Crowding DE: • Thomsen (2004) Modified Selection criteria: child replace most similar parent if it is fitter.  No other change made Consider :Max f(x)=sin6(5πx) See Next Slide for a demonstration

5. Crowding DE Example Max f(x)=sin6(5πx) x* = {0.1,0.3,0.5,0.7,0.9} f(x*)=1

6. EQUILIBRIUM Optimization describes a single agent When this single agent encounters others doing exactly the same  We get a GAME situation • From Newtonian Physics due to Cournot A body at rest stays at rest Concept of EQUILIBRIUM Fundamental concept in Economics and Social Sciences  For Games most relevant concept is NASH Equilibrium

7. Nash Equilibrium • Player strategy • Strategy Space for Player : • Combined Action Space for Players • notation to denote everyone else other than player Nash equilibrium (Nash 1950,1951)  If players cannot profitably deviate (no incentive), we must be at a NE

8. Cournot Nash Example Two producers {i=1,2} producing an homogenous product • no fixed costs, Variable costs per unit = 9 Demand function: p(Q)=24-Q, Q = q1+q2 Each producer maximizes profit: For firm 1  For firm 2  The first order conditions for producer i’s profit maximum Solve Simultaneously and obtain q1=5 and q2=5 as NE

9. NO INCENTIVE TO DEVIATE At NE no player cannot increase profit from deviating Compare solution {5 5} with say {2 3} At {5,5} profit = 25 for each Player 1 deviates and plays 2 Player 2 stays with 5 i.e. {2,5} Player 1 profit = 16 < 25 Player 2 deviates, plays 3 Player 1 stays with 5 i.e. {5,3} Player 2 profit = 21 < 25 No incentive to deviate Nash Equilibrium • Idea has been implemented in NDEMO algorithm Nash Dominance Evolutionary Multiplayer Optimization

10. Nash Domination – Lung and Dumitrescu (2008)/Koh (2012) Compare two strategy profiles (A vs B): • x => num players that are better off playing B when everyone else is playing A: • x = k(A,B) • y =num players that are better off playing A when everyone else is playing B: • y = k(B,A) We have the following relationships: x < y  ANASH DOMINATEs B  k(A,B) < k(A,B) y < x  BNASH DOMINATEs A  k(B,A) < k(A,B) x = y A does not Nash DOMINATE B and B does not NashDOMINATE A A & B are NON NASH DOMINATED wrt each other The closer A is to an NE the less likely someone can improve their payoffs by unilaterally playing B  NASH EQUILIBRIUM

11. Demonstration of NDEMO Refer to Koh (2012) for full algorithm Recall the Two producer game and Solution found by First order conditions algebraically was q1=5 and q2 = 5 NDEMO has no difficulty in detecting this NE

12. Applications to Transportation Networks Bilevel Programming now a new focus of research by EA pioneers such as Deb, Special Session in World Congress on Computational Intelligence in 2012 etc Focus of this paper: Multiple Leaders: Extension of Bilevel Problem

13. Behaviour of Leaders at Upper Level Fully Competitive Nash Equilibrium Problem Followers are road users routing according to e.g Wardrop’s equilibrium constraint EQUILIBRIUM PROBLEM WITH EQUILIBRIUM CONSTRAINT

14. Equilibrium Problem with Equilibrium Constraint (EPEC) Examples in transportation Equilibrium Constraint: Wardrop Equilibrium Constraint e.g. competing toll road operators e.g. public transport operators following deregulation e.g. Deregulated Pool Based Bidding Electricity Markets: See Koh (2012) in Applied Soft Computing

15. Competition between Cities ‘At one level, [competitiveness] is equated, usually loosely, with the performance’ of an economy, an absolute measure. At another, because it relates to competition, it implies a comparative element, with the implication that to be competitive, a city has to undercut its rivals or offer better value for money. In this sense, competitiveness is essentially about securing (or defending) market-share.’ Begg, (1999), ‘Cities and Competitiveness’, Urban Studies, 36 (5/6), p. 796.

16. Example: Game step • Two Local Authorities/player A & B • 12 one-way links • Standard BPR Function for all links • Route choice Through town centre or using Bypass Town Centre of A Town Centre of B

17. Example: Game set-up • OD matrix: reside in Suburb of A (zone 1) and work in Town Centre (TC) of A (zone 2) or work in TC B (zone 4) or reside in B (zone 5) and work in B or A • A set tolls on Link 1 and Link 6: cordon around TC A • B set tolls on Link 7 and Link 12: cordon around TC B

18. Player’s Payoff/Objective Tolls paid by traffic with origins in B to auth A Marshallian Measure of trips with origins in Authority A Authority A’s problem System cost of links in A Tolls paid by traffic with origins in A to auth B Wardrop User Equilibrium Constraint. Authority B’s problem

19. Behaviour of Leaders at Upper Level Fully Competitive Nash Equilibrium Problem Competing Cities Followers are road users routing according to e.g Wardrop’s equilibrium constraint EQUILIBRIUM PROBLEM WITH EQUILIBRIUM CONSTRAINT

20. Recall Cournot Nash Example Two producers {i=1,2} producing an homogenous product • no fixed costs, Variable costs per unit = 9 Demand function: p(Q)=24-Q, Q = q1+q2 Each producer maximizes profit: For firm 1  For firm 2  The first order conditions for producer i’s profit maximum Solve Simultaneously we can obtain q1=5 and q2=5

21. Contours Hence we need to gradients of the objective function of each Authority • Finite Difference Estimate of and • Vertical lines: • Horizontal lines: • Matlab “contours” • plot the points where these estimated gradients equal to 0 • 9 intersections

22. Multiple NE • We had 9 intersections but some can be immediately ruled out as NE • NE need both objectives to be maximized (so called second order condition) • Consider {504 for A ,107 for B}

23. Objective of both NOT Simultaneously Maximized {504, 107} What’s happening here? minimum maximum

24. Multiple NE • Recall 9 intersections • We can eliminate 5 of them this way but still 4 NE Algorithms, designed to detect 1 at the very most Mutiple NE example: Husband and wife want to go out for the evening: prefer to go together than alone. First figure in table is payoff to wife. Without more info, we cannot say which one is more likely! So best detect all

25. CrDENash Algorithm Child created (y) replaces most similar in population (x) if it Nash Dominates it or it is Nash Non Dominated

26. Successful Example 1: Recall Objective for Authority A (analogous for Authority B) We assumed α = 0.5 for both A and B (base level of α is 1) Travel time function for each link BPR function with power of 4 This is parameter set 1 as mentioned in paper

27. CRDENash successfully found both NE CrDENash successfully cluster around the 2 NE for this problem

28. Unsuccessful Example 2: • Use Parameter Set 2  network link travel cost : (BPR function) for bypass links: for local roads: the only changes made • CrDENash FAILED to locate 4 optima Converged to single NE

29. Summary • Crowding DE was formulated for tackling optimization problem • Optimization describes single economic actor’s behavior • Others doing the same simultaneously  equilibrium Game  search for equilibrium when players with conflicting objective NASH EQUILIBRIUM  NO INCENTIVE TO DEVIATE Koh (2012) proposed NDEMO  only find SINGLE Nash Equilibrium Aim to apply it to Equilibrium Problem with Equilibrium Constraint

30. Summary • Focus on Competition Between Cities: consider two cities attempting to toll traffic coming into its town centre. Each city aims to maximize welfare by tolling i.e. charging traffic entering • Multiple NE can be easily obtained…parameter set 1 and 2 only differ in terms of functional form of link travel time, α(revenue collection). But we obtain different possible NE depending on these very basic parameters that describe traffic network • Improve on NDEMO algorithm of Koh (2012) by using crowding to be able to detect multiple NE • Proposed CrDENash algorithm to embed suggested method of CDE (Thomsen (2004)) into NDEMO

31. Summary • CrDENash only partially successful • Cannot always locate all the NE • Further work required:  • archiving to store detected NE • Implementing Clustering algorithm • Using halton sequences or other pseudo random sequences to cover more of search space