Game Theoretical Insights in Strategic Patrolling: Model and Analysis

1. Game Theoretical Insights in Strategic Patrolling: Model and Analysis Nicola Gatti – ngatti@elet.polimi.it DEI, Politecnico di Milano, Piazza Leonardo da Vinci 32, Milano, 20133, Italy

2. Topic, Results, and Outline • Topic • Study of strategic models for capturing patrolling situations in presence of opponents • Main results • Modeling result: • Problems in the current state-of-the-art • Proposal of an alternative model • Computational result: • Exploitation of game theoretical analysis for reducing the solving algorithm complexity • Outline • Strategic patrolling state-of-the-art • Proposal of an alternative model • Towards integration between game theoretical analysis and algorithmic game theory • Conclusions and future works

3. Game Theory Groundings for Strategic Patrolling • Definition of game • Protocol: rules of the game (e.g., number of players, sequential structure, available actions) • Strategic-form games: the players act simultaneously (e.g., rock-paper-scissors) • Extensive-form games: the players act according to a given sequential structure (e.g., chess) • Strategies: players’ behavior in the game • Solution: a strategy profile σ = (σ1, …, σn) that is somehow in equilibrium • Nash equilibrium: the players act simultaneously without meeting themselves before playing the game [Nash, 1950] • Leader-follower equilibrium: a player can commit to a specific strategy and the follower acts on the basis of the commitment [von Stengel and Zamir, 2004]

4. von Neumann’s Hide-and-Seek Game S 1 2 3 H 4 5 6 S H 7 8 9 H

5. Paruchuri et al.’s Strategic Patrolling (1) 1 G 2 3 4 5 6 7 8 9 R

6. Paruchuri et al.’s Strategic Patrolling (2) • Assumptions: • Time is discretized in turns • Time needed by the guard to patrol one area is exactly 1 turn • Time needed by the guard to move between two areas is negligible with respect to time needed to patrol an area • Time needed by the robber to rob an area is d turns • The robber can observe the strategy of the guard • Game protocol: • Two–player: • Guard • Robber • General–sum: each player assigns each area and the robber’s caught a value • Strategic–form: the players act simultaneously • Actions: • Guard: a route of d areas, e.g. <1, 2, …, d> • Robber: a single area

7. Paruchuri et al.’s Strategic Patrolling (3) • Solution concept: leader-follower equilibrium • Strategies: the guard randomizes over a portion of the actions, while the robber follows a pure strategy • Multiple types: the payoffs of the robber could be known with uncertainty by the guard • By Harsanyi transformation: the robber can be of different types (each type has a specific payoff) according to a given probability distribution • Solving algorithms: • Multi Linear Programming [Conitzer and Sandholm, 2005] • Mixed Integer Linear Programming [Paruchuri et al., 2008]

8. Problems in Paruchuri et al.’s Strategic Patrolling (1) • A simple setting • 3 areas • 1 type • Two turns are needed by the robber to rob an area (d=2) • Each player has the same evaluations over the areas

9. Problems in Paruchuri et al.’s Strategic Patrolling (2) Guard’s optimal strategy (.16 <1,2>, .16 <2,1>, .16 <1,3>, .16 <3,1>, .16 <2,3>, .16 <3,2>) G realization <3,1> realization <1,2> 1 G 2 3 G R The robber’s expected utility is -.33 R Robber’s optimal strategy (2)

10. Problems in Paruchuri et al.’s Strategic Patrolling (2) Guard’s optimal strategy (.16 <1,2>, .16 <2,1>, .16 <1,3>, .16 <3,1>, .16 <2,3>, .16 <3,2>) G realization <1,2> realization <3,2> 1 G 2 3 G R The robber’s expected utility is .33 R

11. Problems in Paruchuri et al.’s Strategic Patrolling (3) • The model by Paruchuri et al. does not consider all the possible implications due to the observation of the robber • According to the assumption of observation, the robber can enter an area when the guard is patrolling and not exclusively when the guard starts to patrol a route

12. An Alternative Strategic Patrolling Model • The “natural” model is an extensive-form game wherein • Guard: the next area to patrol • Robber: the area to enter or wait • In this work we search for a strategic-form model alternative to Paruchuri et al.’s model • The proposed model is a strategic-form model wherein • Guard: the next area to patrol • Robber: the area to enter and the guard’s strategy will be the same at each turn • In this way the robber cannot improve its expected utility by waiting • In this model no “consistency“problem there is (the proof can be found in the paper)

13. Searching for a Nash Equilibrium • We use the strategic patrolling as case study for the integration of game theoretical analysis and algorithmic game theory • Idea • Game theoretical analysis allows one to derive some insights • Singularities: some strategy profiles are never of equilibrium independently of the values of the parameters (payoffs) • Regularities: some strategy profiles are of equilibrium with a probability higher than others • These insights can be exploited to improve searching efficiency and to make hard problems affordable

14. One Robber Type Analysis • Proposition 1: Independently of the number of the robber’s types, at the equilibrium the guard will randomize over all the possible actions • On the basis of Proposition 1, except for a null-measure subspace of the parameters, with one type of robber the Nash equilibrium: • Is unique, and • Prescribes that both the guard and the robber will randomize over all their available actions • In this case the Nash equilibrium can be computed in closed form as a single problem of linear programming

15. More Robber Types Analysis (1) • With more types, the equilibrium cannot be computed in closed form • Anyway, game theoretical insights can be exploited to reduce the complexity of the search • Searching in the space of the supports • A complete method for searching a Nash equilibrium is to enumerate all the possible strategy supports and check them one by one (A strategy support is the set of actions over which agents randomize with a strict positive probability) • Anyway, such a space rises exponentially in the number of players’ actions and then heuristics are needed • [Porter et al., 2005] provides some heuristics for ordering the supports and shows that their approach is more efficient than Lemke-Howson algorithm

16. More Robber Types Analysis (2) • By Proposition 1, the support of the guard will be the whole set of actions • The supports of all the robber’s types can depict as a matrix M = • By game theoretical analysis we can: • Reduce the space of the matrices M • Produce an ordering where the first Ms are the most probable to lead to an equilibrium

17. Experimental Results • We have studied random settings with 4, 5, 6, 7 areas and different number of robber’s types • Our approach outperforms Porter et al. approach in term of computational time, dramatically reducing the space of the search • Our approach outperforms Multi-LP algorithm, although the computation of a Nash equilibrium is harder than the computation of a leader-follower equilibrium

18. Conclusions and Future Works • Conclusions • Analysis of state-of-the-art model of strategic patrolling • Proposal of a strategic model in normal-form • Attempt to exploit game theoretical analysis to improve the algorithm efficiency • Future works • Patrolling models and solving algorithms • Exploiting game theoretical analysis in algorithmic game theory

19. Thank you for your attention!