Partial-information Games with Reachability Objectives

1. Partial-information Games with Reachability Objectives KrishnenduChatterjee Formal Methods for Robotics and Automation July 15, 2011 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA

2. Games on Graphs • Games on graphs. • Perfect-information graph games • Zermelo’s theorem about Chess in 1913 • From every configuration • Either player 1 can enforce a win. • Or player 2 can enforce a win. • Or both players can enforce a draw.

3. Chess: Games on Graph • Chess is a game on graph. • Configuration graph.

4. Graphs vs. Games Two interacting players in games: Player 1 (Box) vs Player 2 (Diamond).

5. Game Graph

6. Game Graphs • A game graph G= ((S,E), (S1, S2)) • Player 1 states (or vertices) S1 and similarly player 2 states S2, and (S1, S2) partitions S. • E is the set of edges. • E(s) out-going edges from s, and assume E(s) non-empty for all s. • Game played by moving tokens: when player 1 state, then player 1 chooses the out-going edge, and if player 2 state, player 2 chooses the outgoing edge.

7. Game Example

8. Game Example

9. Game Example

10. Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1 D(S). • ¼: S*S2! D(S).

11. Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1! D(S). • History dependent and randomized. • History independent: depends only current state (memoryless or positional). • ¾: S1! D(S) • Deterministic: no randomization (pure strategies). • ¾: S*S1! S • Deterministic and memoryless: no memory and no randomization (pure and memoryless and is the simplest class). • ¾: S1! S • Same notations for player 2 strategies ¼.

12. Objectives • Objectives are subsets of infinite paths, i.e., õ S!. • Reachability: there is a set of good vertices (example check-mate) and goal is to reach them. Formally, for a set T if vertices or states, the objective is the set of paths that visit the target T at least once.

13. Applications: Verification and Control of Systems • Verification and control of systems • Environment • Controller M satisfies property (Ã) E C

14. Applications: Verification and Control of Systems • Verification and control of systems • Question: does there exists a controller that against all environment ensures the property. C M E satisfies property (Ã) || ||

15. Applications: Systems for Specification • Synthesis of systems from specification • Input/Output signals. • Automata over I/O that specifies the desired set of behaviors. • Can the input player present input such that no matter how the output player plays the generated sequence of I/O signals is accepted by automata ? • Deterministic automata: Games.

16. Game Models Applications -synthesis [Church, Ramadge/Wonham, Pnueli/Rosner] -model checking of open systems -receptiveness [Dill, Abadi/Lamport] -semantics of interaction [Abramsky] -non-emptiness of tree automata [Rabin, Gurevich/ Harrington] -behavioral type systems and interface automata[deAlfaro/ Henzinger] -model-based testing [Gurevich/Veanes et al.] -etc.

17. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

18. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

19. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

20. Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. • Fix-point X T

21. Reachability Games • Winning set for a partition: Determinacy • Player 1 wins: then no matter what player 2 does, certainly reach the target. • Player 2 wins: then no matter what player 1 does, the target is never reached. • Memoryless winning strategies. • Can be computed in linear time [Beeri 81, Immerman 81].

22. Chess Theorem • Zermelo’s Theorem Win1 Win2 Both draw

23. Partial-information Graph Games

24. Why Partial-information • Perfect-information: controller knows everything about the system. • This is often unrealistic in the design of reactive systems because • systems have internal state not visible to controller (private variables) • noisy sensors entail uncertainties on the state of the game • Partial-observation Hidden variables = imperfect information. Sensor uncertainty = imperfect information.

25. Partial-information Games • A PIG G =(L, A, , O) is as follows • L is a finite set of locations (or states). • A is a finite set of input letters (or actions). • µ L £ A £ L non-deterministic transition relation that for a state and an action gives the possible next states. • O is the set of observations and is a partition of the state space. The observation represents what is observable. • Perfect-information: O={{l} | l 2 L}.

26. PIG: Example b a a,b b a

27. New Solution Concepts • Sure winning: winning with certainty (in perfect information setting determinacy). • Almost-sure winning: win with probability 1. • Limit-sure winning: win with probability arbitrary close to 1. • We will illustrate the solution concepts with card games.

28. Card Game 1 • Step 1: Player 2 selects a card from the deck of 52 cards and moves it from the deck (player 1 does not know the card). • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret card or goes back to Step 2 a. • Player 1 wins if the guess is correct.

29. Card Game 1 • Player 1 can win with probability 1: goes back to Step 2 a until all 51 cards are seen, and plays uniformly at random for choosing cards till then. • Player 1 cannot win with certainty: there are cases (though with probability 0) such that all cards are not seen. Then player 1 either never makes a guess or makes a wrong guess with positive probability.

30. Card Game 2 • Step 1: Player 2 selects a new card from an exactly same deck and puts is in the deck of 52 cards (player 1 does not know the new card). So the deck has 53 cards with one duplicate. • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret duplicate card or goes back to Step 2 a. • Player 1 wins if the guess is correct.

31. Card Game 2 • Player 1 can win with probability arbitrary close to 1: goes back to Step 2 a for a long time and then choose the card with highest frequency (choosing uniformly at random till the choice is made). • Player 1 cannot win probability 1, there is a tiny chance that not the duplicate card has the highest frequency, but can win with probability arbitrary close to 1, (i.e., for all ² >0, player 1 can win with probability 1- ², in other words the limit is 1).

32. Sure winning for Reachability • Result from [Reif 79] • Memory is required. • Exponential memory required. • Subset construction: what subsets of states (knowledge) player 1 can be. Reduction to exponential size perfect-information games. • EXPTIME-complete.

33. Partial-information Games a b a • In starting play a. • In yellow play a and b at random. • In purple: • if last was yellow then a • if last was starting, then b. • Requires both randomization and memory a b a b a a b a b

34. Almost-sure winning for Reachability • Result from [CDHR 06, CHDR 07] • Standard subset construction fails: as it captures only sure winning, and not same as almost-sure winning. • More involved subset construction is required. • EXPTIME-complete.

35. Almost-sure winning for Reachability • Brief Idea of the proof. • Suppose GOD gives us the winning knowledge sets. • Given a knowledge cannot play an action that is unsafe (takes out of the winning knowledge set). • Must play safe actions. • Play all safe actions uniformly at random.

36. Almost-sure winning for Reachability • Brief Idea of the proof. • Suppose GOD gives us the winning knowledge sets. • What GOD can do we can do by non-determinism. • This gives NEXPTIME algorithm, for EXPTIME algorithm see [CDHR 07].

37. Summary: Theory of Graph Games

38. Limit-sure winning for Reachability • Limit-sure winning for reachability is undecidable [GO 10, CH 10]. • Reduction from the Post-correspondence problem (PCP).

39. Limit-sure winning for Reachability • The undecidable result is first proved for probabilistic automata. • Probabilistic automata are special case of blind stochastic games. • For partial-information games, stochastic transitions can be simulated by deterministic transition [CDGH 10]. • Consequently we obtain the undecidability result.

40. Summary: Theory of Graph Games

41. Conclusion and Related Problems • Theory of graph games • Perfect-information and partial-information games. • Different solution concepts and different complexity. • Several algorithmic questions open. • Partial-observation stochastic games with pure strategies (Working manuscript of CD 11) • Many surprising results (such as non-elementary complete memory bounds).

42. Conclusion and Related Problems • Partial information games • Problem with clear practical motivation. • Challenging to establish the right frontier of complexity. • Important generalization of perfect-information games. • Unfortunately, undecidable and also high complexity. • Looking for new definitions (of sub-classes).

43. Collaborators • Laurent Doyen • Hugo Gimbert • Thomas A. Henzinger • Jean-Francois Raskin

44. The end Thank you ! Questions ?