Staying alive in the dark

1. Staying alive in the dark Laurent DoyenLSV, ENS Cachan & CNRS joint work with Aldric Degorre1, Raffaella Gentilini1, Jean-François Raskin1, Szymon Torunczyk2 1Uni. Brussels, Belgium 2ENS Cachan, France

2. Synthesis problem Specification avoid failure,ensure progress,etc. Correctness relation

3. Synthesis problem System - Model Specification avoid failure,ensure progress,etc. Correctness relation Solved as a game – system vs. environment solution = winning strategy This talk: quantitative games (resource-constrained systems)

4. Energy games(staying alive)

5. Energy games (CdAHS03,BFLM08) Maximizer Minimizer positive weight = reward play: (1,4) (4,1) (1,4) (4,1) … weights: -1 +2 -1 +2 … energy level: 1 0 2 1 3 2 4 3 …

6. Energy games (CdAHS03,BFL+08) Maximizer Minimizer positive weight = reward play: (1,4) (4,1) (1,4) (4,1) … weights: -1 +2 -1 +2 … energy level: 1 0 2 1 3 2 4 3 … 1 Initial credit

7. Energy games Strategies: Maximizer Minimizer play: Infinite sequence of edges consistent with strategies and outcome is winning if: Energy level

8. Energy games Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative.

9. Energy games Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

10. Energy games c0=2 c0=2 c0=1 c0=0 A memoryless strategy is winning if all cycles are nonnegative when is fixed. Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

11. Energy games c0=2 c0=2 c0=1 c0=0 A memoryless strategy is winning if all cycles are nonnegative when is fixed. Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

12. Algorithm

13. Algorithm for energy games Initial credit is useful to survive before a cycle is formed Length(AcyclicPath) ≤ Q Q: #statesE: #edgesW: maximal weight

14. Algorithm for energy games Initial credit is useful to survive before a cycle is formed Length(AcyclicPath) ≤ Q Q: #statesE: #edgesW: maximal weight Minimum initial credit is at most Q·W

15. Algorithm for energy games The minimum initial credit is such that: in Maximizer state q: in Minimizer state q: Compute successive under-approximations of the minimum initial credit.

16. Algorithm for energy games 0 0 Fixpoint algorithm: - start with 0 0

17. Algorithm for energy games 0 1 0 2 Fixpoint algorithm: - start with - iterate at Maximizer states: 0 1 0 0 at Minimizer states:

18. Algorithm for energy games 0 1 2 0 2 2 Fixpoint algorithm: - start with - iterate at Maximizer states: 0 1 1 0 0 0 at Minimizer states:

19. Algorithm for energy games 0 1 2 0 2 2 Fixpoint algorithm: - start with - iterate at Maximizer states: 0 1 1 0 0 0 at Minimizer states: Termination argument: monotonic operators, and finite codomain Complexity: O(E·Q·W)

20. Mean-payoff games

21. Mean-payoff games (EM79) Maximizer Minimizer positive weight = reward play: (1,4) (4,1) (1,4) (4,1) … weights: -1 +2 -1 +2 … mean-payoff value: (limit of weight average)

22. Mean-payoff games (EM79) Given a rational threshold , decide if there exists a strategy of the maximizer to ensure mean-payoff value at least . Note: we can assume e.g. by shifting all weights by . Mean-payoff value: either or Decision problem:

23. Mean-payoff games Given a rational threshold , decide if there exists a strategy of the maximizer to ensure mean-payoff value at least . Assuming A memoryless strategy is winning if all cycles are nonnegative when is fixed. Mean-payoff value: either or Decision problem:

24. Mean-payoff games Given a rational threshold , decide if there exists a strategy of the maximizer to ensure mean-payoff value at least . Assuming A memoryless strategy is winning if all cycles are nonnegative when is fixed. Mean-payoff value: either or Decision problem: log-space equivalent to energy games [BFL+08]

25. Complexity Deterministic Pseudo-polynomial algorithms

26. Outline ► Perfect information  Mean-payoff games  Energy games  Algorithms ► Imperfect information  Energy with fixed initial credit  Energy with unknown initial credit  Mean-payoff

27. Imperfect information(staying alive in the dark)

28. Imperfect information – Why ? System - Model Specification avoid failure,ensure progress,etc. Correctness relation • Private variables/internal state • Noisy sensors • Strategies should not rely on hidden information

29. Imperfect information – How ? a b • Coloring of the state space • observations = set of states with the same color

30. Imperfect information – How ? a b a a a,b b Maximizer states only Playing the game: 1. Maximizer chooses an action (a or b) 2. Minimizer chooses successor state (compatible with Maximizer’s action) 3. The color of the next state is visible to Maximizer

31. Imperfect information – How ? a,1 a,-1 a,b,0 b,2 Actions Observations

32. Imperfect information – How ? a,1 a,-1 a,b,0 b,2 Observation-based strategies Goal: all outcomes have - nonnegative energy level, - or nonnegative mean-payoff value Actions Observations

33. Complexity

34. Imperfect information a a a,b b Observation-based strategies Goal: all outcomes have - nonnegative energy level, - or nonnegative mean-payoff value Two variants for Energy games: - fixed initial credit - unknown initial credit

35. Fixed initial credit Can you win with initial credit = 3 ? Actions Observations

36. Fixed initial credit Can you win with initial credit = 3 ? • Keep track of • which can be the current state, and • what is the worst-case energy level Initially: (3,,)

37. Example (3,,) a,b (,2,2)

38. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,)

39. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) • Stop search whenever • negative value, or • comparable ancestor

40. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) • Stop search whenever: • negative value, or • comparable ancestor

41. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) Initial credit = 3 is not sufficient !

42. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) Search will terminate because is well-quasi ordered.

43. Example Upper bound: non-primitive recursive (3,,) a,b Lower bound: EXPSPACE-hard Proof (not shown in this talk): reduction from the infinite execution problem of Petri Nets. (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) Search will terminate because is well-quasi ordered.

44. Complexity

45. Memory requirement With imperfect information: Corollary: Finite-memory strategies suffice in energy games

46. Memory requirement With imperfect information: Corollary: Finite-memory strategies suffice in energy games • In mean-payoff games: • infinite memory may be required • limsup vs. liminf definition do not coincide

47. Memory requirement

48. Unknown initial credit Theorem The unknown initial credit problem for energy games is undecidable. (even for blind games) Proof: Using a reduction from the halting problem of 2-counter machines.

49. 2-counter machines • 2 counters c1, c2 • increment, decrement, zero test q1: inc c1 goto q2 q2: inc c1 goto q3 q3: if c1 == 0 goto q6 else dec c1 goto q4 q4: inc c2 goto q5 q5: inc c2 goto q3 q6: halt

50. 2-counter machines • 2 counters c1, c2 • increment, decrement, zero test q1: inc c1 goto q2 q2: inc c1 goto q3 q3: if c1 == 0 goto q6 else dec c1 goto q4 q4: inc c2 goto q5 q5: inc c2 goto q3 q6: halt