1 / 79

Surviving in the Dark with Energy Games Algorithm

This talk delves into energy games where staying alive is crucial, addressing decision problems and the intricacies of maintaining energy levels. It discusses strategies, memoryless techniques, and algorithmic approaches to ensure survival in challenging environments.

nayef
Télécharger la présentation

Surviving in the Dark with Energy Games Algorithm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Staying alive in the dark Laurent DoyenLSV, ENS Cachan & CNRS joint work with Aldric Degorre1, Raffaella Gentilini1, Jean-François Raskin1, Szymon Torunczyk2 1Uni. Brussels, Belgium 2ENS Cachan, France

  2. Synthesis problem Specification avoid failure,ensure progress,etc. Correctness relation

  3. Synthesis problem System - Model Specification avoid failure,ensure progress,etc. Correctness relation Solved as a game – system vs. environment solution = winning strategy This talk: quantitative games (resource-constrained systems)

  4. Energy games(staying alive)

  5. Energy games (CdAHS03,BFLM08) Maximizer Minimizer positive weight = reward play: (1,4) (4,1) (1,4) (4,1) … weights: -1 +2 -1 +2 … energy level: 1 0 2 1 3 2 4 3 …

  6. Energy games (CdAHS03,BFL+08) Maximizer Minimizer positive weight = reward play: (1,4) (4,1) (1,4) (4,1) … weights: -1 +2 -1 +2 … energy level: 1 0 2 1 3 2 4 3 … 1 Initial credit

  7. Energy games Strategies: Maximizer Minimizer play: Infinite sequence of edges consistent with strategies and outcome is winning if: Energy level

  8. Energy games Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative.

  9. Energy games Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

  10. Energy games c0=2 c0=2 c0=1 c0=0 A memoryless strategy is winning if all cycles are nonnegative when is fixed. Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

  11. Energy games c0=2 c0=2 c0=1 c0=0 A memoryless strategy is winning if all cycles are nonnegative when is fixed. Decision problem: Decide if there exist an initial credit c0 and a strategy of the maximizer to maintain the energy level always nonnegative. For energy games, memoryless strategies suffice.

  12. Algorithm

  13. Algorithm for energy games Initial credit is useful to survive before a cycle is formed Length(AcyclicPath) ≤ Q Q: #statesE: #edgesW: maximal weight

  14. Algorithm for energy games Initial credit is useful to survive before a cycle is formed Length(AcyclicPath) ≤ Q Q: #statesE: #edgesW: maximal weight Minimum initial credit is at most Q·W

  15. Algorithm for energy games The minimum initial credit is such that: in Maximizer state q: in Minimizer state q: Compute successive under-approximations of the minimum initial credit.

  16. Algorithm for energy games 0 0 Fixpoint algorithm: - start with 0 0

  17. Algorithm for energy games 0 1 0 2 Fixpoint algorithm: - start with - iterate at Maximizer states: 0 1 0 0 at Minimizer states:

  18. Algorithm for energy games 0 1 2 0 2 2 Fixpoint algorithm: - start with - iterate at Maximizer states: 0 1 1 0 0 0 at Minimizer states:

  19. Algorithm for energy games 0 1 2 0 2 2 Fixpoint algorithm: - start with - iterate at Maximizer states: 0 1 1 0 0 0 at Minimizer states: Termination argument: monotonic operators, and finite codomain Complexity: O(E·Q·W)

  20. Mean-payoff games

  21. Mean-payoff games (EM79) Maximizer Minimizer positive weight = reward play: (1,4) (4,1) (1,4) (4,1) … weights: -1 +2 -1 +2 … mean-payoff value: (limit of weight average)

  22. Mean-payoff games (EM79) Given a rational threshold , decide if there exists a strategy of the maximizer to ensure mean-payoff value at least . Note: we can assume e.g. by shifting all weights by . Mean-payoff value: either or Decision problem:

  23. Mean-payoff games Given a rational threshold , decide if there exists a strategy of the maximizer to ensure mean-payoff value at least . Assuming A memoryless strategy is winning if all cycles are nonnegative when is fixed. Mean-payoff value: either or Decision problem:

  24. Mean-payoff games Given a rational threshold , decide if there exists a strategy of the maximizer to ensure mean-payoff value at least . Assuming A memoryless strategy is winning if all cycles are nonnegative when is fixed. Mean-payoff value: either or Decision problem: log-space equivalent to energy games [BFL+08]

  25. Complexity Deterministic Pseudo-polynomial algorithms

  26. Outline ► Perfect information  Mean-payoff games  Energy games  Algorithms ► Imperfect information  Energy with fixed initial credit  Energy with unknown initial credit  Mean-payoff

  27. Imperfect information(staying alive in the dark)

  28. Imperfect information – Why ? System - Model Specification avoid failure,ensure progress,etc. Correctness relation • Private variables/internal state • Noisy sensors • Strategies should not rely on hidden information

  29. Imperfect information – How ? a b • Coloring of the state space • observations = set of states with the same color

  30. Imperfect information – How ? a b a a a,b b Maximizer states only Playing the game: 1. Maximizer chooses an action (a or b) 2. Minimizer chooses successor state (compatible with Maximizer’s action) 3. The color of the next state is visible to Maximizer

  31. Imperfect information – How ? a,1 a,-1 a,b,0 b,2 Actions Observations

  32. Imperfect information – How ? a,1 a,-1 a,b,0 b,2 Observation-based strategies Goal: all outcomes have - nonnegative energy level, - or nonnegative mean-payoff value Actions Observations

  33. Complexity

  34. Imperfect information a a a,b b Observation-based strategies Goal: all outcomes have - nonnegative energy level, - or nonnegative mean-payoff value Two variants for Energy games: - fixed initial credit - unknown initial credit

  35. Fixed initial credit Can you win with initial credit = 3 ? Actions Observations

  36. Fixed initial credit Can you win with initial credit = 3 ? • Keep track of • which can be the current state, and • what is the worst-case energy level Initially: (3,,)

  37. Example (3,,) a,b (,2,2)

  38. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,)

  39. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) • Stop search whenever • negative value, or • comparable ancestor

  40. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) • Stop search whenever: • negative value, or • comparable ancestor

  41. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) Initial credit = 3 is not sufficient !

  42. Example (3,,) a,b (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) Search will terminate because is well-quasi ordered.

  43. Example Upper bound: non-primitive recursive (3,,) a,b Lower bound: EXPSPACE-hard Proof (not shown in this talk): reduction from the infinite execution problem of Petri Nets. (,2,2) a b a b (3,,) (,2,1) (,1,3) (3,,) b a a b (4,,) (,1,0) (,1,4) (2,,) Search will terminate because is well-quasi ordered.

  44. Complexity

  45. Memory requirement With imperfect information: Corollary: Finite-memory strategies suffice in energy games

  46. Memory requirement With imperfect information: Corollary: Finite-memory strategies suffice in energy games • In mean-payoff games: • infinite memory may be required • limsup vs. liminf definition do not coincide

  47. Memory requirement

  48. Unknown initial credit Theorem The unknown initial credit problem for energy games is undecidable. (even for blind games) Proof: Using a reduction from the halting problem of 2-counter machines.

  49. 2-counter machines • 2 counters c1, c2 • increment, decrement, zero test q1: inc c1 goto q2 q2: inc c1 goto q3 q3: if c1 == 0 goto q6 else dec c1 goto q4 q4: inc c2 goto q5 q5: inc c2 goto q3 q6: halt

  50. 2-counter machines • 2 counters c1, c2 • increment, decrement, zero test q1: inc c1 goto q2 q2: inc c1 goto q3 q3: if c1 == 0 goto q6 else dec c1 goto q4 q4: inc c2 goto q5 q5: inc c2 goto q3 q6: halt

More Related