Algorithm for Solving Timed Games with Observations

Solving Timed Games with Variable Observations: Proof of Concept Peter Bulychev Franck Cassez Alexandre David Kim G. Larsen Jean-François Raskin Pierre-Alain Reynier

Timed Game Automata • Timed Game Automata is a Timed Automata where transitions are split into controllable and uncontrollable • We support safety objectives: • control: AG (not Bad) • Memoryless strategy: • state  action • UPPAAL Tiga can be used to solve safety timed games a b GASICS Workshop

Timed Game Automata control: AG (not Bad) x≤1 : a a Strategy True : DELAY True : DELAY x≤1 : b b GASICS Workshop

Controller synthesis with partial observation • Consider that controller doesn’t have full information about the current state of a system • Observation is a valuation of a finite number of state-based boolean predicates (sensors) • We allow predicates of the form: • (L1 or L2 or L3) and (1≤x<2) • Controller makes its decisions based on history of the observations seen so far • Controller sees only changes on observations => stuttering-invariant strategy GASICS Workshop

Controller synthesis with partial observation: the algorithm Partition the state-space w.r.t. values of the predicates. Predicates p1, p2 Losing is observable. p1p2 LOSING p1p2 DELAY b a p1p2 a b a p1p2 GASICS Workshop

Running example (LH boxes) control: AG (not Bad) • Possible sets of observations: • {H, L} • {H, L, y≥3} • {y ≥ 1} • {H, L, y≥5} • full information • {H, L, y≥1} EJECT RESET GASICS Workshop

Controller synthesis with partial observation: the algorithm • Algorithm, described in F. Cassez et al., 2007: • Symbolic • On-the-fly • Subset construction-based • Implemented in UPPAAL Tiga Partition the state-space w.r.t. observations. Observations O1 O2 O3. Winning/losing is observable. GASICS Workshop

Running example (LH boxes) control: AG (not Bad) Available observations: {H, L, y ≥ 5} E0,x==y==0 {} H,x==y==0 E1\/E2,x==y==0 E1\/E2,x==y==5 DELAY DELAY DELAY {} {y ≥ 5} {H} RESET DELAY EJECT {} {y ≥ 5} E1\/E2,x==10, y==5 E1\/E2,x==5, y==0 EJECT DELAY DELAY {} {y ≥ 5} {L} H,x==y==0 E3\/E4,x==y==0 E3\/E4,x==y==5 GASICS Workshop

Problem statement • Assume a finite set of available sensors and each sensor has some cost • We want to synthesize a controller that will achieve its goal by using a set of sensors with a minimal cost • Input: • Timed Game Automata A • Safety propertyφ • A set of predicates Pred = {p1, …, pn} • Cost function ω = {p1->c1, …, pn->cn} • Goal: • To find a set of predicates P with a minimal total cost such that A,P|=φ is true GASICS Workshop

Basic algorithm {φ} Consider a lattice of all possible predicates sets {φ} U Pred GASICS Workshop

Basic algorithm {φ} 1. Check if φ is controllable on A with full information {φ} U Pred Full information GASICS Workshop

Basic algorithm {φ} 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P P {φ} U Pred GASICS Workshop

Basic algorithm {φ} 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P 3. If A,P|=φ is true, then we • remove from further consideration all sets P’s.t.P⊆P’ P {φ} U Pred GASICS Workshop

Basic algorithm {φ} 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P 3. If A,P|=φ is true, then we • remove from further consideration all sets P’s.t.P⊆P’ • remove from further consideration all sets P’s.t.ω(P’) ≥ ω(P) P {φ} U Pred GASICS Workshop

Basic algorithm {φ} 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P 3. If A,P|=φ is true, then we • remove from further consideration all sets P’s.t.P⊆P’ • remove from further consideration all sets P’s.t.ω(P’) ≥ ω(P) 4. Otherwise, we • remove from further consideration all sets P’s.t.P’⊆P {φ} U Pred GASICS Workshop

Basic algorithm {φ} The set of possible observation sets is finite, so the algorithm will converge {φ} U Pred GASICS Workshop

Basic algorithm {φ} Optimizations: • Which exploration strategy to use? • Random • Top-bottom • Bottom-top • Midpoint • What information to reuse? • Losing states from below • Winning states from above • State space from below {φ} U Pred GASICS Workshop

State space reusage {φ} (L1, x≥4) ∨ (L2, x≥5) ∨ (L3, x<2) (L4, x≥8) ∨ (L5, x≥7) ∨ (L6, x<2) a b L6, x<2 a L1, x≥4 L4, x≥8 a {φ} U Pred L5, x≥7 L2, x≥5 a b L6, x<2 L3, x<2 GASICS Workshop

Implementation details Efficient Stable Ready for industry applications Has a nice GUI Easy to prototype new very specific features

Python framework for timed automata manipulation • PyDBM– Python wrapper for UPPAAL DBM library • pyuppaal – syntactic parser of UPPAAL models • dbmpyuppaal – parses a model using pyuppaal and replaces all guards and invariants by their DBMs • opaal – model checker for timed automata More information at: http://cs.aau.dk/~adavid/python GASICS Workshop

Results Possible observations and their cost: {H -> 1, L ->1, y≥1 -> 10, y≥2 -> 9, …, y≥10 -> 1} Optimal solution: {H, y≥5} EJECT RESET 27 GASICS Workshop

Results (average running time) GASICS Workshop

Questions? GASICS Workshop

Algorithm for Solving Timed Games with Observations