Significant Diagnostic Counterexamples in Probabilistic Model Checking

1. Significant Diagnostic Counterexamples in Probabilistic Model Checking Miguel E. Andrés Radboud University, The Netherlands Pedro D’Argenio Famaf, Argentina Peter van Rossum Radboud University, The Netherlands

2. ? MODEL (Not satisfaction) Á h R j j : e a c • Counterexamples = = Motivation • Classic Model Checking (Qualitative)

3. h i h i I t t t t n s c a s e e p r o p e r y s n o i ¯ d i f 0 6 t < s a s e p . ; h R e a c • Counterexamples (MORE COMPLEX) • Counterexamples (MORE COMPLEX) j = … … · , , , , p Motivation • Quantitative Model Checking

4. j Proposed Solution = h R · 0 5 e a c : Motivation • How do we deal with Counterexamples (so far) Problems • Not aqurate evidences • Similar evidences • Low probability evidences • Infinite evidences

5. j = h R · 0 5 e a c : The property is satisfied if for every possible way to resolve the nondeterminism the reachability probability is at most 0.5 Motivation • Non Determinism is allowed

6. Overview • Motivation • Background • Markov Chains • LTL for probabilistic systems • Counterexamples • Solution Reduced Case (Reachability and deterministic) • Reduction to Acyclic (SCC analysis) • Rails and Torrents • Solution General Case • From general formulas to reachability • From MDPs to MCs • Implementation • Conclusion • Future work

7. Overview • Motivation • Background • Markov Chains • LTL for probabilistic systems • Counterexamples • Solution Reduced Case (Reachability and deterministic) • Reduction to Acyclic (SCC analysis) • Rails and Torrents • Solution General Case • From general formulas to reachability • From MDPs to MCs • Implementation • Conclusion • Future work

8. 0 2 ( ) D T M C S L P s s s s = 0 1 3 0 . ; ; ; h ¯ S i i t t t t ² s e n e s a e s p a c e ; 0 1 s s s s 0 1 1 3 . h l S i i i i t t t t 2 ² 0 0 5 s s e n a s a e ; 0 s s s s s . 0 1 1 1 3 l b l f L i i i t 0 0 2 5 ² s a a e n g u n c o n ; s s s s s s . 0 1 1 1 1 3 [ ] 0 0 1 2 5 h P S S i i i 0 1 t t t £ ² : s a s o c a s c m a r x ! s s s s s s s . 0 1 1 1 1 1 3 ; . Finite Paths Prob Backgorund • Discrete Time Markov Chains

9. f g · ¸ 2 < > . / ² ; ; ; , ( ) f ( ) j j g Á h Á S P D t t 2 ² a ¾ a s ¾ = j ( ( ) ) Á Á P D S t . / r a p = , d § D ¤ i t t . / p _ j ( ) L a n a r e s y n a c c s u g a r ! 2 ¾ v v ¾ = , 0 ; ; ; D j ( j ) Á Á t ¾ : n o ¾ = , = D D j j j Á Á d ^ ¾ ° ¾ a n ¾ ° = , = = D D D • Semantic j j j Á 9 d 8 Á U ¾ ° ¾ ° a n ¾ = , = = # # ¸ · i i j i j 0 0 < : : D D j D j j Á Á Á Á Á Á V U ^ : : : = • Probabilistic Semantic Background • Linear Temporal Logic (LTL) • Sintaxis

10. j ( ( ) ) b Á Á R P D S t . / e m e m e r : r a p = , D . / p , , ( ) h C P D C C t [ a s 1 2 j j ( ( ) ) ( ( ) ) Á Á Á Á h h h h C C P P C C D D S S 1 µ µ t t t t t t ¡ > > ² ² r r : : a a : s u s c u c a a p p = = , i f ( ) j ( ) g h 9 C P D t j ( ) 2 · ¸ § D ½ a s ½ s s s p p = ¸ _ i 1 0 0 1 3 v v : = 1 2 Reachability property , i f ( ) j ( ) g 1 < h 9 C P D t 2 ½ a s ½ s s s = ¸ i 2 0 0 2 4 : • Example Backgorund • Counterexamples

11. Overview • Motivation • Background • Markov Chains • LTL for probabilistic systems • Counterexamples • Solution Reduced Case (Reachability and deterministic) • Reduction to Acyclic (SCC analysis) • Rails and Torrents • Solution General Case • From general formulas to reachability • From MDPs to MCs • Implementation • Conclusion • Future work

12. ( ( ( ) ) ( ( ( ) ) ) ) h l P P P P D T A T D R i t t r r r a ¾ ¾ s o r r c o ¾ a a r r e n s a a s = = = = = j ( ) s c c l d f ! ! ! C A D t t o u n e r e x a m p e s a r e g e n e r a e o r c Ac j à § D = · p ( ) D A D c Torr Solution Reduced Case We focus on: Preserves reachability probabilities!

13. Reduction Solution Reduced Case [SCC Analysis I] • Identify SCCs • Identify Input/Output states • Compute reachability probability from input to output states

14. Acyclic MC Solution Reduced Case [SCC Analysis II] • Example • Identify SCCs • Identify Input/Output States • Compute reachability probability from input to output states

15. S S S S 0 0 2 2 f S S S S S S S 5 8 1 1 6 6 6 4 4 ¹ 6 6 h f i i v ¾ ! t t , ´ ´ s s s s s s s s s s s s s s s s s s s ! ¾ ! e x s s s u c a u n c o n 0 0 2 2 6 5 1 1 1 4 1 4 0 2 0 6 2 1 6 1 1 1 4 1 1 4 , ¾ d h d F I i v t ¾ ! a n r e s n e s s a n n e r a Solution Reduced Case [Rails and Torrents] • Subsequences • Issues • Freshness • Inertia • Subsequences* (Torrents)

16. ) S ( ) ( ) h T P D 1 t , o r r ¾ a s = ( ) f ( ) j g h T P D ¹ ( ( ) ) t h P A D t 2 2 o r r ¾ ! a s ¾ ! ¾ a s c , ( ( ) ) l h R P A D i t ) 0 ( ) ( 0 ) ; h T 6 T T a s a s c 2 \ ¾ ¾ o r r ¾ o r r ¾ e o r e m = ) = ) ( ) ( ( ) ) P P T 3 r r ¾ o r r ¾ = ( ) A D D c ) ( ) j j § Ã f d l f § Ã A D D i i 4 c a n o n y = = · · p p Solution Reduced Case [Rails and Torrents] • Torrents and Rails We Generate Counterexamples on the Acyclic Chain!!!

17. Overview • Motivation • Background • Markov Chains • LTL for probabilistic systems • Counterexamples • Solution Reduced Case (Reachability and deterministic) • Reduction to Acyclic (SCC analysis) • Rails and Torrents • Solution General Case • From general formulas to reachability • From MDPs to MCs • Implementation • Conclusion • Future work

18. Deterministic Rabin Automota j j M A Á j Á M = . / p ; . / p M End Components Á General Case [Reduction to Reachability] • Reduction to Reachability Probabilistic LTL Model Checker MDP ? LTL formula Maximum Probabilities and Paths are related!!!

19. f j g d h F S i ( ) f g P ( ) t t f h l l S · 2 t 2 n x s a ¢ w o r e a r e ¿ s s ¼ ¼ ¼ ¼ x x = 1 2 1 t s S n s t ; ; : : : ; 2 P i i i P ( ) · t m n m z e x ¢ ¼ x x S 2 t s 2 S s t s 2 b h j t t t t s u e c o e s e . . f i t o c o n s r a n s . P ( ) · t ¢ ¼ x x t S n s t 2 General Case [Reduction to Markov Chains I] • Reduction to Markov Chains The calculation of a maximal probability on a reachability problem can be performed by solving a linear minimization problem

20. 0 j l à § C M i t t s a c o u n e r e x a m p e o = · 0 p j j à à § § M M + = , = · · p p j l à § C M i t t s a c o u n e r e x a m p e o = · p General Case [Reduction to Markov Chains II] Theorems:

21. Overview • Motivation • Background • Markov Chains • LTL for probabilistic systems • Counterexamples • Solution Reduced Case (Reachability and deterministic) • Reduction to Acyclic (SCC analysis) • Rails and Torrents • Solution General Case • From general formulas to reachability • From MDPs to MCs • Implementation • Conclusion • Future work

22. Implementation [Computability] • Reduce to MC problem • Using the output from the minimization problem [Bianco/de Alfaro] • Reduce to acyclic MC • Tarjan or Kosaraju or Gabow Algorithm + steady state analysis • Generate counterexamples on an Acyclic MC • K-SP problem [Han/Katoen]

23. µ ¶ ( ) ( ) P T R T r o r e p o r a r g m a x ! = For Free! T 2 ! o r EXPAND • Reachability to: • Output States • Goal States Implementation [Debugging Issues] • Torrent Representative • Expanding SCCs

24. Overview • Motivation • Background • Markov Chains • LTL for probabilistic systems • Counterexamples • Solution Reduced Case (Reachability and deterministic) • Reduction to Acyclic (SCC analysis) • Rails and Torrents • Solution General Case • From general formulas to reachability • From MDPs to MCs • Implementation • Conclusion • Future work

25. Conclusion • Counterexample generation for probabilistic LTL without restrictions • Show how to generalize counterexample generators on MC to MDP • Defined the notion of Torrents as collections of paths behaving similarly • Show how to compute Torrents-Counterexamples

26. Future work • Implementing a practical tool • Visualization of Torrents (Regular Expressions) • Case studies • Extension to Timed Systems

27. Questions Thanks for your attention!