1 / 69

Symbolic dynamics of M arkov chains

Symbolic dynamics of M arkov chains. P S Thiagarajan School of Computing National University of Singapore Joint work with: Manindra Agrawal , S Akshay , Blaise G enest. Acknowledgements. Samarjit and Javier IAS TÜV SÜD Foundation. Agenda. Study complex dynamical systems

norris
Télécharger la présentation

Symbolic dynamics of M arkov chains

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Symbolic dynamics of Markov chains P S Thiagarajan School of Computing National University of Singapore Joint work with: ManindraAgrawal, S Akshay, BlaiseGenest

  2. Acknowledgements • Samarjit and Javier • IAS • TÜVSÜD Foundation

  3. Agenda • Study complexdynamical systems • verification of temporal logic specification • High dimensional • Continuous time; continuous value domains. • Differential equations • Hybrid automata • Networks of such dynamical systems

  4. Strategy • Approximate the dynamics. • Discretize the time and value domains • Today’s first talk will have this flavor. • Sample the dynamics and do a statistical analysis. • Today’s second talk • To handle large networks: Deterministic synchronizations • Tomorrow’s talk

  5. Probabilistic dynamics • Illustrate these ideas in the setting of probabilistic dynamical systems. • Yield useful approximations in the presence intractability and lack of knowledge. • Scalability can be achieved through sampling (simulations) based statistical analysis techniques. • Statistical model checking • Key application: (Networks) of biochemical networks.

  6. Study I: Symbolic dynamics of Markov chains • Well-known probabilistic dynamical systems. • Rich theory • Widely applied. • Our focus (finite state, discrete time): • Symbolic dynamics • Discretize the probability value space [0, 1].

  7. Symbolic dynamics of Markov chains • Analyze the symbolic dynamics via: • classical(linear) temporal logic specifications. • Probabilities are sneaked in through the atomic propositions. • Model checking methods • Details in: • LICS’2012 • Journal paper under review.

  8. Two views of Markov chains • View I • a finite state probabilistic transition system. • View II • a linear transform of probability distributions over the states.

  9. The transition system view 1 3 2 1 3/5 1 1 4 2/5

  10. The transition system view 0 1 1 3 2 0 0 1 3/5 1 1 4 2/5

  11. The transition system view 3 1 1 3 2 1 4 1 3/5 1 2/5 3/5 1 4 2 1 2/5 3 3 4 4 3/5 2/5 2/5 1

  12. The transition system view • Outcomes: Infinite paths • Basic cylinder: The set of infinite paths that have a common finite prefix. • Path space: -algebra generated by the basic cylinders. • Probabilities are assigned to basic cylinders in a natural way. • Extends canonically to a probability measure over the path space.

  13. The transition system view 3 1 1 1 3 2 4 1 3/5 1 2/5 3/5 1 4 2 1 2/5 1 1 B – The set of all paths that have the prefix 3 4 1 3 4 3 3 1 1 Pr(B) = 1  2/5  1  1 = 2/5 4 4 B 1

  14. Measurable properties • The state 3 will be visited infinitely often. PCTL, CTL, …. Baier, C. and Katoen, J.-P. Principles of Model Checking

  15. VIEW II • The Markov chain transforms (linearly) a given distribution over its nodes to a new one. • The graph of the chain is represented as a stochastic matrix • (all row sums are 1) • The Perron-Frobinius theory (for non-negative matrices) applies.

  16. VIEW II • Leads to natural sub-classes: • Irreducible and aperiodic, • periodic, … • transient states, recurrent states…. • Stationary distributions….

  17. VIEW II 1 3 2 1 3/5 1 1 4 2/5

  18. The second view 0 0 1 3 2 1 3/5 0 1 1 1 4 2/5

  19. Trajectory of distributions • Markov chain as (linear)transformer of probability distributions 1 0 3 2 1 1 3/5 1 1 4 Transition matrix M 2/5 0 0

  20. Trajectory of distributions 0 0 1 1 3 2 0 • Each initial distribution generates a trajectory of distributions: 1 3/5 1 1 4 2/5 Transition matrix M

  21. Trajectory of distributions 0 0 1 1 3 2 0 • Each initial distribution generates a trajectory of distributions: 1 3/5 1 1 4 2/5 Transition matrix M

  22. Trajectory of distributions • Dynamics: • The set of trajectories generated from an initial set of distributions • This set can be infinite • We can ask whether all (none) of the trajectories satisfy some desired dynamical property • What we can ask in this setting is incomparable with View I specifiable properties

  23. Trajectory of distributions • Eventually the probability of being in 1 is always greater than being in 3. • There is a future time at which 90% of the probability mass lies on {1, 4}.

  24. Symbolic dynamics • A trajectory: • an infinite sequence of probability distributions (over a finite set of nodes). • The alphabet of probability distributions -over a finite set of nodes- is (uncountably) infinite. • Hence a trajectory is an infinite sequence over an infinite alphabet, • But exactly tracking probability distributions may be neither necessary nor possible. • Bio-chemical networks • Sensor networks…

  25. Symbolic dynamics • We propose to reason about sequences of distributions in a symbolic way. • Using a finite alphabet of discretized distributions.

  26. Symbolic dynamics • A classical tradition dynamical systems theory. • Jacques Hadamard(1898), Morse and Hedlund (1921), …. • Shannon, Smale… • Significant applications in data storage and transmission (via coding theory), linear algebra.

  27. Symbolic dynamics • We know how to do this for: • Timed automata • In some restricted settings for hybrid automata. • For finite state Markov chains this has been done under very restricted setting and unnatural restrictions (Chada et.al 2011, Korthikantiet.all 2010.) • We shall instead give up on bisimulation-based equivalence classes: • Instead, fix a discretization of [0, 1] • With no restrictions handle all finite state Markov chains .

  28. Symbolic dynamics c d F2(x) F(x) F3(x) e a x b Finite alphabet!

  29. Symbolic dynamics • Each block is a letter. • Each trajectory is now recorded as a sequence of letters taken from a finite alphabet. • If each block bisimualtion equivalence class it is called a Markov partition! • Study the system dynamics in terms of these sequences. • Sophic shift sequences • Shift sequences of finite type.

  30. Symbolic dynamics • We know how to get “Markov partitions”) do this for: • Timed automata • For hybrid automata, in a few settings. • For finite state Markov chains this has been done under very restricted setting and unnatural restrictions (Chada et.al 2011, Korthikantiet.all 2010.) • We do not look for bisimulations of finite index.: • Instead, we fixa discretization of [0, 1] • Handle all finite state Markov chains ; no restrictions.

  31. Symbolic dynamics of Markov chains • Discretization • We partition [0,1] into a finite set of intervals, a b c

  32. Symbolic dynamics of Markov chains a b c Each (probability) value is mapped to (identified with) the interval it falls in.

  33. Symbolic dynamics of Markov chains

  34. Symbolic dynamics of Markov chains  = D Each distribution𝜇 maps to D, a unique n-tuple of intervals. 𝜇 ∈DmeansΓ(𝜇) = D

  35. Symbolic dynamics of Markov chains • Each distribution𝜇 maps to D, a unique n-tuple of intervals. • Several distributions can map to the same D, • in fact Γ-1(D) can be infinite  = D 

  36. Symbolic dynamics of Markov chains • Discretization • We partition [0,1] into a finite set of intervals, • Γ-1(D) can be empty  ? = D

  37. Symbolic dynamics of Markov chains  = D  = D ø × • Discretized distribution • A tuple of intervals D is discretized distribution ( 𝒟- distribution for short ) ifΓ-1(D) ≠ ø

  38. Symbolic dynamics of Markov chains • The set of discretized distributions 𝒟 is finite

  39. Symbolic dynamics of Markov chains • A Trajectory of M starting from a distribution 𝜇 : • Induces a symbolic trajectory • a word over 𝒟ω • ξ 𝜇 = (Γ(𝜇) Γ(𝜇1) Γ(𝜇2) ….)

  40. Symbolic dynamics of Markov chains 1 3 2 . . 0.45 0.12  1 3/5 1 1 4 2/5 0.08 0.35

  41. Symbolic dynamics of Markov chains • Fix a set of initial distributions IN. • We use a 𝒟-distribution Din to specify IN. That is IN = Din This can be an infinite set . • IN = = , ……….. • We can fix the set of initial distributions in many other ways.

  42. Symbolic dynamics of Markov chains • The symbolic dynamics of (M,Din ) is: • We wish to reason about this -language.

  43. Symbolic dynamics of Markov chains • The discretization need not be uniform • Can be different for each dimension (node). • {[0, 1]} can be used to mask out “don’t care” nodes. • Dimension-reduction.

  44. A temporal logic to reason about the symbolic dynamics • I, a discretization of [0, 1] : • < i , d > is an atomic proposition • nodeiinterval d • “In the current distribution 𝜇, 𝜇(i)falls in the interval d ”. • In the current discretized distribution D, D(i) = d • Probabilistic linear-time temporal logic LTL I

  45. The verification problem • Given, • a Markov chain M, • a discretization I, • an initial set of distributions represented as Din , • and a specification φ as an LTLIformula, • Determine if M, Din ⊨ φ. In other words, • (0) ⊨  for every  in LM • Does every symbolic trajectory of M satisfy φ, i.e., • is it the case LM ⊆ L φ?

  46. Example formulas • Whenever probability of nodeiis “high”, the probability of nodej is “low” : • The 𝒟 -distribution (d1, d2, . . . , dn) appears infinitely often:

  47. Example formulas • Extending with FO theory of reals, we can express much more: • e.g., Infinitely often, the probability of node i is at least twice the sum of probabilities of all other nodes. • Logics based on path spaces (PCTL etc.) and logics based sequencesof probability distributions are incomparable

  48. Examples • We have modeled a simple version of the Google pageranking algorithms. • We have also modeled a small pharmacokinetics system for drug delivery.

  49. The verification problem • Given, M, I, Din , φ, determine if M, Din ⊨ φ, i.e., LM ⊆ L φ • If LMis ω-regular and effectively computable then we can use standard model checking techniques. • But LMis not always ω-regular !

  50. The verification problem LMis not always ω-regular.

More Related