1 / 73

Junction Tree Algorithm

Junction Tree Algorithm. Brookes Vision Reading Group. Outline. Graphical Models What are Graphical Models ? Conditional Independence Inference Junction Tree Algorithm Moralizing a graph Junction Tree Property Creating a junction tree Inference using junction tree algorithm .

ronald
Télécharger la présentation

Junction Tree Algorithm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Junction Tree Algorithm Brookes Vision Reading Group

  2. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  3. Don’t we all know … • P(A) = 1 , if and only if A is certain • P(A or B) = P(A)+P(B) if and only if A and B are mutually exclusive • P(A,B) = P(A|B)P(B) = P(B|A)P(A) • Conditional Independence • A is conditionally independent of C given B • P(A|B,C) = P(A|B)

  4. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  5. Graphical Models Compact graphical representation of joint probability. A A ‘causes’ B B P(A,B) = P(A)P(B|A)

  6. Graphical Models Compact graphical representation of joint probability. A B ‘causes’ A B P(A,B) = P(B)P(A|B)

  7. Graphical Models Compact graphical representation of joint probability. A B P(A,B)

  8. A Simple Example P(A,B,C) = P(A)P(B,C | A) = P(A) P(B|A) P(C|B,A) = P(A) P(B|A) P(C|B) C is conditionally independent of A given B Graphical Representation ???

  9. A B C Bayesian Network Directed Graphical Model P(U) =  P(Vi | Pa(Vi)) P(A,B,C) = P(A) P(B | A) P(C | B)

  10. A B C Markov Random Fields Undirected Graphical Model

  11. Markov Random Fields Undirected Graphical Model AB BC B Clique Separator Clique P(U) =  P(Clique) /  P(Separator) P(A,B,C) = P(A,B) P(B,C) / P(B)

  12. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  13. Bayesian Networks • A is conditionally independent of B given C • Bayes ball cannot reach A from B

  14. Markov Random Fields • A, B, C - (set of) nodes • C is conditionally independent of A given B • All paths from A to C go through B

  15. Markov Random Fields

  16. Markov Random Fields A node is conditionally independent of all others given its neighbours.

  17. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  18. MAP Estimation c*,s*,r*,w* = argmax P(C=c,S=s,R=r,W=w)

  19. Computing Marginals P(W=w) = c,s,r P(C=c,S=s,R=r,W=w)

  20. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  21. Aim • To perform exact inference efficiently • Transform the graph into an appropriate data structure • Ensure joint probability remains the same • Ensure exact marginals can be computed

  22. Junction Tree Algorithm • Converts Bayes Net into an undirected tree • Joint probability remains unchanged • Exact marginals can be computed • Why ??? • Uniform treatment of Bayes Net and MRF • Efficient inference is possible for undirected trees

  23. Junction Tree Algorithm • Converts Bayes Net into an undirected tree • Joint probability remains unchanged • Exact marginals can be computed • Why ??? • Uniform treatment of Bayes Net and MRF • Efficient inference is possible for undirected trees

  24. Let us recap .. Shall we A B C D P(U) =  P(Vi | Pa(Vi)) =  a(Vi , Pa(Vi)) Potential Lets convert this to an undirected graphical model

  25. The cliques of this graph are inconsistent with the original one. Node D just lost a parent. Let us recap .. Shall we A B C D Wait a second …something is wrong here.

  26. Marry the parents for a happy family Solution A B C D Ensure that a node and its parents are part of the same clique Now you can make the graph undirected

  27. Solution A B C D But we have added extra edges, haven’t we ??? A few conditional independences are lost.

  28. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  29. Moralizing a graph • Marry all unconnected parents • Drop the edge directions Ensure joint probability remains the same.

  30. Moralizing a graph

  31. Clique Potentials a(Ci) Separator Potentials a(Si) Moralizing a graph C S R Initialize a(Ci) = 1 a(Si) = 1 W CSR SR SRW

  32. Choose one node Vi Find one clique Ci containing Vi and Pa(Vi) Moralizing a graph C S R W Multiply a(Vi,Pa(Vi)) to a(Ci) CSR Repeat for all Vi SR SRW

  33. Moralizing a graph C Choose one node Vi S R Find one clique Ci containing Vi and Pa(Vi) W Multiply a(Vi,Pa(Vi)) to a(Ci) CSR Repeat for all Vi SR SRW

  34. Moralizing a graph C Choose one node Vi S R Find one clique Ci containing Vi and Pa(Vi) W Multiply a(Vi,Pa(Vi)) to a(Ci) CSR Repeat for all Vi SR SRW

  35. Moralizing a graph C Choose one node Vi S R Find one clique Ci containing Vi and Pa(Vi) W Multiply a(Vi,Pa(Vi)) to a(Ci) CSR Repeat for all Vi SR SRW

  36. Moralizing a graph P(U) =  a(Ci) /  a(Si) Now we can form a tree with all the cliques we chose. That was easy. We’re ready to marginalize. OR ARE WE ???

  37. A few more examples … A B A B C D C D

  38. A few more examples … A B A B C D C D

  39. Inconsistency in C A few more examples … AB BD AB BCD Clearly we’re missing something here AC CD

  40. Outline • Graphical Models • What are Graphical Models ? • Conditional Independence • Inference • Junction Tree Algorithm • Moralizing a graph • Junction Tree Property • Creating a junction tree • Inference using junction tree algorithm

  41. Junction Tree Property In a junction tree, all cliques in the unique path between cliques Ci and Cj must contain CiCj So what we want is a junction tree, right ??? Q. Do all graphs have a junction tree ??? A. NO

  42. Decomposable Graphs Decomposition (A,B,C) A C B Undirected graph G = (V,E) • V = A  B  C • All paths between A and B go through C • C is a complete subset of V

  43. Decomposable Graphs • A, B and/or C can be empty • A, B are non-empty in a proper decomposition

  44. Decomposable Graphs • G is decomposable if and only if • G is complete OR • It possesses a proper decomposition (A,B,C) such that • GAC is decomposable • GBC is decomposable

  45. Decomposable Graphs A B A B C D C D Decomposable Not Decomposable

  46. Decomposable Graphs A A B C B C D E D E Decomposable Not Decomposable

  47. An Important Theorem Theorem: A graph G has a junction tree if and only if it is decomposable. Proof on white board.

  48. OK. So how do I convert my graph into a decomposable one.

  49. Time for more definitions • Chord of a cycle • An edge between two non-successive nodes • Chordless cycle • A cycle with no chords • Triangulated graph • A graph with no chordless cycles

  50. Another Important Theorem Theorem: A graph G is decomposable if and only if it is triangulated. Proof on white board. Alright. So add edges to triangulate the graph.

More Related