Practical Guide to Message Passing in Probabilistic Graphical Models

1. Inference Probabilistic Graphical Models Message Passing BP in Practice

2. Misconception Revisited A D B C

3. Nonconvergent BP Run

4. synchronous Different Variants of BP Synchronous BP:all messages areupdated in parallel x 100 6 # messages converged 11 11 12 12 13 13 4 2 21 21 22 22 23 23 Ising Grid 0 2 4 6 8 10 12 14 31 31 32 32 33 33 Time (seconds)

5. asynchronous order 2 x 100 asynchronous 6 synchronous # messages converged 4 2 Ising Grid 0 2 4 6 8 10 12 14 Time (seconds) Different Variants of BP Asynchronous BP:Messages are updated one at a time 11 12 13 21 22 23 31 32 33

6. Observations • Convergence is a local property: • some messages converge soon • others may never converge • Synchronous BP converges considerably worse than asynchronous • Message passing order makes a difference to extent and rate of convergence

7. Informed Message Scheduling • Tree reparameterization (TRP) • Pick a tree and pass messages to calibrate 11 12 13 21 22 23 31 32 33

8. Informed Message Scheduling • Tree reparameterization (TRP) • Pick a tree and pass messages to calibrate • Residual belief propagation (RBP) • Pass messages between two clusters whose beliefs over the sepset disagree the most

9. Smoothing (Damping) Messages • Dampens oscillations in messages

11. Summary • To achieve BP convergence, two main tricks • Damping • Intelligent message ordering • Convergence doesn’t guarantee correctness • Bad cases for BP – both convergence & accuracy: • Strong potentials pulling in different directions • Tight loops • Some new algorithms have better convergence: • Optimization-based view to inference

12. END END END