Markov Random Fields and Gibbs Distributions

1. Markov Random Fields and Gibbs Distributions Qiang He School Of EE & CS Oregon State University

3. 1. Introduction

4. Markov random fields (MRFs) • A statistical theory for analyzing spatial & contextual dependencies of physical phenomena. • A Bayesian labeling problem • A method to establish the probabilistic distributions of interacting labels • Widely used in image processing and computer vision

5. Properties of MRF • Not ad hoc, can be solved based on sound mathematical principles (maximum a posterior probability, MAP) • Incorporating prior contextual information • Using local properties, which can be implemented in parallel

6. An example: image restoration using MRF

7. Image restoration process Goals • Restore degraded and noisy images • Infer the true pixels from noisy ones • Build the neighborhood systems and cliques • Define the clique potentials for prior probability • Derive the likelihood energy • Compute the posterior energy • Solve the MAP

8. Definition for symbols = set of sites or nodes = neighbors = a nondirected graph = hidden “true “ pixel = observed “noisy “ pixel

9. 2. Nondirected graphs

10. Neighborhood Systems A neighborhood system for is defined as where is the set of sites neighboring . The neighboring relationship has the following properties: • a site is not neighboring to itself • the neighboring relationship is mutual

11. Neighborhood Systems

12. A clique is defined as a subset of sites in , where every pair of sites are neighbors of each other. The collections of single- site, double-site, and triple-site cliques are denoted by , , and ,… A collection of cliques is • Cliques

13. Cliques

14. 3. Markov Random Fields

15. Basics • Random field: A family of rvs defined on the set • Configuration: a value assignment on a random field • Probability: --discrete case: joint probability --continuous case: joint PDF

16. Markov random fields • Positivity: • Markovianity: • Homogeneity: probability independent of positions of sites • Isotropy: probability independent of orientations of sites

17. Bayesian labeling problem

18. 4. Gibbs Random Fields (GRFs)

19. Gibbs distribution: Partition function: Temperature: Energy function: Clique potentials: Special case: Gaussian distribution

20. 5. Markov-Gibbs Equivalence

21. Conditional probability: • Proof: MRF=GRF Extended from clique potentials: Factor into two terms Containing i or not: Remove the term containing i:

22. MRF prior and Gibbs distribution

23. Posterior MRF energy Likelihood function: Likelihood energy: Posterior probability: Posterior energy: MAP solution:

24. 6. Inference tasks

25. Goals • Solve the Bayesian labeling problem, that is, find the maximum a posterior (MAP) configuration under the observation (simulated annealing process) • Compute a marginal probability (Gibbs sampling) • Solve MRF prior probability through Gibbs distribution (since MRF=GRF) • Solve likelihood function by estimating the likelihood energy and the posterior energy: coding method or least square error method • Solve the MAP • Parameter estimation

26. Look back at image restoration • Build the neighborhood systems and cliques 4-neighborhood system and two-site cliques • Define the prior clique potentials

27. Compute the likelihood energy • Compute the posterior energy

28. 7. Summary

29. The MRF modeling is to solve the Bayesian labeling problem, that is, find the maximum a posterior (MAP) configuration under the observation • The MRF factors joint distribution into a product of clique potentials • The MRF modeling provides a systematic approach in solving image processing and computer vision problems

30. Thank you very much!