1 / 50

Statistical Performance Analysis by Loopy Belief Propagation in Bayesian Image Modeling

Statistical Performance Analysis by Loopy Belief Propagation in Bayesian Image Modeling. Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University, Japan http://www.smapip.is.tohoku.ac.jp/~kazu/. Collaborators D. M. Titterington (University of Glasgow, UK)

rclemons
Télécharger la présentation

Statistical Performance Analysis by Loopy Belief Propagation in Bayesian Image Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical Performance Analysis byLoopy Belief Propagation in Bayesian Image Modeling Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University, Japan http://www.smapip.is.tohoku.ac.jp/~kazu/ Collaborators D. M. Titterington (University of Glasgow, UK) M. Yasuda (Tohoku University, Japan) S. Kataoka (Tohoku University, Japan) IW-SMI2010 (Kyoto)

  2. MRF, Belief Propagation and Statistical Performance • Geman and Geman (1986): IEEE Transactions on PAMI • Image Processing by Markov Random Fields (MRF) • Tanaka and Morita (1995): Physics Letters A • Cluster Variation Method for MRF in Image Processing • CVM= Generalized Belief Propagation (GBP) • Nishimori and Wong (1999): Physical Review E • Statistical Performance Estimation for MRF • (Infinite Range Ising Model and Replica Theory) Is it possible to estimate the performance of belief propagation statistically? IW-SMI2010 (Kyoto)

  3. Outline • Bayesian Image Analysis by Gauss Markov Random Fields • Statistical Performance Analysis for Gauss Markov Random Fields • Statistical Performance Analysis in Binary Markov Random Fields by Loopy Belief Propagation • Statistical Analysis of Trajectory in EM algorithm in Bayesian Image Analysis • Concluding Remarks IW-SMI2010 (Kyoto)

  4. Image Restoration by Bayesian Statistics Original Image IW-SMI2010 (Kyoto)

  5. Image Restoration by Bayesian Statistics Noise Transmission Original Image Degraded Image IW-SMI2010 (Kyoto)

  6. Image Restoration by Bayesian Statistics Noise Transmission Original Image Degraded Image Estimate IW-SMI2010 (Kyoto)

  7. Image Restoration by Bayesian Statistics Noise Transmission Posterior Original Image Degraded Image Estimate Bayes Formula 11 March, 2010 7 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)

  8. Image Restoration by Bayesian Statistics Noise Transmission Posterior Original Image Degraded Image Estimate Assumption 1: Original images are randomly generated by according to a prior probability. Bayes Formula 11 March, 2010 8 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)

  9. Image Restoration by Bayesian Statistics Noise Assumption 2: Degraded images are randomly generated from the original image by according to a conditional probability of degradation process. Transmission Posterior Original Image Degraded Image Estimate Bayes Formula 11 March, 2010 9 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)

  10. Assumption: Prior Probability consists of a product of functions defined on the neighbouring pixels. Bayesian Image Analysis Prior Probability IW-SMI2010 (Kyoto)

  11. Bayesian Image Analysis Assumption: Prior Probability consists of a product of functions defined on the neighbouring pixels. Prior Probability 11 March, 2010 11 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)

  12. Bayesian Image Analysis Assumption: Prior Probability consists of a product of functions defined on the neighbouring pixels. Prior Probability Patterns by MCMC. 11 March, 2010 12 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)

  13. Bayesian Image Analysis Assumption: Degraded image is generated from the original image by Additive White Gaussian Noise. V:Set of all the pixels IW-SMI2010 (Kyoto)

  14. Bayesian Image Analysis Assumption: Degraded image is generated from the original image by Additive White Gaussian Noise. 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 14

  15. Bayesian Image Analysis Assumption: Degraded image is generated from the original image by Additive White Gaussian Noise. 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 15

  16. Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate IW-SMI2010 (Kyoto)

  17. Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate Data Dominant Smoothing 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 17

  18. Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate Bayesian Network Data Dominant Smoothing 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 18

  19. Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate Bayesian Network Data Dominant Smoothing 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 19

  20. Image Restorations by Gaussian Markov Random Fields and Conventional Filters Original Image Degraded Image Restored Image V: Set of all the pixels Gauss Markov Random Field (3x3) Lowpass (5x5) Median IW-SMI2010 (Kyoto)

  21. Statistical Performance by Sample Average of Numerical Experiments Original Images IW-SMI2010 (Kyoto)

  22. Statistical Performance by Sample Average of Numerical Experiments Noise Pr{G|F=f,s} Original Images Observed Data 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 22

  23. Statistical Performance by Sample Average of Numerical Experiments Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images Observed Data Estimated Results 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 23

  24. Statistical Performance by Sample Average of Numerical Experiments Sample Average of Mean Square Error Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images Observed Data Estimated Results 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 24

  25. Statistical Performance Estimation Original Image Degraded Image Additive White Gaussian Noise Restored Image Posterior Probability Additive White Gaussian Noise IW-SMI2010 (Kyoto)

  26. Statistical Performance Estimation for Gauss Markov Random Fields s=40 s=40 a a IW-SMI2010 (Kyoto)

  27. = > = Statistical Performance Estimation for Binary Markov Random Fields Light intensities of the original image can be regarded as spin states of ferromagnetic system. = > = Free Energy of Ising Model with Random External Fields It can be reduced to the calculation of the average of free energy with respect to locally non-uniform external fields g1, g2,…,g|V|. IW-SMI2010 (Kyoto)

  28. Statistical Performance Estimation for Binary Markov Random Fields IW-SMI2010 (Kyoto)

  29. Statistical Performance Estimation for Markov Random Fields Spin Glass Theory in Statistical Mechanics Loopy Belief Propagation Multi-dimensional Gauss Integral Formulas 0.8 0.6 0.4 s=40 0.2 s=1 a a IW-SMI2010 (Kyoto)

  30. Image Restoration of Loopy Belief Propagation a*=0.465 IW-SMI2010 (Kyoto)

  31. Statistical Performance Estimation for Markov Random Fields IW-SMI2010 (Kyoto)

  32. Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images IW-SMI2010 (Kyoto)

  33. Statistical Performance Estimation for Binary Markov Random Fields 11 March, 2010 IW-SMI2010 (Kyoto) 33

  34. Statistical Performance Estimation for Binary Markov Random Fields Original Images 11 March, 2010 IW-SMI2010 (Kyoto)

  35. Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 35

  36. Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 36 Estimated Results

  37. Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 37 Estimated Results

  38. Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 38 Estimated Results

  39. Statistical Performance Estimation for Binary Markov Random Fields Loopy Belief Propagation IW-SMI2010 (Kyoto)

  40. Statistical Performance Estimation for Binary Markov Random Fields Loopy Belief Propagation 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 40

  41. Statistical Performance Estimation for Binary Markov Random Fields Loopy Belief Propagation Loopy Belief Propagation 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 41

  42. Statistical Performance Estimation for Markov Random Fields IW-SMI2010 (Kyoto)

  43. Statistical Performance Estimation for Markov Random Fields IW-SMI2010 (Kyoto)

  44. Marginal Likelihood Maximization of Marginal Likelihood by EM Algorithm Q-Function EM Algorithm IW-SMI2010 (Kyoto)

  45. EM Algorithm for Gauss Markov Random Fields Exact Generalized Belief Propagation Loopy Belief Propagation IW-SMI2010 (Kyoto)

  46. EM Algorithm for Gauss Markov Random Fields Numerical Experiments for Standard Image Statistical Behaviour of EM Algorithm IW-SMI2010 (Kyoto)

  47. 0.8 0.6 EM Algorithm for Binary Markov Random Fields 0.4 0.2 Numerical Experiments for Snapshot Images 1 2 3 4 5 0.8 0.6 0.4 0.2 Statistical Behaviour of EM Algorithm 1 2 3 4 5 IW-SMI2010 (Kyoto)

  48. Summary • Formulation of probabilistic model for image processing by means of conventional statistical schemes has been summarized. • Statistical performance analysis of probabilistic image processing by using Gauss Markov Random Fields has been shown. • One of extensions of statistical performance estimation to probabilistic image processing with discretestates has been demonstrated. IW-SMI2010 (Kyoto)

  49. References • K. Tanaka and D. M. Titterington: Statistical Trajectory of Approximate EM Algorithm for Probabilistic Image Processing, Journal of Physics A: Mathematical and Theoretical, vol.40, no.37, pp.11285-11300, 2007. • M. Yasuda and K. Tanaka: The Mathematical Structure of the Approximate Linear Response Relation, Journal of Physics A: Mathematical and Theoretical, vol.40, no.33, pp.9993-10007, 2007. • K. Tanaka and K. Tsuda: A Quantum-Statistical-Mechanical Extension of Gaussian Mixture Model, Journal of Physics: Conference Series, vol.95, article no.012023, pp.1-9, January 2008 • K. Tanaka: Mathematical Structures of Loopy Belief Propagation and Cluster Variation Method, Journal of Physics: Conference Series, vol.143, article no.012023, pp.1-18, 2009 • M. Yasuda and K. Tanaka: Approximate Learning Algorithm in Boltzmann Machines, Neural Computation, vol.21, no.11, pp.3130-3178, 2009. • S. Kataoka, M. Yasuda and K. Tanaka: Statistical Performance Analysis in Probabilistic Image Processing, Journal of the Physical Society of Japan, vol.79, no.2, article no.025001, 2010. IW-SMI2010 (Kyoto)

  50. Correlation for Prior Probability IW-SMI2010 (Kyoto)

More Related