1 / 42

Variational Pairing of Image Segmentation and Blind Restoration

Variational Pairing of Image Segmentation and Blind Restoration. Leah Bar Nir Sochen* Nahum Kiryati. School of Electrical Engineering *Dept. of Applied Mathematics. Tel Aviv University. Segmentation : images meet concepts. Borrowed from Georges Koepfler.

Télécharger la présentation

Variational Pairing of Image Segmentation and Blind Restoration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Variational Pairing of Image Segmentation and Blind Restoration Leah Bar Nir Sochen*Nahum Kiryati School of Electrical Engineering *Dept. of Applied Mathematics Tel Aviv University

  2. Segmentation: images meet concepts Borrowed from Georges Koepfler

  3. Segmentation: images meet concepts Borrowed from Georges Koepfler Formalization (Mumford & Shah) Segmentation by minimizing a functional min [(fidelity to image) + β (gradients within segments) + α (total edge length)]

  4. Segmentation: images meet concepts Borrowed from Georges Koepfler Formalization (Mumford & Shah) Segmentation by minimizing a functional min [(fidelity to image) + β (gradients within segments) + α (total edge length)] Calculus of Variations PDE’s Numerical Techniques Linear Systems of Equations

  5. Mumford-Shah Segmentation fidelity to image gradients within segments total edge length Ω: image domainK: edge setf: segmented imageg: observed image

  6. Mumford-Shah Segmentation fidelity to image gradients within segments total edge length Ω: image domainK: edge setf: segmented imageg: observed image Problem: Discontinuities in the domains (Ω/K, K) make minimization difficult

  7. Mumford-Shah Segmentation fidelity to image gradients within segments total edge length Ω: image domainK: edge setf: segmented imageg: observed image Problem: Discontinuities in the domains (Ω/K, K) make minimization difficult Solution: Continuous approximation of F(f,K) (Gamma-convergence framework)

  8. Mumford-Shah Segmentation fidelity to image gradients within segments total edge length Ω: image domainK: edge setf: segmented imageg: observed image Problem: Discontinuities in the domains (Ω/K, K) make minimization difficult Solution: Continuous approximation of F(f,K) (Gamma-convergence framework) fidelity to image gradients in segments total edge length v(x):smooth function v(x)~0 at edges v(x)~1 otherwise (in segments) (Ambrosio & Tortorelli, 1990)

  9. In a blurred image, edges are degraded and segmentation is difficult.

  10. Image Restoration Given the image g and the blur kernel h, restore the original image f . 1. Brute force ... ill posed. Minimize 2. Tikhonov regularization ... oversmoothing. Minimize 3. Total Variation (TV) regularization ... better edge preservation. Minimize

  11. Blind Image Restoration Given the image g, restore the original image f (and the blur-kernel h). - Ill posed (1): sensitivity to small changes in g. - Ill posed (2): maybe the original image was already blurred?

  12. Blind Image Restoration Given the image g, restore the original image f (and the blur-kernel h). - Ill posed (1): sensitivity to small changes in g. - Ill posed (2): maybe the original image was already blurred? Chan & Wong (1998) TV-regularization with respect to both the image and the kernel. Minimize - The restored image is very sensitive to the recovered kernel - The recovered kernel depends on the contents of the image (bad news)

  13. Chan & Wong - The recovered kernel depends on the contents of the image. source image isotropic blur blind restoration recovered kernel

  14. Chan & Wong (1998) - Performance original blurred restored isotropic gaussian kernel, =2.1 - The restored image is very sensitive to the recovered kernel. - The recovered kernel depends on the contents of the image. recovered kernel

  15. In blind image restoration, one can’t get it all. Borrowed from Mickey Mouse (The Sorcerer’s Apprentice)

  16. Some related work... (Blind) Restoration Segmentation You & Kaveh, 1996 Vogel & Oman, 1998 Mumford & Shah, 1989 Kim et al, 2002 Chan & Wong, 1998 Chambolle, 1995 Carasso, 2001 Hewer et al, 1998 Mathematics, Foundations Ambrosio & Tortorelli, 1992 Tikhonov & Arsenin, 1977 Rudin, Osher & Fatemi, 1992 Aubert & Kornprobst, 2002

  17. The suggested approach Why? - Segmentation is hard, but easier if the image is sharp - Blind restoration is hard, but easier if the edges are known What? Blind restoration and segmentation as mutually supporting processes How? Unified variational framework, iterative algorithm

  18. Combined objective functional • Mumford-Shah segmentation + blind restoration • Make it work: Use the Γ-convergence approximation • Make it work well: Use a parametric blur-kernel fidelity, parametric blur gradients in segments “smooth v” “total edge length” “wide kernel” Reminderv(x): smooth function v(x)~0 at edges v(x)~1 otherwise (in segments)

  19. Minimizing the functional • Iterate • Minimize with respect to v(segmentation / edge detection) • Minimize with respect to f(image restoration) • Minimize with respect to σ(blur-kernel recovery)

  20. Iterative Minimization Equations • Minimization with respect to v(Euler equation)

  21. Iterative Minimization Equations • Minimization with respect to v(Euler equation) • Minimization with respect to f(Euler equation)

  22. Iterative Minimization Equations • Minimization with respect to v(Euler equation) • Minimization with respect to f(Euler equation) • Minimization with respect to σ(derivative)

  23. Iterative Minimization Equations • Minimization with respect to v(Euler equation) • Minimization with respect to f(Euler equation) • Minimization with respect to σ(derivative) Calculus of Variations PDE’s Numerical Techniques Linear Systems of Equations

  24. Frequently Asked Questions • What are the initial values? We use f=g (output=input), v=1 (no edges) and σ=ε(small blur). • What is the stopping condition? We stop when the radius σ of the recovered kernel has converged. • Does it converge? To a global optimum? • Nice theoretical properties • Excellent experimental behavior • Additional analytic work in progress Typical convergence: σ vs. iteration number

  25. Experimental Results (1): Known Blur Kernel Blurred

  26. Experimental Results (1): Known Blur Kernel Blurred Lucy-Richardson restoration

  27. Experimental Results (1): Known Blur Kernel Blurred Lucy-Richardson restoration Suggested restoration

  28. Experimental Results (1): Known Blur Kernel Blurred Lucy-Richardson restoration Suggested restoration Suggested edges (v function)

  29. Experimental Results (2): Blind Blurred

  30. Experimental Results (2): Blind Blurred Chan-Wong restoration

  31. Experimental Results (2): Blind Blurred Chan-Wong restoration Suggested restoration

  32. Experimental Results (2): Blind Blurred Chan-Wong restoration Suggested restoration Suggested edges (v function)

  33. Experimental Results (3): Blind Blurred

  34. Experimental Results (3): Blind Blurred Chan-Wong restoration

  35. Experimental Results (3): Blind Blurred Chan-Wong restoration Suggested restoration

  36. Experimental Results (3): Blind Blurred Chan-Wong restoration Suggested restoration Suggested edges (v function)

  37. Experimental Results (4): Blind Blurred

  38. Experimental Results (4): Blind Blurred Chan-Wong restoration

  39. Experimental Results (4): Blind Blurred Chan-Wong restoration Suggested restoration

  40. Experimental Results (4): Blind Blurred Chan-Wong restoration Suggested restoration Suggested edges (v function)

  41. Conclusions Image segmentation and (blind) restoration, sont les mots qui vont tres bien ensemble* . The whole is larger than the sum of its parts (in this case). Blind restoration is easier if you can use a parametric blur model. *these are words that go together well.

More Related