1. Image Inpainting Marcelo Bertalmío, Minnesota Guillermo Sapiro, Minnesota Vicent Caselles, Barcelona Coloma Ballester, Barcelona SIGGRAPH ‘00

2. Overview • What is image inpainting • Related work • Digital inpainting • Examples • Concluding remarks

3. What is inpainting? • Modifying an image in a non-detectable form Detail of "Cornelia, Mother of the Gracchi" by J. Suvee (Louvre). Taken from Emile-Male “The Restorer’s Handbook of easel painting”.

4. Photo restoration • Restoration courtesy of D. Henry, Precious Photos Inc.

5. Object removal • From D. King, “The Commissar vanishes”.

6. Related work: Films • e.g. Kokaram et al. • Doesn’t work for stills or static objects n-1 n n+1

7. Related work: Texture synthesis • Hirani, Efros, Heeger, DeBonet, Simoncelli, etc. • Not practical for rich regions • Not designed for structured regions • “Copy” information instead of “learn and interpolate”

8. Related work: Disocclusion • Masnou-Morel, Nitzberg-Mumford, etc. • Limitations: Topology, angles See also Chan-Shen ‘00

9. Our Contribution • User only selects region to inpaint • Rich background and topology not an issue • Less than 5 minutes on a PC + =

10. How conservators inpaint • Minneapolis Institute of Art

11. Propagate information Evolutionary form Automatic digital inpainting

12. Digital inpainting (cont’d) • L = smoothness estimator (Laplacian) • N = isophote direction (time variant)

13. The equation • Plus numerical schemes (Osher) • Boundary conditions • Gray values (in a band) • Directions (in a band)

14. Example

15. Example: Text removal

16. Example: Photo restoration

17. Example: Special effects

18. Example: Scratch removal

19. Example: The evolution

20. Example: Structure but not texture is reproduced.

21. Extension: A variational formulation • Solved via E-L: Coupled 2nd order PDE’s • Full theory given • Similar practical results than 3rd order equation See also Chan-Shen ‘00

22. Concluding remarks • Technique imitates professionals • Key concepts • Information propagation • Both gray values and directions are needed • Use a band surrounding the region • Sharp results • Low complexity • Texture is not reproduced

23. Concluding remarks (cont.) • Connected to thin film and fluid dynamics (A. Betozzi) • Opens then door to high order PDE’s • Extended to a variational formulation

24. Acknowledgments • Institute Henri Poincare in Paris, France. • Tom Robbins, Elizabeth Buschor, Santiago Betelú, Stan Osher, Eero Simoncelli, Andrea Bertozzi. • Supported by ONR-Math, ONR Young Investigator Award, Presidential Early Career Awards for Scientists and Engineers, NSF CAREER Award, NSF-LIS, and IIE-Uruguay.

25. The end Thank you