Poisson Image Editing

1. Poisson Image Editing Patrick Perez Michel Gangnet Andrew Blake Microsoft Research UK

2. Outline • Introduction • Poisson solution to guided interpolation • Guided Interpolation • Discrete Poisson solver • Seamless cloning • Importing gradients • Mixing gradients • Selection editing • Texture flattening • Local illumination change • Local color change • Seamless tiling • Conclusion

3. Introduction • The Poisson equation has been used extensively in computer vision. • The mathematical tool at the heart of this approach is the Poisson partial differential equation with Dirichlet boundary conditions which specifies the Laplacian of an unknown function over the domain of interest, along with the unknown function values over the boundary of domain. • Solving the Poisson equation also has an alternative interpretation as a minimization problem: • It computes the function whose gradient is the closet to some prescribed vector field-the guidance vector field-under given boundary conditions

4. Partial differential equation (PDE) • 方程式類別 判斷式 代表性範例 橢圓型 Laplace方程式 拋物線型 Poisson方程式 雙曲線型 波動方程式

5. 起始條件和邊界條件的分類 • 為了能獲得偏微分方程式之解答，其起始條件和邊界條件可依其特性區分為三類。 • 第一類：Dirichlet Condition • 第二類：Neumann condition • 第三類：Robbins condition

6. Dirichlet Condiction 若依變數(T)本身，在某個獨立變數值時，被指定，則此條件稱為Dirichlet Condition，亦稱為essential邊界條件。下圖為一典型的Dirichlet條件示意圖 由圖中很清楚的顯示，該平板之邊界條件為 圖 平版Dirichlet Condition 示意圖

7. Neumann condition Neumann condition係指依變數之變化率之邊界條件為定值，抑或獨立變數之函數之情況。 例如 或 Neumann型邊界條件，亦稱為natural boundary condition。

8. Poisson solution to guided interpolation • Guided Interpolation • Let S, a closed subset of , be the image definition domain, and let be a closed subset of S with boundary . • Let f* be a know scalar function defined over S minus the interior of and let f be an unknown scalar function defined over the interior of • Let v be a vector field defined over

9. Guided Interpolation • The simplest interpolant f of f* over is the membrane interpolant defined as the solution of the minimization problem: The minimizer must satisfy the associated Euler-Language equation

10. Guided Interpolation • Equation 2 is a Laplace equation with Dirichlet boundary conditions.

11. Guided Interpolation • A guided field is a vector field v used in an extended version of minimization problem (1): • This solution is the unique solution of following Poisson equation with Dirichlet boundary condition: • Three Poisson equations of the form (4) are solved independently in three color channels of the chosen color space.

12. Discrete Poisson solver • S now become finite point sets defined on an infinite discrete grid. • Note that S can include all the pixels of an image or only a subset of them. • For each pixel p in S, let Np be the set of its 4-connected neighbors which are in S, and let <p,q> denote a pixel pair such that • The boundary of is now • Let fp be the value of f at p and

13. Discrete Poisson solver • The finite difference discretization of (3) yields the following discrete, quadratic optimization problem: • Its solution satisfies the following simultaneous linear equation:

14. Discrete Poisson solver • Equation (7) form a classical, sparse (banded), symmetric, positive-definite system. • Results shown in this paper have been computed using either Gauss-Seidel iteration with successive overrelaxation or V-cycle multigrid. • Both methods are fast enough • i.e., 0.4s. per system on a Pentium 4 for a disk-shaped region of 60,000 pixels.

15. Seamless cloning • Importing gradients • The basic choice for the guidance field v is a gradient field taken directly from a source image. • Denoting by g this source image, the interpolation is performed under the guidance of • As for the numerical implementation, the continuous specification (9) translates into

16. Importing gradients • The seamless cloning tool thus obtained ensures the compliance of source and destination boundaries. • It can be used to conceal undesirable image features or to insert new elements in an image, but with much more flexibility

17. Importing gradients

18. Importing gradients

19. Seamless cloning • Mixing gradients • There are situations where it is desirable to combine properties of f* with those of g, for example to add objects with holes ,or partially transparent ones, on top of a textured or cluttered background. • An example is shown in Fig.6

20. Mixing gradients • One possible approach is to define the guidance field v as a linear combination of source and destination gradient fields but not good. • At each point of ,we retain the stronger of the variations in f* or in g, using the following guidance field: • The discrete counterpart of this guidance field is

21. Mixing gradients

22. Selection edition • Texture flattening • The image gradient is passed through a sparse sieve that retains only the most salient features: Where M is a binary mask turned on at a few locations of interest • A good choice for M is an edge detector, in which case the discrete version of (14), to be plugged into (7)

23. Selection edition

24. Selection editing • Local illumination changes • A natural extension is to restrict the correction to a selected region ,using appropriate Dirichlet conditions on • Using simplified version of the Fattal et al. transformation, the guidance field is defined in the log-domain by:

25. Selection editing

26. Selection editing • Local color changes • Given an original color image and a selection , two differently colored versions of this image can be mixed seamless: • One version provides the destination functions outside ,the other one provides the source function g to be modified within according to (10) • Poisson editing frees the user from the tedium of precise selection

27. Local color changes • Given a source color image g • (a) the destination image is set to be the luminance channel from g, • (b) the user selects a region containing the object, and this may be some what bigger than the actual object, • (c) the Poisson equation (10) is solved in each color channel.

28. Local color changes

29. Selection editing • Seamless tiling • When the domain is rectangular, its content can be made tileable by enforcing periodic boundary conditions with the Poisson solver. • The source image g is the original image, and the boundary conditions are derived from the boundary values of g, such that opposite sides of the rectangular domain correspond t identical Dirichlet conditions. • In Fig. 12, we have chosen ,and similarly for the east and west borders.

30. Seamless tiling

31. Conclusion • Using the generic framework of guided interpolation, we have introduced a variety of tools to edit in a seamless and effortless manner the contents of an image selection. • An important common characteristic of all these tools is that there is no need for precise object delineation, in contrast with the classic tools that address similar tasks.