Shape from Shading #1

1. Shape from Shading #1

2. Topics Reflectance map and Photometric stereo • irradiance and radiance • basic concepts of reflection • reflection map • photometric stereo

3. radiance A R Θ Brightness amount of light falling on a surface falling energy measured by a unit surface area [watt/m2] irradiance amount of light radiated from a surface emitting energy measured from a unit forshorted light source surface area to a unit solid angle [watt/ m2・Sr] solid angle --- steradian

4. N i e V L g i=incidence angle e=emitting angle g=phase angle Reflection geometry • irradiance at a pixel depends on • illumination • materials • geometry • under the same illuminate condition, we observe irradiance difference on the same material surface • there is a relationship between pixel irradiance and geometry • Reflectance geometry L=illumination N=normal V=viewer

5. Gradient space reflection functions are defined in the local coordinate system(e,i,g) For our development, we will redefine the reflectance geometry in the gradientspace viewers is always on the Z axis q p

6. surface reflection body reflection incident light internal pigment air body Surface and body reflection • Surface reflection and body reflection surface reflection=gloss,highlights very directional(specular) body reflection =object color all direction(diffuse) plastic, paint have both metal has only surface reflection

7. Model for body reflection Diffuse---scatters in all directions common approximation: equal in all directions “lambertian”Lambertian’s cosine law “perfectly diffuse reflector” reflectance=constant * geometric factor f(i,e,g) = Kb * cos i why cos i ? • angle of incidence affects “density” of illumination.(irradiance) irradiance=light/area light=1 area=1/cos i irradiance = cos i

9. q iso-brightness contour 0.5 0.8 0.9 p Calculating a reflection map (Lambertian) • for each(p,q), N=(p,q,1) • light source direction, S=

11. q p Reflectance map(continue) • Lambertian Self-shadow line

12. N S’ S Surface reflection metals have the only surface reflection dielectrics(plastics,paint)have the surface reflection as well as the body reflection simplest approximation: perfect mirror i i reflection is specular direction, S’ S’ is coplanar with S,N SN = i = NS’: opposite sides

13. Real surfaces are rough : light scatters tells amount of light at each angle Phong’s model calculate angle between S’ and V ---α f-surface(i,e,g)=Ks * cos α typical : n = 10 to 500 heuristic model n=1 3 n n Cos α 5

15. Reflectance map q bright dark p q p

16. Better model of surface reflection • Phong’s model R=Ks cosnα not based on physics just looks OK for graphics, not really accurate off-specular effect Phong’s modelreal surface composite surface reflection Phong’s modelreal surface Torrance and Sparrow --- geometrical optics Beckmann and Spizzichino --- physical optics

17. α Torrance and Sparrow model • Geometrical optics • a collection of planar mirror-like facets • surface reflection caused only by these microfacets • their sizes are much larger than wave length average normal direction facet normal microfacet facet slopes to be normally distributed V-shaped valleys

18. Surface reflectance • surface reflectance = constant for material • effect of one ray • % not blocked by others (geometrical attenuation) • % of all facets involved

19. i e Effect of one ray incoming energy = A cos i outgoing energy =(A cos i) / (A cos e) =cos i / cos e

20. 1) masking 2) shadowing Geometric attenuation g(i,e)

21. N reflection distribution facet normal distribution α i % of all facets involved α

22. Beckmann and Spizzichino modelphysical optics surface is continuous h(x,y) light is wave reflection off of surface roughness is amplitude and spatial frequency of variations in h(x,y) E(x,y,z) “field” of light energy surface is assumed to be a perfect conductor(metal) --- > Maxwell’s equation exact solution is vicious integral where

23. Our model (Nayer,Ikeuchi,Kanade89) • Torrance and Sparrow + Beckmann and Spizzichino • diffuse lobe --- cosine function • specular spike --- delta function • specular lobe --- gaussian function

25. Calculating reflectance mapspecular lobe + diffuse lobe recall q Lambertian (diffuse lobe) contours p Specular peak

26. Shape-from-shading recover object shape (orientation) from image irradiance (brightness) 0.8 (p,q,1) brightness surface orientation E(x,y)=R(p,q) -- image irradiance equation 0.8 q p • gives one constraint on the gradient space at each pixel • --- > ill-posed problem (cannot solve !!!!!)

27. q p Photomotric stereo • one image irradiant equation gives only one constraint • --- > use multiple equations at each pixel. • take multiple images from the same points under different light source directions q p • recall different light source directions give different reflectance map • at each pixel, multiple irradiance values

28. PhotometricStereo

29. Analytical solution • real world gives complicated light source direction • --- > look-up table method

30. Look-up table method

32. Summary • Basic concepts of reflection • radiance and irradiance • reflection geometry • surface reflection and body reflection • Shape-from-shading problem • reflectance map • image irradiance equation • photometric stereo

33. Shape-from-shading #2

34. Shape-from-shading #2 get a depth map from a needle map get a needle map from a single image

35. 1 Depth from surface orientation 1 dimensional case recall 2 dimensional case

36. Recovering depth map from a needle map(direct integration method) Photometric stereo gives a needle map assume a depth at the origin get depth along the x of the needle map get the depth map

37. Direct integration rapid accumulates errors

38. Relaxation method • observed orientation (p,q) should be same as those of the depth map (zx,zy) • reduce the total error within a boundary (the calculus of variations See Horn pp.469-474) • an iterative formula

39. iterative method Relaxation method (Example) needle map brightness depth map

40. Jacobi’s method Iteratively computing the equation itself  convergence is very slow The number of iteration needed to converge is when the error decreased to 10-p times, for the pixels whose size is J×J Eg. r=1200000 when J=400, p=15

41. Gauss-Seidel method Jacobi: Gauss-Seidel: scan-line order checkerboard pattern Iteration number: Eg. r=600000 when J=400, p=15

42. SOR (successive overrelaxation) method same as Gauss-Seidel ω: overrelaxation parameter Converge if 0<ω<2 0<ω<1 (underrelaxation): slower than Gauss-Seidel 1<ω<2 (overrelaxation): faster than Gauss-Seidel Optimal ω: Eg. ω=1.984 when J=400 Iteration number: Eg. r=2000 when J=400, p=15 Chebyshev acceleration (faster convergence): change ω properly for each iteration

43. ADI (alternating-direction implicit) method Iteration number: Eg. r=104 when J=400, p=15 : changed properly for each iteration if =0  Gauss-Seidel H(1,y) * H(x,1) * +2 +2 -1 -1 -1 -1 … … … … … … … … … … … … +2 +2 -1 -1 H(J,y) -1 -1 H(x,J) * * A: tridiagonal matrix (given) b: vector (given) x: solve Ax=b by using linear system solver (forward substitution)  O(N)

44. Natural boundary condition Height known  just use it Boundary: Height unknown  natural boundary condition Natural boundary condition of "gradient-to-height problem" is s… arc length of boundary ·… dot product … the normal of boundary Calculate the height "H" at the boundary by Algorithm: or for each iteration Natural boundary condition Same height boundary Slow convergence Fast convergence [Truth] [Correct result] [Wrong result]

45. Shape-from-shading with a single view Photometric stereo uses multiple images. Is there a way to recover shape from a single image? Yes, there is a way. 1. characteristics strip expansion method:obtain surface orientation along characteristics strips of image irradiance equation (Horn 75) 2. relaxation method:obtain surface orientation using image irradiance equation and smoothness constraint (Ikeuchi and Horn 81) 3. global method: assume a surface is a part of sphere (Pentland 83)

46. Characteristic strip expansion method the steep descent direction of the reflectance map (gradient space) the steep descent direction of the image brightness (image brightness)

47. SDD of RM SDD of IB SDD of RM SDD of IB Characteristic Strip • move towards the SDD of • the reflectance map on the image plane • move towards the SDD of • the image brightness on the gradient space

48. Proof 1. Taylor expansion of p(x,y) and q(x,y) 2. derivative of the image irradiance equation

49. Move towards the SDD of the reflectance map on the image plane then, what happen to (p(x,y),q(x,y)) ? move towards the SDD of the image brightness on the gradient space on the image on

50. y q x p 0.5 0.1 0.4 0.3 0.2 0.1 0.2 0.3 0.4 from a known point, (you know (p,q) and E) you can determine (p,q) along a characteristic strip