Relative Magnitude of Gaussian Curvature from Shading Images Using Neural Network

1. Relative Magnitude of Gaussian Curvature from Shading ImagesUsing Neural Network Yuji Iwahori1, Shinji Fukui2 , Chie Fujitani1, Yoshinori Adachi1 and Robert J. Woodham3 1 Chubu University, Japan 2Aichi University of Education, Japan 3 University of British Columbia, Canada

2. Introduction Surface Curvature is the invariant feature for the viewing direction. It can be used to many applications in the field of computer vision. • Using a fixed camera, multiple light sources, that is, multiple shading images are used as input. • Empirical implementation with the neural network has been performed to obtain the relative magnitude of Gaussian Curvature. 2D Shading Images Recovering Gaussian Curvature for 3D Shape description

3. Previous Approaches (1) • [Woodham, 1994] Look Up Table (LUT) based Photometric Stereo: Empirical estimation of surface gradient and surface curvature under the different light source directions. • [Iwahori, Woodham, 1995] Neural network implementation of Photometric Stereo (to obtain the surface orientation) with PCA (principal component analysis) to the co-linear light sources to remove the correlation

4. Previous Approaches (2) • [Angelopoulou and Wolff, 1998] Recovering the sign of Gaussian curvature without knowing the surface gradient • [Okatani and Deguchi, 1998] also formulated the method to recover the sign of Gaussian curvature. These approaches are applicable for the diffuse reflectance. Also, the sign of GC is simple but limited information.

5. New Proposed Approach • Extension of [Iwahori, Fukui, Woodham, 1998] which proposed the classification of surface curvature using neural network. • Empirical approach using Radial Basis Function neural network (RBF-NN) to perform the non-parametric functional approximation. • In addition, the relative magnitude of Gaussian curvature could be obtained for the general surface reflectance (not limited to the diffuse reflectance).

6. Principle (Empirical Constraint) • Orthographic projection is assumed. • Test object and a calibration sphere with the same reflectance property, are observed. As far as the observed image irradiance (intensity) vector is the same for both objects, the corresponding surface normal vector should be the same. • Three shading images with different light sources are used to determine the curvature sign and the relative magnitude.

7. RBF neural network • The corresponding coordinate (x,y) on a sphere which has the same triple of the image irradiance for a test object and a sphere can used to obtain the information of the surface curvature. • RBF neural network is used to realize this purpose. • Features: • Non-parametric functional approximation • (interpolation) in multidimensional spaces

8. f ( E - c1 ) xsph S f ( E - c2 ) … … … ysph S f ( E - cm ) E1 E2 E3 RBF neural network RBF NN learns the mapping of (E1,E2,E3) to (x,y) of each point on a sphere. After the learning, given the triple of (E1,E2,E3) of local five points on a test object, we get the corresponding (x,y) onto a sphere.

9. Convex (b) Concave (d) plane Local Surface Curvature G>0 G>0 G<0 (c) hyperbolic G=0 G=0 G=0 (e) convex parabolic (f) concave parabolic

10. Mapping onto Sphere by Neural Net (1) Local convex surface (2) Local concave surface

11. Mapping onto Sphere by Neural Net (3) Local hyperbolic surface (4) Local plane

12. Mapping onto Sphere by Neural Net (5) Convex parabolic surface (6) Concave parabolic surface

13. Relative Magnitude of Gaussian Curvature G • It can be estimated from the area value surrounded by four mapped points onto a sphere. Let be the height, This is the definition of G.

14. Relative Magnitude of Gaussian Curvature G The area value S surrounded by four mapped points is

15. Experiments (Procedures) • Three shading images are used for a test object and a sphere with the same reflectance property, • Input images are taken under the same illuminating conditions for both a test object and a sphere. • Neural network learning is done for a sphere. • Generalization for a test object is done.. • Classification and the Encoding the relative magnitude are done.

16. Experiments (Results)

17. Evaluation (Simulation) Normalized true distribution of Gaussian curvature from Hessian matrix of 2-D Sinc Function Normalized estimated distribution of Gaussian curvature based on the proposed approach

18. Evaluation (Accuracy) Red curve is true, while blue curve is estimated curve of the average cross section.

19. Conclusion • A new method is proposed to recover the relative magnitude of Gaussian curvature with the neural network implementation. • Entire approach is empirical under the condition that no explicit assumptions are not used for the surface reflectance function nor the illuminating directions. • Robust results are directly obtained in addition to the classification of local surfaces.

20. Further works • Extension without using any calibration object. • For a test object with multiple color textures. • For cast-shadow or inter-reflection. Thank you