All-Frequency Rendering of Dynamic, Spatially-Varying Reflectance

1. All-Frequency Rendering of Dynamic, Spatially-Varying Reflectance Jiaping Wang, PeiranRen, Minmin Gong, John Snyder, BainingGuo SIGGRAPH Asia 2009 Presenter: Kevin April 14, 2010

2. Authors PeiranRen [2] Minmin Gong [1] John Snyder [3] BainingGuo [2] Jiaping Wang [1] Microsoft Research Asia [1] Tsinghua University [2] Microsoft Research [3]

3. Introduction

4. Introduction • Final goal of real-time realistic rendering • Dynamic lighting • Changeable viewpoint • All-frequency effects • Dynamic, editable, and spatially-varying materials • Dynamic, deformable scenes Ng et al. SIGGRAPH ‘04 Wang et al. SIGGRAPH Asia ‘09 Sloan et al. SIGGRAPH ‘05

5. Introduction-2 • Challenges • BRDF complexity • Modeling the complex reflectance of real world materials • Light integration complexity • Integration over the whole hemisphere (cannot afford especially when environment maps are used) • Light transport complexity • Interreflection, shadows, …etc

6. Introduction-2 Off-line High Quality Real-time Precomputed Radiance Transfer Real-time Ray-tracing Programmable Graphics Hardware Wang et al. SIGGRAPH ‘09 Ritschelet al. SIGGRAPH Asia ‘08 Dachsbacheret al. SIGGRAPH ‘07

7. Introduction-3 • Precomputed Radiance Transfer (PRT) • The term comes from [Sloan ’02] • Precompute“light transport function” • Compress by basis (SH, Wavelet, SRBF…) • The computation of rendering equation reduces toinner/vector products in the run time

8. Introduction-4 • PRT timeline (02~05) Triple Product Zhou et al. SIGGRAPH ‘05 Ng et al. SIGGRAPH ‘03 Ng et al. SIGGRAPH ‘04 Wavelet Dynamic Scenes 02 03 04 05 Sloan et al. SIGGRAPH ‘05 Sloan et al. SIGGRAPH ‘02 Wang et al. EGSR ‘04 Pioneer, SH Sloan et al. SIGGRAPH ‘03 CPCA In-Out Fac. Lui et al. EGSR ‘04

9. Introduction-5 SVBRDF, BRDF-Editing BRDF-Editing With G.I. • PRT timeline (06~09) Wang et al. SIGGRAPH Asia ‘09 Ben-Artzi et al. ACMTOG ‘08 BRDF-Editing Green et al. EGSR ‘07 Green et al. I3D ‘06 Ben-Artzi et al. SIGGRAPH ‘06 SRBF 06 07 08 09 Ramamoorthi CG&V ‘09 Sun et al. SIGGRAPH ‘07 Wang et al. ACMTOG ‘06 BRDF-Editing With G.I. Survey Xu et al. TVCG ‘08 CTA Tsai et al. SIGGRAPH ‘06

10. Introduction-6 • Summary • Compression • PCA, Clustered PCA (CPCA), Clustered Tensor Approximation (CTA) … • Basis • Spherical harmonics (SH) • Wavelets • Zonal harmonics (ZH) • Spherical Radial Basis Function (SRBF)

11. Introduction-7 • Summary (cont.) • How to choose good basis for representation? • Can model all-frequency effects • Rotational invariant • Accuracy • Compact Fit clamped cosine term to basis Wang et al. SIGGRAPH Asia ‘09 – supplement materials

12. Introduction-8 • Spherical Gaussian (SG) • A type of SRBF, symmetric around a specific lobe axis • All advantages in the previous page • Inner product & cross product can be efficiently computed lobe axis lobe amplitude lobe sharpness

13. Introduction-9 • SG mixtures Single Lobe Two Lobes Multiple Lobes Rotated version

14. Overview

15. This Paper • Real-time • Dynamic(editable, change with time), spatially-varyingBRDFs • All-frequency effects from both environmental and local point lights • Static scenes • No interreflection

16. Contributions • Propose two new representations for BRDFsandVisibility to compact the size of data • SG mixtures for microfacet-based reflectance • Accurate and compact • Parametric BRDFs can be fit on-the-fly • Fast rotation, warping, and products in run time • SSDFs for visibility • Ghost-free, per-pixel interpolation • Dynamic local point lights

17. Approach Overview • Decoupling BRDF from visibility Represent Fit into SGs 6D SVBRDF 4D NDF (Microfacet) Mixture of SGs (Spherical Gaussians) Eigen- Vectors Map PCA Visibility (binary) SSDF (Spherical Signed Distance Functions)

18. SVBRDF Representation

19. SVBRDF Representation • Microfacet Model • Why use microfacet model? • General • Compact Geometry Term Normal Distribution Function Fresnel Term

20. SVBRDF Representation-2 • Reflectance representation using SGs • Parametric BRDFs (on-the-fly fitting) • Example: Cook-Torrance Model remaining factor (shadowing+Fresnel) NDF Fit into SGs Very Smooth High-Frequency

21. SVBRDF Representation-3 Cook-Torrance m=0.1 Cook-Torrance m=0.045 single-lobe SG (this paper) 256-term 64-term 16-term BRDF factorization ground truth

22. SVBRDF Representation-4 Ashikhmin-Shirley nu=8,nv=128 Ashikhmin-Shirley nu=25,nv=400 Ashikhmin-Shirley nu=75,nv=1200 7-lobe SG (this paper) ground truth 256-term 64-term BRDF factorization

23. SVBRDF Representation-5 • Accuracy • Parametric isotropic models

24. SVBRDF Representation-6 • Accuracy (cont.) • Parametric anisotropic models

25. SVBRDF Representation-7 • Reflectance representation using SGs • Measured BRDFs (preprocessing) • Usingtabulated NDF [Wang et al. SIGGRAPH ‘08] andshadowing factor S at each surface point • Compress shadowing function by PCA (8D) • Fit NDF with a small number of SGs

26. SVBRDF Representation-8 fabric green phenolic yellow albm bronze violet acrylic delrin steel ground truth 1SG 2SG 3SG 256-Term Fac. 64-Term Fac.

27. SVBRDF Representation-9 • Accuracy • Measured BRDFs

28. Visibility Representation

29. Visibility Representation • Spatially-varying visibility is represented with Spherical signed distance function (SSDF) • Directly interpolate binary visibilities will produce ghost effects • SSDF maps binary visibility to continuous function

30. Visibility Representation-2 • SSDF mapping • Positive: visible; negative: occluded • Value: the angular distance to the nearest direction t on the shadow boundary

31. Visibility Representation-3 Reconstructed Visibility δ: elevation angle Inner product / vector product of SGs and V’(i) can be efficiently evaluated in the run time!

32. Visibility Representation-3 • Compression • Using PCA • PCA coefficients are stored as vertex attributes and Interpolated to each pixel during rasterization • Eigenvectors are encoded in multiple textures PCA eigenvectors PCA coefficients

33. Visibility Representation-4 • Compression results Ray-Traced Uncompressed SSDF SSDF/PCA 384 Terms SSDF/PCA 144 Terms SSDF/PCA 48 Terms SSDF/PCA 16 Terms

34. Lighting Representation

35. Lighting Representation • Local point lights • Approximated with a single-lobe SG • Yielding a spatially-varying radiance field • Infinitely-distant light from direction I l: 3D light position s: intensity

36. Lighting Representation-2 • Distant environmental lighting • Apply to diffuse component • Fit the environment map with a SG mixture • [Tsai. et al. SIGGRAPH 2006] • Apply to specular component • Preconvolve environmental radiance with SG kernels • The run-time inner product is reduced to a MIPMAP texture fetch • [Kautz et al. EGWR 2000], [McAllister et al. GH 2002], [Green et al. I3D 2006]

37. Run-Time Rendering

38. Run-Time Rendering • Per-vertex attributes • Position, texture coordinates, local coordinate frame, PCA coefficient for SSDF • BRDF parameters, tabulated SG lobes (for measured BRDFs), and PCA-compressed shadow factors are stored in textures interploated across triangle mesh when passing from vertex shader to pixel shader

39. Run-Time Rendering-2 Cosine Approximation Lighting Approximation Spherical Warp Multiplied remaining factor BRDF Approximation SGs for NDF PCA coefficients of SSDFs Visibility Approximation Uncompressed on GPU

40. Experimental Results

41. Experimental Results • Data size and precomputation time

42. Experimental Results-2 • Per-vertex vs. per-pixel per-vertex shading per-pixel shading wireframe

43. Experimental Results-3 • Distant environmental light + nearby point light

44. Experimental Results-4 • Results for isotropic BRDFs

45. Experimental Results-5 • Results for anisotropic BRDFs

46. Experimental Results-6 • Video

47. Conclusion • Solution for highly-specular, spatially varying, dynamic materials, natural lighting, changeable viewpoints realistic rendering • SG mixtures for microfacet-based reflectance • Accurate and compact • Fast rotation, warping, and products in run time • SSDFs for visibility • Ghost-free, per-pixel interpolation • Allow sparse set of per-vertex visibility samples

48. End Thanks for your Attention

49. [Sloan et al. 02] • Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environment • Sloan, Kautz, and Snyder • SIGGRAPH 2002

50. [Sloan et al. 02]-2 • Algorithm (diffuse) • Express lighting as SH • Reflection Equation becomes • Definetransfer function and project to SH • Rendering reduces to Transfer vector (diffuse) Transfer Matrix (glossy)