Real-time Shading with Filtered Importance Sampling

1. Real-time Shading with Filtered Importance Sampling Mark Colbert Ph.D. Candidate, University of Central Florida Graduate Intern, DX10 Driver Group - VCG

2. Real-time Glossy Surface Reflection Gaming Lighting & Material Design Architecture Pre-visualization Car paints Entertainment

3. Video Folder Link

4. Goals High-fidelity interactive rendering (IBL/HDR) Accelerate performance using existingGPU hardware Minimal pre-computation Minimal code base Fast and easy integration into any production pipeline (Single GPU Shader!) Integrate the Illumination Integral IBL � rendering using high dynamic range (HDR) environment maps HDR � high dynamic range images using floats instead of U8s IBL � rendering using high dynamic range (HDR) environment maps HDR � high dynamic range images using floats instead of U8s

5. Illumination Integral Summation of all surrounding light rays multiplied by the material function Ignores occlusion/visibility Material defined by the Bidirectional Reflectance Distribution Function(BRDF)

6. Illumination Integral

7. Illumination Integral Evaluation Discretize environment into pixels applied onto a 2D surface with some mapping Multiply each environment map pixelby the BRDF for an outgoing direction Requires thousands of pixels to be multiplied and summed for thousands of pixels on the screen Computationally expensive

8. Fast Integral Evaluation Reduce the number of samples Borrow a technique from Stochastic Monte Carlo integration:Importance Sampling

9. Monte Carlo Integration Sampling Evaluating a function at a discrete point Point chosen by a uniform distribution Provides estimate of the integral Integral Approximation (Estimator) Average of all samples Infinite samples = Integral

10. Monte Carlo Estimator Estimator Assuming a uniform distribution

11. Monte Carlo Estimator Variation (variance) from the actual integral solution Most research in reducing variance Markov Chain Monte Carlo (MCMC) Importance Sampling

12. Importance Sample Given a priori knowledge of the integral, sample the function at points providing the best guess estimate Sample according to a weighted distribution

13. Which Distribution? Product of illumination and BRDF Optimal Too expensive to compute on GPU Illumination Too many samples needed forglossy reflection BRDF Provides best choice May over sample dark regions

14. Importance Sampling

15. Sampling BRDFs Analytical BRDFs Phong [1975] Lafortune [1997] Ward [1992] Cook-Torrence [1982] Compute through PDFs and CDFs

16. PDFs and CDFs PDF Probability Density Function Normalized form of the BRDF CDF Cumulative Distribution Function For multi-dimensional functions use Marginal PDF Conditional PDF

17. Discrete Example Note, we use multiple random numbers for each dimensionNote, we use multiple random numbers for each dimension

18. Monte Carlo Estimator Importance Sampling Estimator Divide by the Jacobian of the CDF (the PDF) since we are re-parameterizing the integral Intuitively, many samples representing 1Intuitively, many samples representing 1

19. Importance Sampling

20. Importance Sampling Problem Currently, unable to efficiently generate random numbers on the GPU Random number textures will fail due to the number of samples needed per pixel

21. Quasi-Random Numbers Numbers that are NOT conditionally independent Provides an optimal sampling pattern when using only a few samples Star low discrepancy sequences What does this mean?

22. Quasi-Random Numbers For small samples � stratified sampling is most optimal It is the most uniform a non-power-of-two number of samples can be distributed Low Star Discrepancy It is the most uniform a non-power-of-two number of samples can be distributed Low Star Discrepancy

23. Quasi-Random Numbers Many available sequences Hamersley Halton Folded Halton Folded Hamersley Radical inverse

24. Quasi-Random Sampling

25. Quasi-Random Importance Sampling Multiple mirror reflection artifact How do we remove the artifact/alias?Filter it!

26. Filtered Importance Sampling

27. Efficient Filtering Ideal Filter for sample k Filter across entire illumination integral Too expensive

28. Efficient Filtering Approximation Why? Since, BRDF ~ PDF = relatively constant Cosine low frequency

29. Filter Support How much to filter? Ideally precise region between a sample Isotropic Approximation Determine solid angle around sample Example 1: Uniform sampling We want to divide it evenly for uniform sampling

30. Uniform Filtered Sampling Equal sampling

31. BRDF-Proportional Filtered Sampling Example 2: Small PDF-value Less important region � More filtering Example 3: High PDF-value More important region � Use more samples (more variation/energy) Less filtering

32. Filter Implementation Pre-compute the filter Filtering can be expensive Need to utilize existing GPUaccelerated features Need minimal code base Mipmaps Remember needRemember need

33. Implementing Prefiltering Mipmap Pyramid image structure used for storingprefilters Often used to remove texturealiasing

34. Prefilter Optimization Need to keep prefiltering efficient Use a 2D map representation Cube Map ideal (low distortion) DX9/OGL filter decimation and reconstruction is independent per face (seam artifacts) Use dual paraboloid Captures entire environment in 2 images

35. Dual Paraboloids

36. Dual Paraboloids Still exhibit seams on the edges Scale the mapping

37. Solid Angle to Mipmap Level Convert support size to Mipmapped texture lookup Convert solid angle topixels Use ratio of solidangle of the sampleto the solid angle of anenvironment texel

38. Solid Angle to Mipmap Level Number of pixels is converted to Mipmap level by using logarithmic scaling Where ?p is the solid angle of a texel

39. Solid Angle to Mipmap Level Distortion or solid angle of a direction fora dual paraboloid mapping Found by the Jacobian of the dual paraboloid mapping

40. Filtered Importance Sampling

41. Pseudocode

42. Hybrid Solution More diffuse, slower convergence Use diffuse spherical harmonics Ramamoorthi and Hanrahan [2002] Simple vertex shader involving 3 vector-matrix-vector multiplications

43. Examples Spheres - Grace Probe

44. ExamplesBunny � Ennis Probe

45. RMS Error Follows standard Follows standard

46. Performance

47. Performance Analysis Dual Paraboloid causes branched texture lookups Dramatic differences between one face or two faces of paraboloidin display Can reduce performance closeto half

48. Future Applications SHINE - Surface Highlighting via Interactive NeuroEvolution Design materials directly from reflected light under an HDR environment map

49. More Information GPU-based Importance SamplingMark Colbert &Jaroslav Krivanek(Releasing at SIGGRAPH �07)

50. Signal Processing Theory Assume the illumination multiplied by the BRDF is a signal Analyze the DC term of Fourier spectrum Equivalent to illumination integral Define alias-free filter that for DC term Using isotropic approximations, filter is equivalent to intuitive meaning