Efficient Irradiance Normal Mapping

Efficient Irradiance Normal Mapping Ralf Habel, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of Technology

Motivation • Combining Light Mapping and Normal Mapping • Also know as: • Radiosity Normal Mapping • Directional Light Mapping • Spherical Harmonics Light Mapping • Popular in Games • Half-Life 2, Halo 3 … • Cheap and good looking: • Normal maps can be reused • Per vertex/per texel light map pipeline • Fast and trivial evaluation Ralf Habel

Motivation Light mapped Ralf Habel

Motivation Irradiance normal mapped Ralf Habel

Motivation Irradiance normal mapped no albedo Ralf Habel

Introduction • Goal: Represent irradiance on all surfaces for all possible directions (S x Ω) • Allows illumination to be stored sparsely similar to light mapping • Local variation is transported by normal maps • Representation: • Environment maps (piecewise linear) • Basis function sets (Spherical Harmonics) • Evaluation: Look up/calculate irradiance value in normal direction Ralf Habel

Irradiance Environment Maps • Ramamoorthi et al. 2001: Spherical Harmonics up to the quadratic band (RGB: 27 coefficients) is enough for an accurate representation (avg. error < 3%). • 9 RGB textures containing SH coefficients • Irradiance over all directions is a low frequency signal • Can we do better? • Only hemispherical signal (Ω+) needed on opaque surfaces • Other basis functions than Spherical Harmonics? Ralf Habel

Hemispherical bases • Set of functions defined over the hemisphere (Ω+) • Desired attributes for irradiance: • No discontinuities for smooth interpolation • Orthonormality: simplifies projections and other calculations (just like in Euclidian space) • Band structure for LOD/increasing accuracy (like Spherical Harmonics) • Not important: • Locality • High-frequency behavior Ralf Habel

Half-Life 2 Basis • Consists of 3 orthonormal cosine lobes (linear SRBFs) • Orthonormal over Ω+ • Equivalent to • Directional occlusion (one general cosine lobe) • Linear Spherical Harmonics band normed on Ω+ All require 3 coefficients and arelinear • No quadratic terms Ralf Habel

Hemispherical bases • General orthonormal hemispherical bases: • Hemispherical Harmonics [Gautron et al. 04] • Makhotkin Basis [Makhotkin 96] • All basis functions are 0 or constant on border of Ω+ due to generation through shifting • Non-polynomial • Zernike Basis [Koenderink 96] • Different band structure: 1,2,3..instead of 1,3,5.. • Non-polynomial Ralf Habel

Creating Directional Irradiance • We need irradiance on all surface points in all Ω+ directions: • Convolution with diffuse kernel far too expensive in Cartesian coordinates • Tens of millions of convolutions • Instead: Spherical Harmonics as an intermediate basis [Ramamoorthi 01, Basri and Jacobs 00] Ralf Habel

Creating Directional Irradiance • Create radiance estimate in precomputation • From photon mapping, path tracing, shadow mapping… • In tangent space (for tangent space normal maps) • Expand radiance into Spherical Harmonics by integrating against SH basis functions: Ralf Habel

Creating Directional Irradiance • Perform diffuse convolution directly in SH to get • Using Funk-Hecke Theorem, diffuse convolution is carried out by scaling SH coefficients in each band l with al : a0 = 1, a1 = 2/3, a2 = ¼, a3 = 0, a4 = -1/24 • There is never a cubic contribution in an SH irradiance signal • All l >=4 are very small • This is why SH up to the quadratic band is so efficient for irradiance! Ralf Habel

H-Basis • We would like something similar to SH on Ω+ • Polynomial • As fast as SH to evaluate • Same interpolation behavior • Orthonormal on Ω+ • Targeted for irradiance representation • Take a close look at SH functions and polynomial Hilbert space to derive basis functions Ralf Habel

H-Basis • SH functions that are symmetric to the z-axis are orthogonal on the hemisphere as well: • Y00,Y1-1,Y11,Y2-2,Y22 • Renormed to Ω+ Ralf Habel

H-Basis • Apply shifting to Y10 (cos θ = 2 cos θ -1) • Similar to Hemispherical Harmonics/ Makhotkin basis Ralf Habel

H-Basis • Results in Ω+ orthonormal polynomial basis with 1 constant, 3 linear and 2 quadratic basis functions • There is a mathematical rigorous derivation! Ralf Habel

H-Basis • Band structure allows to use only the constant+linear functions (H4) or all six (H6) similar to SH Ralf Habel

SH to H-Basis • Directional irradiance signals are calculated in SH • Project SH coefficient vector into H-Basis with matrix multiplication: • Sparse due to closeness to SH • Both bases are polynomial • No loss due to change in used function space Ralf Habel

Bases Comparison • Visual/perceptual comparison of all bases • Replace H-Basis with any other • In “very bad case” lighting situation • All basis functions are contributing • SH comparison is least-square hemispherically projected [Sloan 03] • Makes optimal use of SH on Ω+ • Shown with increasing number of coefficients • Only few are shown • See paper for all of them Ralf Habel

Bases Comparison: 3 Coefficients Ground truth Half-Life 2 (not how the game evaluates) Zernike 2 bands Ralf Habel

Bases Comparison: 4 Coefficients Ground truth H4 SH 2 bands (Ω+ projected) Makhotkin 2 bands (Artefacts at border) Ralf Habel

Bases Comparison: 6/9 Coefficients Ground truth H6 (6 coefficients) SH 3 bands (9 coefficients) Ralf Habel

Bases Comparison • Integrated Mean Square Error • averaged over 10 000 random irradiance signals • 6 coefficients is enough for a numerically accurate representation • What about difference between H4 andH6? Ralf Habel

H-Basis Comparison H4 - 4 coefficients Ralf Habel

H-Basis Comparison H6 - 6 coefficients Ralf Habel

Conclusion • H-Basis is very efficient and very simple solution for hemispherical irradiance signals • 4 coeffs. for perceptually accurate representation • Probably sufficient for almost all practical cases • 6 coeffs. for numerically accurate representation • Some lighting situations may benefit from 6 coeffs. • Orthonormality : • Shader LOD (functions are delocalized) • Easy expansion of other low frequency signals Ralf Habel

Future Work • There is a general mathematical description and derivation similar to Spherical Harmonics • H-Basis is a special case • Efficient generating procedures • Clarify correlations to SH • Other hemispherical signals • Visibility? • BRDFs? Ralf Habel

Thanks for your attention Ralf Habel

Efficient Irradiance Normal Mapping