K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

1. Photo-realistic Rendering and Global Illumination in Computer Graphics Spring 2012Ultimate Realism and Speed K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

2. Introduction • The quest for realism and speed has not yet come to an end. • Derivation of the rendering equation was made based on several restrictions imposed on light transport. • Wave effects could be ignored. • Radiance is conserved along its path between mutually visible surfaces • Light scattering happens instantaneously. • Scattered light has the same wavelength as the incident beam • It scatters from the same location where it hits a surface.

3. Introduction • However, such assumptions are not always true. • Phenomena such as polarization, diffraction, interference, fluorescence and phosphorescence, which do not fall within the assumptions for the rendering equation, should be considered in order to obtain high realism.

4. Beyond the Rendering Equation: Participating Media • It was assumed that radiance is conserved along its path between unoccluded surfaces. • The underlying idea was that all photons leaving the first surface needed to land on the second one because nothing could happen to them along their path of flight. • This is not always true!!! • Photons reflected or emitted by a car in front of us on the road will often not reach us. • They will rather be absorbed or scattered by billions of tiny water or fog droplets immersed in the air. • Light coming from the sky above will be scattered towards us. • The net effect is that distant objects fade away in gray. • Our assumption of radiance conservation between surfaces is only true in a vacuum. • In this case the relation between emitted radiance and incident radiance at mutually visible surface points x and y along direction θ is given by L(x->θ) = L(y<- -θ).

5. Beyond the Rendering Equation: Participating Media • If a vacuum is not filling the space between object surfaces, this will cause photons to change direction and to transform into other forms of energy. • Four processes are considered: volume emission, absorption, out-scattering and in-scattering.

6. Beyond the Rendering Equation: Participating Media • Considering such phenomena allows us to generalize the rendering equation.

7. Beyond the Rendering Equation: Volume Emission • The intensity by which a medium, like fire, glows can be characterized by a volume emittance function ε(z) (units [W/m3]). • It tells us basically how many photons per unit of volume and per unit of time are emitted at a point z in three-dimensional space. • There is a close relationship between the number of photons and energy. • Each photon of a fixed wavelength λ carries an energy quantum equal to 2πhc/λ; h is Planck’s constant and c is the speed of light. • Many interesting graphics effects are possible by modeling volume light sources. • Usually, volume emission is isotropic, meaning that the number of photons emitted in any direction around z is equal to ε(z)/4π (units [W/m3sr]).

8. Beyond the Rendering Equation: Volume Emission • Consider a point z = x + s·θ along a thin pencil connecting mutually visible surface points x and y. • The radiance added along direction θ due to volume emission in a pencil slice of infinitesimal thickness ds at z is

9. Beyond the Rendering Equation: Absorption • Photons traveling along our pencil form x to y will collide with the medium, causing them to be absorbed or change direction (scattering). • Absorption means that their energy is converted into a different kind of energy, for instance, kinetic energy of the particles in the medium. • Transformation into kinetic energy is observed at a macroscopic level as the medium heating up by radiation.

10. Beyond the Rendering Equation: Absorption • The probability that a photon gets absorbed in a volume, per unit of distance along its direction of propagation, is called the absorption coefficient σa(z) (units [1/m]). • A photon traveling a distance Δs in a medium has a chance σa Δs of being absorbed. • The absorption coefficient can vary from place to place. • Absorption is usually isotropic: a photon has the same chance of being absorbed regardless of its direction of flight. • Absorption causes the radiance along the thin pencil from x to y to decrease exponentially with distance. • Consider a pencil slice of thickness Δs at z = x + s·θ. • The number of photons entering the slice at z is proportional to the radiance L(z-> θ) along the pencil.

11. Beyond the Rendering Equation: Absorption • Assuming that the absorption coefficient is equal everywhere in the slice, a fraction σa(z)Δs of these photons will be absorbed. • The radiance coming out on the other side of the slice at z + s·θ will be • Ina homogeneous nonscattering and nonemissive medium, the reduced radiance at z along the pencil will be

12. Beyond the Rendering Equation: Absorption • This exponential decrease of radiance with distance is sometimes called Beer’s Law. • It is a good model for colored glass, for instance, and has been used for many years in classic ray tracing. • If the absorption varies along the considered photon path, Beer’s Law looks like this:

13. Beyond the Rendering Equation: Out-Scattering, Extinction Coefficient, and Albedo • The radiance along the pencil will not only reduce because of absorption, but also because photons will be scattered into other directions by the particles along their path. • The effect of out-scattering is almost identical to that of absorption. • One just needs to replace the absorption coefficient by the scattering coefficient σs(z) (units[1/m]), which indicates the probability of scattering per unit of distance along the photon path. • Rather than using σa(z) and σs(z), it is sometimes more convenient to describe the processes in a participating medium by means of the total extinction coefficient σt(z) and the albedo α(z).

14. Beyond the Rendering Equation: Out-Scattering, Extinction Coefficient, and Albedo • The extinction coefficient σt(z) = σa(z)+σs(z), (units[1/m]) gives us the probability per unit distance along the path of flight that a photon collides with the medium. • Then the reduced radiance at z is given by • In a homogeneous medium, the average distance between two subsequent collisions can be shown to be 1/σt (units[m]). • The average distance between subsequent collisions is called the mean free path.

15. Beyond the Rendering Equation: Out-Scattering, Extinction Coefficient, and Albedo • The albedo α(z)= σs(z)/σt(z) (dimensionless) describes the relative importance of scattering versus absorption. • It gives us the probability that a photon will be scattered rather than absorbed when colliding with the medium at z. • The albedo is the volume equivalent of the reflectivity ρ at surfaces. • Note that the extinction coefficient was not needed for describing surface scattering since all photons hitting a surface are supposed to scatter or to be absorbed. • In the absence of participating media, one could model the extinction coefficient by means of a Dirac delta function along the photon path. • It is zero everywhere, except at the first surface boundary met, where scattering or absorption happens for sure.

16. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • The out-scattered photons change direction and enter different pencils between surface points. In the same way, photons out-scattered from other pencils will enter the pencil between the x and y we are considering. This entry of photons due to scattering is called in-scattering. • Similar to volume emission, the intensity of in-scattering is described by a volume density Lvi(z->θ) (units [W/m3sr]). • The amount of in-scattered radiance in a pencil slice of thickness ds will be

17. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • A first condition for in-scattering at a location z is that there is scattering at z at all, σs(z)=α(z)σt(z)≠0. • The amount of in-scattered radiance further depends on the field radiance L(z,Ψ) along other direction Ψ at z and the phase function p(z,Ψ<->θ). • Field radiance is the usual concept of radiance. It describes the amount of light energy flux in a given direction per unit of solid angle and per unit area perpendicular to that direction. • The product of field radiance with the extinction coefficient Lv(z,Ψ)=L(z,Ψ)σt(z) describes the number of photons entering collisions with the medium at z per unit of time.

18. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Being a volume density, it is sometimes called the volume radiance (units [1/m3sr]). • Note that the volume radiance will be zero in empty space. • The field radiance, however, does not need to be zero and fulfills the law of radiance conservation in empty space. • For surface scattering, no distinction is needed between field radiance and surface radiance, since all photons interact at a surface. • Of these photons entering collision with the medium at z, a fraction α(z) will be scattered. • Unlike emission and absorption, scattering is usually not isotropic.

19. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Photons may scatter with higher intensity in certain directions than in others. • The phase function p(z,Ψ<->θ) plays the role of the BSDF for volume scattering. • Just like BSDFs, it is reciprocal, and energy conservation must be satisfied. • It is convenient to normalize the phase function so that its integral over all possible directions is one: • Energy conservation is then clearly satisfied, since α(z) < 1.

20. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Putting this together, we arrive a the following volume scattering equation: • The volume scattering equation is the volume equivalent of the surface scattering equation. • It describes how scattered volume radiance is the integral over all directions of the volume radiance Lv(z->Ψ), weighted with α(z)p(z,Ψ<->θ).

21. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Lv(z->Ψ) is the volume equivalent of surface radiance. • α(z)p(z,Ψ<->θ) is the equivalent of the BSDF.

22. Beyond the Rendering Equation: The Rendering Equation in the Presence of Participating Media • The above two equations model how volume emission and in-scattering add radiance along a ray from x to y. • This equation describes how radiance is reduced due to absorption and out-scattering.

23. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Not only the surface radiance L(x->θ) inserted into the pencil at x is reduced in this way, but also all radiance inserted along the pencil due to in-scattering and volume emission is reduced. • The combined effect is • For compactness, let z = x+rθ and

24. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • The transmittance τ(z,y) indicates how radiance is attenuated between z and y: • The equation below replaces the law of radiance conservation in the presence of participating media.

25. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • The rendering equation was obtained by using the law of radiance conservation in order to replace the incoming radiance L(x<-Ψ) in by the outgoing radiance L(y->-Ψ) at the first surface point y seen from x in the direction Ψ. • Then the following rendering equation is obtained in the presence of participating media

26. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • The rendering equation was obtained by using the law of radiance conservation in order to replace the incoming radiance L(x<-Ψ) in by the outgoing radiance L(y->-Ψ) at the first surface point y seen from x in the direction Ψ.

27. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Then the following rendering equation is obtained in the presence of participating media

28. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • The in-scattered radiance Lvi in L+(z->-Ψ)=ε(z)/4π+Lvi(z->-Ψ) is expressed in terms of field radiance. • Field radiance is expressed in terms of surface radiance and volume emitted and in-scattered radiance elsewhere in the volume. • Participating media can be handled by extending the rendering equation in two ways: • Attenuation of radiance received from other surfaces: the factor τ(x,y) in the former integral. • A volume contribution: the latter integral.

29. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Spatial Formulation • In order to better understand how to trace photon trajectories in the presence of participating media, it is instructive to transform the integrals to a surface and volume integral. • The relation between differential solid angle and surface is available: rxy2dωΨ=V(x,y)cos(-Ψ,Ny)dAy. • A similar relationship exists between drdωΨ and differential volume: rxz2drdωΨ=V(x,z)dVz.

30. Beyond the Rendering Equation: In-Scattering, Field-and Volume-Radiance, and the Phase Function • Spatial Formulation

31. Beyond the Rendering Equation: Global Illumination Algorithms for Participating Media • There exist two types of methods: deterministic and stochastic • Deterministic Approaches • Classic and hierarchical radiosity methods have been extended to handle participating media bydiscretizing the volume integral above into volume elements and assuming that the radiance in each volume element is isotropic. • Many other deterministic approaches have been proposed based on spherical harmonics and discrete ordinates methods. • Deterministic approaches are valuable in relatively easy settings, for instance, homogeneous media with isotropic scattering, or simple geometries.

32. Beyond the Rendering Equation: Global Illumination Algorithms for Participating Media • Stochastic Approaches • Various authors have proposed extensions to path tracing to handle participating media. • The extension of bidirectional path tracing to handle participating media has been proposed. • These path-tracing approaches are flexible and accurate. • But they become enormously costly in optically thick media, where photons suffer many collisions and trajectories are long. • A good compromise between accuracy and speed is offered by volume photon density estimation methods. • In particular, the extension of photon mapping to participating media is a reliable and affordable method capable of rendering highly advanced effects such as volume caustics.

33. Beyond the Rendering Equation: Global Illumination Algorithms for Participating Media • Monte Carlo and volume photon density estimation are methods of choice for optically thin media, in which photons undergo only relatively few collisions. • For optically thick media, they become intractable. • Highly scattering optically thick media can be handled with the diffusion approximation.

34. Beyond the Rendering Equation: Tracing Photon Trajectories in Participating Media • Photon trajectory tracing can be extended to deal with participating media. • Sampling volume emission • Light particles may not only be emitted at surfaces, but also in midspace. • Frist, a decision needs to be made whether to sample surface emission or volume emission. • This decision can be a random decision based on the relative amount of self-emitted power by surfaces and volumes. • If volume emission is to be sampled, a location somewhere in midspace needs to be selected, based on the volume emission density ε(z) • Bright spots in the participating media shall give birth to more photons than dim regions. • Finally, a direction needs to be sampled at the chosen location. • Since volume emission is usually isotropic, a direction can be sampled with uniform probability across a sphere.

35. Beyond the Rendering Equation: Tracing Photon Trajectories in Participating Media • Sampling a next collision location • In the absence of participating media, aphotonemitted from point x into direction θ always collides on the first surface seen from x along the direction θ. • With participating media, however, scattering and absorption may also happen at every location in the volume along a line to the first surface hit. • A good way to handle this problem is to sample a distance along the ray from x into θ, based on the transmittance factor. • One draws a uniformly distributed random number ζ∈[0,1) and finds the distance τ corresponding to

36. Beyond the Rendering Equation: Tracing Photon Trajectories in Participating Media • Sampling a next collision location • In a homogeneous medium, r = -log(1-ζ)/σt. • In a heterogeneous medium, sampling such a distance is less trivial. • It can be done exactly if the extinction coefficient is given as a voxel grid, by extending ray-grid traversal algorithms. • For procedurally generated media, one can step along the ray in small, possibly adaptively chosen intervals. • If the selected distance becomes greater than or equal to the distance to the first surface hit point of the ray, surface scattering shall be selected as the next event. • If the sampled distance is nearer, volume scattering is chosen.

37. Beyond the Rendering Equation: Tracing Photon Trajectories in Participating Media • Sampling a next collision location

38. Beyond the Rendering Equation: Tracing Photon Trajectories in Participating Media • Sampling scattering or absorption • Sampling scattering or absorption in a volume is pretty much the same as for surfaces. • The decision whether to scatter or absorb will be based on the albedo α(z) for volumes just like the reflectivity ρ(z) is used for surfaces. • Sampling a scattered direction is done by sampling the phase function p(z,θ<->Ψ) for volumes.

39. Beyond the Rendering Equation: Tracing Photon Trajectories in Participating Media • Connecting path vertices • Algorithms such as path tracing and bidirectional path tracing require us to connect path vertices, for instance, a surface or volume hit with a light source position for a shadow ray. • Without participating media, the contribution associated with such a connection between points x and y is • In the presence of participating media, the contribution shall be With Cx(θ)=cos(Nx,θ) if x is a surface point or 1 if it is a volume point.

40. Beyond the Rendering Equation: Volume Photon Density Estimation • In order to render participating media, it is necessary to estimate the volume density of photons colliding with the medium in midspace. • A histogram method, would discretize the space into volume bins and count photon collisions in each bin. • The ratio of the number of photon hits over the volume of the bin is an estimate for the volume radiance in the bin. • Photon mapping has been extended with a volume photon map, a third kd-tree for storing photon hit points, in the same spirit as the caustic and global photon map. • The volume photon map contains the photons that collide with the medium in midspace. • Rather than finding the smallest disc containing a given number N of photon hit points, as was done on surfaces, one will search for the smallest sphere containing volume photons around a query location. • Again, the ratio of the number of photons and the volume of the sphere yields an estimate for the volume radiance.

41. Beyond the Rendering Equation: Volume Photon Density Estimation • Viewing Precomputed Illumination in Participating Media

42. Beyond the Rendering Equation: Light Transport as a Diffusion Process • The volume rendering equation is not the only way light transport can be described mathematically. • An interesting alternative is to consider the flow of light energy as a diffusion process. • This point of view has been shown to result in efficient algorithms for dealing with highly scattering optically thick participating media, such as clouds.

43. Beyond the Rendering Equation: Light Transport as a Diffusion Process • Diffuse Approximation Method. • The idea is to split field radiance in a participating media into two contributions, which are computed separately: • The first part Lr, called reduced radiance, is the radiance that reaches point x directly from a light source, or from the boundary of the participating medium. It needs to be computed first. • The second part, Ld, called diffuse radiance, is radiance scattered one or more times in the medium. • In a highly scattering optically thick medium, the computation of diffuse radiance is hard to do according to the rendering equation. • Multiple scattering, however, tends to smear out the angular dependence of diffuse radiance. • Indeed, each time a photon scatters in the medium, its direction is randomly changed as dictated by the phase function. • After many scattering events, the probability of finding the photon traveling in any direction will be nearly uniform.

44. Beyond the Rendering Equation: Light Transport as a Diffusion Process • One approximates diffuse radiance by the following function, which varies only a little with direction: • Ud(x) represents the average diffuse radiance at x

45. Beyond the Rendering Equation: Light Transport as a Diffusion Process • The diffuse flux vector models the direction and magnitude of the multiple scattered light energy flow through x. • The average diffuse radiance Ud(x) fulfills a steady-state diffusion equation:

46. Beyond the Rendering Equation: Light Transport as a Diffusion Process • Once the diffusion equation has been solved, the reduced radiance and the gradient of Ud(x) allows us to compute the flux vector wherever it is needed. • The flux vector, in turn, yields the radiosity flowing through any given real or imaginary surface boundary.