Real-Time Anti-Aliasing

1. Real-Time Anti-Aliasing Matt Pickering

2. Introduction • Anti-aliasing • Something you’ve probably encountered before • Why would we want to use anti-aliasing? • How is it done?

3. Overview • What is aliasing? • What causes it? • Using anti-aliasing to solve the problem • The techniques we can use • Pros and cons • Code stuff • Texture filtering • The future of anti-aliasing • Summary • Resources

4. Background • Aliasing can affect any kind of signal processing • Signal processing is a field of electrical engineering and applied mathematics • Not just images • Audio • Sensor data • Radio transmissions • Etc.

5. Context • We want to view a digital image • (The rendered frame of our game) • Image gets reconstructed by display device • Bottleneck • Our eyes and brain are also part of the reconstruction process

6. Description • Bottleneck: • Reconstruction resolution is too low • Detail is lostor distortion occurs • The distorted/low resolution image is an alias of the original

7. Description • Pre-aliasing • Aliasing that happens during sampling • Post-aliasing • Aliasing that happens during reconstruction

8. Spatial Aliasing • Display device limitations What we want What we get

9. Spatial Aliasing • Limited resolution What we want What we get

10. Temporal Aliasing • We can only show so many frames per second… • If objects in our scene are being transformed (moving/rotating) faster than our sample rate (frames per second) • We get temporal aliasing • Objects appear to jump or stutter instead of moving smoothly

11. Temporal Aliasing • Example: • Spoked wheel/propeller • “Wagon-wheel effect” • Appears to rotate much slower than it really is, or even appear to rotate backwards

12. Aliasing • Insufficient resolution • Insufficient sample rate

13. Anti-Aliasing • We want a way to “fool” the viewer into thinking the jagged edges and distortions in our scene are actually nice and smooth • How?

14. Techniques • FSAA • Super-sampling • Multi-sampling • Basic principle:

15. Super-sampling • Brute force • 4x super-sampling @ 800 x 600 = 1600 x 1200 • So for every pixel on the 800 x 600 screen • We’re effectively drawing 4 “sub-pixel” samples • Those 4 sub-pixels are then combined • We get the final pixel colour for the 800 x 600 screen

16. Super-sampling • Visualising super-sampling: • We want to draw the red objects • The small squares are our pixels • The pixels that are part red, part white get averaged into a shade of orange.

17. Super-sampling • Pros: • We get higher sampling resolution for the entire image • Cons: • It’s extremely expensive to make the image much larger than we actually want it to be, then scale it down by combining samples • Consider 16x SSAA @ 1920 x 1080… • 30720 x 17280 samples! • 

18. Multi-sampling • Optimisation over super-sampling • For 4x MSAA we still create just as many sub-pixels • But the pixel pipeline isn’t run for each sample • Instead, we first do the same calculation as rasterization: • If the edge of a polygon covers the centre of a pixel • Then that colour is the same as the polygon • Then, the percentage of the sampling points covered by the polygon is multiplied by that colour • This gives us the final colour for the pixel

19. Multi-sampling • Visualising multi-sampling: • Middle sample • Same as rasterization • Sub-pixel samples • 2 sub-pixels are covered • Final colour = 50% of poly colour

20. Multi-sampling • Pros: • Much lower overheads • Cons: • Multi-sampling extends the rasterized area to include all pixels in which at least some sampling points are covered • Even if the pixel centre is not covered! • This means that if a polygon edge covers some sub-pixels samples, but not the pixel centre, a sample gets taken from beyond the UV boundaries of the polygon

21. Centroid Multi-sampling • We can avoid these artefacts: • “Centroid multi-sampling” • This simply moves the centre sampling point to be between the sample points that are covered by the polygon.

22. Anti-Aliasing – DirectX

23. Anti-Aliasing – Shaders • HLSL: • Assembly shader:

24. Texture Filtering • It’s not just our polygons that have trouble being represented on a screen at an angle • The texture on the polygon is a grid-like arrangement of texels • If the texture is rotated or scaled, it won’t map to our screen pixels accurately, and we’ll get aliasing • Solution: • Mip-mapping

25. Bilinear/Trilinear Filtering • Bilinear Filtering • We can have multiple mip-maps being drawn on one large object (like terrain) • Problem: • We’re only blending pixels from within each mip-map • Very obvious seam between mip-map levels • Trilinear Filtering • Solution: • Blend with the neighbouring mip-map levels as well

26. Anisotropic Filtering • AA makes object edges look better • AF makes object interiors look better • Surfaces at oblique viewing angles IDirect3DDevice9::SetSamplerState() Texture index Mag/Min/Mip filter D3DTEXTF_ANISOTROPIC

27. Temporal Anti-aliasing • Remember we talked about stuff moving/rotating too fast? • How do we fix that? • Sample at least 2x as fast as the fastest moving object • Nyquist Frequency • So we either need very high framerates, or lower detail objects • (Not really an option) • Cheat • Store frames we’ve already drawn • Blend them with the current frame (post processing) • Basic motion blur – can cause “ghosting”

28. Future • MLAA • Still experimental, complicated, being developed by Intel • Post-processed anti-aliasing • Uses shape recognition to identify edges • Breaks edges down into L-shapes • Connect midpoint of secondary edge with the end point of primary edge • This gives us an area • Calculates blending weights • C_new = (1 – area) * C_old + area * C_opposite • This is an approximation of MLAA • Blend each pixel with its neighbours using calculated weights

29. Morphological Anti-Aliasing

30. Summary • We get aliasing when : • We can’t represent an image with a high enough resolution • We can’t sample fast enough • There’s 2 main strategies for dealing with this • Super-sampling • Draw a big image, then downsample by taking average colours • Multi-sampling • Only sample colour once per pixel, then multiply colour by % of covered sub-pixels • Texture filtering • Mip-mapping • Bilinear/Trilinear • Anisotropic filtering • Motion blur • MLAA

31. References MSAA in DirectX SDK Sample Browser includes an anti-aliasing sample + documentation http://www.directxtutorial.com/Tutorial11/B-A/BA2.aspx http://msdn.microsoft.com/en-us/library/bb173422(VS.85).aspx http://msdn.microsoft.com/en-us/library/bb206250(VS.85).aspx http://www.chadvernon.com/blog/resources/managed-directx-2/texture-compression-filters-and-transformations/ AA in general http://www.extremetech.com/article2/0,2845,2136956,00.asp http://www.pantherproducts.co.uk/Articles/Graphics/anti_aliasing.shtml http://www.bit-tech.net/hardware/2005/07/04/aliasing_filtering/1 Moiré pattern http://en.wikipedia.org/wiki/Moir%C3%A9_pattern MSAA http://alt.3dcenter.org/artikel/multisampling_anti-aliasing/index7_e.php MLAA http://www.eurogamer.net/articles/digitalfoundry-saboteur-aa-blog-entry http://www.realtimerendering.com/blog/morphological-antialiasing/ http://igm.univ-mlv.fr/~biri/mlaa-gpu/ http://www.youtube.com/watch?v=Z8UG7g8NRcw http://visual-computing.intel-research.net/publications/papers/2009/mlaa/mlaa.pdf

32. References • “Wagon-wheel effect” – this is cool  • http://www.youtube.com/watch?v=jHS9JGkEOmA • http://www.youtube.com/watch?v=rVSh-au_9aM • http://www.youtube.com/watch?v=LVwmtwZLG88 • http://www.youtube.com/watch?v=T055cp-JFUA • http://www.youtube.com/watch?v=oqUNd5wPGbU • Motion blur/Temporal AA • http://blogs.msdn.com/b/shawnhar/archive/2007/08/21/motion-blur.aspx • http://www.eurogamer.net/articles/digitalfoundry-halo-reach-tech-analysis-article?page=3 • Texture filtering • http://blogs.msdn.com/b/shawnhar/archive/2009/09/08/texture-filtering.aspx • Nyquist frequency • http://www.youtube.com/watch?v=Fy9dJgGCWZI • Basic summaries of AA/AF • http://www.youtube.com/watch?v=OLf03IMLsLI&NR • http://www.youtube.com/watch?v=YM3ieQHRYOc • Oldskool article on AF • http://www.nvnews.net/previews/geforce3/anisotropic.shtml

33. Questions?