The Future of Graphics Hardware

1. The Future of Graphics Hardware Turner Whitted Graphics Group/Hardware Devices Group

2. Bibliography • The Tyranny of Polygons and Pixels (1994) • Graphics Architecture (1996) • Display System Performance (1998) • The Rendering Problem: Architectures (1999) • IBR: Hardware and Software Issues (2000)

3. How did we get to where we are now What is the immediate future of graphics processors The distant future Graphics genealogy Why pixels? Why polygons Performance What does it mean to have to much? New graphics IBR New algorithms, displays, processors Overview

4. Abandoning and rediscovering real time Radiosity Ray tracing Blinn&Newell Warnock Bouknight Bui Tuong Phong Catmull Gouraud Watkins Real Time 1969 1979 1989

5. A brief history of RT rendering

6. Rapidly improving performance Slowly evolving architecture Conventional graphics processor

7. Why polygons? Why pixels? • (x0,y0), (x1,y1) is simple and convenient • Endpoints of a line • Incremental computation • Coherent addressing • (xi,yi) • Works for storage tubes, plotters, oscilloscopes, and digitized television • The classic pipeline: it’s so simple.

8. Implementing the pipeline SGI VGX

9. Generic processing element • Almost all graphics engines are built from DSP-like cores

10. Evolution of the classic pipeline Host process Graphics hardware

11. Mip-mapping address generation is not regular Filtering is ad-hoc Mapping computation is overly complex Bandwidth/filtering/computation

12. Basic MIP-map fetch

13. Multi-pass texturing Simple addition to conventional graphics processor … Combiner but fill rate is reduced proportional to the number of passes.

14. video refresh Graphics chip performance

15. Multi-texture sparse volume rendering Translate excess fill rate into layered volume textures Excess performance = new features from Jed Lengyel Microsoft Research

16. The immediate future … mixed media

17. Image engine or geometry engine?

18. 3D geometry vs. 2D images O • Rich content • Can rotate, scale, etc., but no significant 3D effects • Render geometry, paint texture • Good 3D rendering, but image (still) looks synthetic

19. What is IBR? • View independent re-rendering of acquired imagery • Re-use of synthetic imagery • Rendering from sample-based intermediate representations

20. Graphics/imaging continuum Concentric mosaics Image centric Geometry centric Sprites with depth View-dependent geometry View-dependent texture Light field Fixed geometry Lumigraph Polygon rendering + texture mapping Interpolation Warping

21. IBR issues • Hardware • Conventional architecture is not efficient for IBR functions • Software • How to map IBR onto graphics hardware (for the time being) • Consolidation of functions (instead of numerous ad hoc methods)

22. IBR claims • An image-based rendering architecture is beneficial. • A new architecture is easier to achieve than one might imagine.

23. Generic physical blocks Read Mapper Cache Common Memory Recon- Write Struction/ Filtering Cache

24. { Incremental Mapping Computation, e.g. [McMillan95] Generic IBR function blocks

25. Generic IBR function blocks { Programmable Function Blocks { Hardwired Function Block

26. Polygon engine functional blocks IBR maps onto a subset of the polygon processor

27. 3D text experiment • Extend IBR/volume rendering to text • Superior image reconstruction • Higher image quality than mip-mapped texture Olynyk, Mitchell, Snyder Microsoft Research

28. The distant future … year after next

29. Displays • 60 years of TV at 512x480 • 15 years of desktops at 4X TV resolution • 130 dpi LCDs this year • 200 dpi LCDs in the lab

30. My office • Imagine the whole window as a display

31. Large format displays that I can read from across the room or close up. Smart wallpaper What I really want

32. Displays get bigger Processors get smaller Need faster wire An architectural dilemma

33. Refresh bandwidth • Current Desktop • pixel density 100 dpi • size 12x9 in. • resolution 1280x1024 • bandwidth 236MB/sec • Wall size • size 108x72 in. • resolution 11520x8192 • bandwidth 16.9 GB/sec

34. More refresh bandwidth • Desktop • pixel density 600 dpi • size 12x9 in. • resolution 7200x5400 • bandwidth 7 GB/sec • Wall size • size 108x72 in. • resolution 64800x43200 • bandwidth 504 GB/sec

35. Actual resolution • There are two lies in the previous data: • the pixel density is not uniform • from a distance I can’t see peak resolution • up close I can’t see the whole screen • the update rate is too high by a factor of at least 10 because of temporal redundancy • use sprites

36. How do we efficiently get the graphics processor to where the image is needed No one knows. Architecture for large displays

37. Summary • Conventional pipeline is dated • Image processing dominates computation • Short-term evolution is trivial • Requires only functional changes • Distant future of graphics architecture • No Moore’s law for wire* • Requires structural changes

40. SGI O2 Entire graphics processor is embedded in the memory controller

41. Software for a new engine • Path 1): • Define options of a common operation • Path 2): [Snyder00] • Push fixed functions to the back end • Push programmable functions to the front end • Allow any reasonable algorithm

42. Taxonomy • Memory intensive • Processor intensive • Bandwidth intensive

43. Pixel Density for Single User • pixel density at center of screen: r0 x d

44. Todays graphics processor Near term graphics processor Future visual processor Evolving graphics processor