Nate Andrysco

1. Efficient Data Structures for Interactive Visual Analysis of Large-Scale Unstructured and Meshfree Datasets on Many-Core Architectures Nate Andrysco Preliminary Examination

2. Breakdown of Title • Efficient Data Structures • Those data structures that will enable the following… • Visual Analysis • Visualization, analytics, “visual computing” • Large-Scale • Too large for single workstation • Unstructured and Meshfree • Focus will be those data sets produced through scientific simulation • Many-Core Architectures • Emphasis on GPU, but will include multi-core CPUs

3. Problem Statement • Achieving interactivity for the visual analysis of large unstructured and meshfree data is challenging due to the shear size and unorganized manner of the data sets. • Tradeoff between interactivity and accuracy

4. Example Problem • Fish tank CFD simulation • ~4.8 TB of data • ~35.2 million verts, ~23.6 mill. cells, 3,658 time steps • My goal is to allow for visual analysis that is both interactive and maintains visual accuracy of the original data set.

5. Thesis Statement • Novel dynamic, scalable, and streaming-oriented data structures specifically tailored to emerging many-core architectures  offers a solution to this long-standing problem. • I will bridge the gap between fully interactive data manipulation and faithful data representation. • I will test my general approach in a variety of applications with a focus on engineering problems involving large-scale computational simulations. • My methods will be applicable to the post-processing of these datasets but ultimately also to the simulation process itself.

6. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

7. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

8. Spatial Data Structures for Visual Analysis • Previous work: Dynamic GPU Data Structures • [1] Update octree based on view to show fine grain detail while maintaining performance • [2] Parallel construction of uniform grid to be used in ray tracing [1] Barakat, Garth, and Tricoche. Interative Computation and Rendering of Lagrangian Coherent Structures. (in progress) [2] Kalojanov and Slusallek. A Parallel Algorithm for Construction of Uniform Grids. HPG 2009

9. Spatial Data Structures for Visual Analysis • Previous work: Ray-tree traversal • CPU: Can make use of a stack • GPU: Stackless approaches • [3] KD-Restart: Restart at root if dead end reached • [3] KD-Backtrack: Follow parent pointers back to root • [4] KD-Jump: Keep track of where we have traversed using the bits of an integer [3] Foley and Sugerman. KD-Tree Acceleration Structures for a GPU Raytracer. Graphics Hardware 2005 [4] Hughes and Lim. Kd-Jump: a Path-Preserving Stackless Traversal for Faster Isosurface Raytracing on GPUs. Vis 2009

10. Spatial Data Structures for Visual Analysis • Matrix Trees • General idea: each level of the tree is represented as one more sparse matrix representations • Better speed/space than previous pointer-based or pointerless trees • Suited for CPU or GPU • Suitable for any underlying data type [5] Andrysco and Tricoche. Matrix Trees. EuroVis 2010

11. Spatial Data Structures for Visual Analysis • Ongoing/Future Work • Better sparse matrix representation • Apply matrix tree concept to general BSP trees • Parallel construction and dynamic updates • Streaming • Data and/or user viewpoint guided approximation [6] Andrysco and Tricoche. Implicit and Dynamic BSP Trees (in progress)

12. Spatial Data Structures for Visual Analysis • Previous work: Streaming • [7] Reorganization of mesh vertex pointers to allow for streaming of large triangle models • [8] Lossless compression of terrain data to expedite rendering [7] Isenburg and Lindstrom. Streaming Meshes. Vis 2005 [8] Lindstrom and Cohen. On-the-Fly Decompression and Rendering of Multiresolution Terrain. I3D 2010

13. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxels Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

14. Visualization and Graphics Algorithms on Massively Parallel Architectures • GPUs have surpassed workstation CPUs in pure computational power

15. Many-Core Technologies • GPUs have progressively become easier to program • Assembly → GLSL → CG → Brook / Sh → CUDA • CUDA (Compute Unified Device Architecture) • C-style interface for programming on Nvidia GPUs • Parallel algorithms defined as kernels • Lacks pointer support • Incredible speed-ups for computationally bound applications • Slow global memory access

16. Many-Core Technologies • OpenCL (Open Computing Language) • Heterogeneous programming across CPUs, GPUs, and any other many-core devices • Be able to use same code for CPU and GPU • Similar to programming in CUDA • Not a mature technology • Larrabee • Promises to allow for pointers and “fast” global memory reads • Does not look like it will be available anytime soon

17. Many-Core Technologies • Issues: • High performance computing is I/O bound • GPUs have relatively low memory and data transfer rates • Increased difficulty for building and maintaining data structures on GPU • I must account for these problems when designing the underlying algorithms and data structures

18. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

19. Voxel Data • Simplest data type • Fastest data type to interact with • Natural mapping to texture hardware • Often created from sampling another, more complicated data type • My focus will be in sparse voxel rendering http://en.wikipedia.org/wiki/Regular_grid

20. Sparse Voxel Rendering • Ray caster with voxels (axis aligned boxes) as the only primitive • [9] Octree with grids attached at leaves. Stream parts of the tree as needed. • A publication of a rendering engine idea from John Carmack (creator of Doom/Quake) [9] Crassin, Neyret, Lefebvre, and Eisemann. GigaVoxels: Ray-Guided Streaming for Efficient and Detailed Voxel Rendering. I3D 2009

21. Sparse Voxel Rendering • [10] Similar to [9], but at each leaf use two contours to better approximate the geometry [9] Crassin, Neyret, Lefebvre, and Eisemann. GigaVoxels: Ray-Guided Streaming for Efficient and Detailed Voxel Rendering. I3D 2009 [10] Laine and Karras. Efficient Sparse Voxel Octrees. I3D 2010

22. Sparse Voxel Rendering • Both methods can be improved in space and time by using Matrix Trees • They use pointer based trees and KD-Restart • Of note, [10]’s representation space requirement will approximately be reduced by 50% [10] Laine and Karras. Efficient Sparse Voxel Octrees. I3D 2010

23. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

24. Unstructured Triangle Meshes • Very common for representing objects in graphics • My focus will be in ray tracing http://en.wikipedia.org/wiki/Polygon_mesh

25. Ray Tracing • Previously had major bottleneck during the ray-primitive intersection tests • For dynamic scenes, the bottleneck can now be in data structure build time11 [11] Wald et al. State of the Art in Ray Tracing Animated Scenes. Eurographics 2009

26. Ray Tracing • Dynamic Matrix Trees • Exploit matrix structure for parallel construction and updates • Explore total build/render times for different tree types • Octrees, in general, will be outperformed by a well constructed KD-Tree, but might be much quicker to build

27. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Triangles • Unstructured Grids • Meshfree Data • Summary

28. Unstructured Grids • Common output of engineering simulations • Focus will be post-processing of CFD simulations http://en.wikipedia.org/wiki/Finite_element_method

29. CFD Unstructured Meshes • Previous work • [12] Use KD-Tree for random point queries. Use cell adjacency information for subsequent queries. • [5] Efficient tree storage/traversal on CPU/GPU. • [13] Bounding interval hierarchy on CPU/GPU. [5] Andrysco and Tricoche. Matrix Trees. EuroVis 2010 [12] Langbein, Scheuermann, and Tricoche. An Efficient Point Location Method for Visualization in Large Unstructured Grids. Vision, Modeling, and Visualization 2003 [13] Garth. Fast, Efficient Interpolation over Unstructured Grids on CPUs and GPUs (in submission)

30. CFD Unstructured Meshes • Biggest problem encountered is global memory lag • The vertex, cell, velocity, etc. vectors are not efficient due to the severe lack of memory coherence in CFD data sets • Directions of investigation: • Diagonalization of cell connection matrix • Flow coherent streaming of meshes • Sampling with guaranteed accuracy

31. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

32. Meshfree Data • Catch all term • Also common output of engineering simulations • My focus will be in granular materials http://www.sci.utah.edu/gallery2/v/csafe/2005/ambient_occlusion.jpg.html http://en.wikipedia.org/wiki/Finite_element_method

33. Granular Materials Problem • “Almost every branch of physical sciences or engineering involves some aspect of the behavior of particles” • “Many fundamental questions remain unanswered about the behavior and structural properties” [14] Andrysco, Tricoche, Rosato, and Ratnaswamy. Advanced Visualization Techniques for the Study of Granular Materials (in progress)

34. Granular Materials Problem • Granular material scientists desperately need advanced techniques to analyze their data • Excel graphs • Rendering of 2D data • Some basic analysis of underlying 3D structures http://www.liggghts.com/www.cfdem.com

35. Granular Materials Problem • In real life, can only observe/measure what is happening at the boundaries of a simulation • Use simulation software to generate data for full 3D systems • Hours, if not days of computation

36. Granular Material Analysis • Force Chains • The bulk of granular stress is located in only a small subset of particles • These particles tend to be touching and form chains [15] Aste, Di Matteo, and Galleani d’Agliano. Stress Transmission in Granular Matter. Journal of Physics: Condensed Matter 2002

37. Granular Material Analysis • Density • How closely particles are packed together • Tightly packed particles form crystal structures • Relaxation phenomenon • Change of density as small, repeated forces are applied [14] Andrysco, Tricoche, Rosato, and Ratnaswamy. Advanced Visualization Techniques for the Study of Granular Materials (in progress)

38. Granular Material Analysis • Layers • Curvature indicates force distribution [14] Andrysco, Tricoche, Rosato, and Ratnaswamy. Advanced Visualization Techniques for the Study of Granular Materials (in progress) [15] Aste, Di Matteo, and Galleani d’Agliano. Stress Transmission in Granular Matter. Journal of Physics: Condensed Matter 2002

39. Granular Material Visualization • Rendering 100,000+ particles • Occlusion problems • Discrete data • Graph rendering • Moving Least Squares • Underlying algorithms/data structures for both visualization and analysis will be general enough to fit into various simulation software

40. Presentation Outline • Spatial Data Structures for Visual Analysis • Visualization and Graphics Algorithms on Massively Parallel Architectures • Applications • Voxel Data • Unstructured Triangle Meshes • Unstructured Grids • Meshfree Data • Summary

41. Summary Future Work • Further data structure work • Implicit BSP Trees • Dynamic • Streaming • Explore many-core technologies • In particular, how data structures are created and updated • Allow for interactive analysis of CFD and granular data sets using combination of above

42. Future Paper Timeline • Siggraph Asia (May 11) – dynamic trees • TVCG (Summer) – initial granular work • TVCG (Summer) – streaming of dynamic CFD meshes • I3D (late October) – sparse voxel trees • EuroVis (December) – interactive analysis of large granular data sets • Vis (late March) – CFD work (combinatorial 3D vector field topology, compressive sensing, ?)

43. Bibliography [1] Barakat, Garth, and Tricoche. Interative Computation and Rendering of Lagrangian Coherent Structures. (in progress) [2] Kalojanov and Slusallek. A Parallel Algorithm for Construction of Uniform Grids. HPG 2009 [3] Foley and Sugerman. KD-Tree Acceleration Structures for a GPU Raytracer. Graphics Hardware 2005 [4] Hughes and Lim. Kd-Jump: a Path-Preserving Stackless Traversal for Faster IsosurfaceRaytracing on GPUs. Vis 2009 [5] *Andrysco and Tricoche. Matrix Trees. EuroVis 2010 [6] *Andrysco and Tricoche. Implicit and Dynamic BSP Trees (in progress) [7] Isenburg and Lindstrom. Streaming Meshes. Vis 2005 [8] Lindstrom and Cohen. On-the-Fly Decompression and Rendering of Multiresolution Terrain. I3D 2010 [9] Crassin, Neyret, Lefebvre, and Eisemann. GigaVoxels: Ray-Guided Streaming for Efficient and Detailed Voxel Rendering. I3D 2009 [10] Laine and Karras. Efficient Sparse Voxel Octrees. I3D 2010 [11] Wald et al. State of the Art in Ray Tracing Animated Scenes. Eurographics 2009 [12] Langbein, Scheuermann, and Tricoche. An Efficient Point Location Method for Visualization in Large Unstructured Grids. Vision, Modeling, and Visualization 2003 [13] Garth. Fast, Efficient Interpolation over Unstructured Grids on CPUs and GPUs (in submission) [14] *Andrysco, Tricoche, Rosato, and Ratnaswamy. Advanced Visualization Techniques for the Study of Granular Materials (in progress) [15] Aste, Di Matteo, and Galleanid’Agliano. Stress Transmission in Granular Matter. Journal of Physics: Condensed Matter 2002 [16] *Andrysco, Gurney, Benes, and Corbin. Visual Exploration of the Vulcan CO2 Data. CG&A Jan/Feb 2009 [17] *Aliaga, Benes, Vanegas, Andrysco. Interactive Reconfiguration of Urban Layouts. CG&A May/June 2008 [18] *Benes, Andrysco, Stava. Interactive Modeling of Virtual Ecosystems. Eurographics Workshop on Natural Phenomena 2009 [19] *Gurney et al. The Vulcan Project: Methods, Results, and Evaluation. Poster, North American Carbon Program All Investigator’s meeting. Feb 2009 [20] *Gurney et al. The Vulcan Project: Methods, Results, and Evaluation. Poster, Mid-Continent interim synthesis meeting. Jan 2009 [21] *Gurney et al. The Vulcan Project: Methods, Results, and Evaluation. Talk, American Geophysical Union meeting. Dec 2008 [22] *Andrysco, Benes, and Gurney. Interactive poster: Visual analytic techniques for CO2 emissions and concentrations in the United States. Poster, Vast 2008 [23] *Andrysco, Benes, Brisbin. Permeable and Absorbent Materials in Fluid Simulations. Poster, SCA 2008

44. Questions