Etree — A Database-oriented Method for Generating Large Octree Meshes

1. Etree — A Database-oriented Method for Generating Large Octree Meshes Tiankai Tu1 David R. O’Hallaron1,2 Julio C. López2 tutk@cs.cmu.edu droh@cs.cmu.edu jclopez@cs.cmu.edu 1School of Computer Science 2Electrical and Computer Engineering Department Carnegie Mellon University

2. t Physical model Simulation results Mesh Mesh generation Solver Visuali- zation Motivation Goal :Make it possible to run large-scale physical simulations on PC’s with limited physical memory Approach : Index and store the datasets in databases and compute on the databases directly Requires research at the intersection of computer systems, scientific computing, and databases

3. h2 h4 slave node h m h1 h3 h m e g b l j d f i j k l a b c a c i k d e f g h2 h3 h4 h1 element/octant master node Octree mesh generation • a compromise between structure and modeling power • balance requirement (2-to-1 constraint) • can be implemented in-core or out-of-core

4. transform balance construct etree library etree library etree library Etree mesh generation overview A general method for manipulating large out-of-core octrees by querying databases application-specific input element database unbalanced octree balanced octree node database

5. Etree library components • Etree API— a simple Application Programming Interface • Linear quadtree— an encoding scheme to assign keys to octants • Auto navigation— a mechanism for constructing an octree automatically • Local balancing— a technique that speeds up balancing operation • B-tree — a database index structure to store and access octants on disk Application (e.g.construct, balance) Etree API Etree library Linear quadtree Local balancing Auto navigation B-Tree

6. Etree API Unix file I/O style, three classes : • initialization and cleanup; example: etree_t *etree_open(const char *path, int flag, …); • octant-level operations; example: int etree_insert(etree_t *ep, location_t loc, void *value); • octree-level operations; example: int etree_balance(etree_t *ep, decom_t *baldecom);

7. h m i j k l a b c d e f g g k a j i h b f c d e l m Linear quadtree — how it works y 8 7 h m 6 5 4 e g b l j 3 d f 2 a c i k 1 0 x 0 1 2 3 4 5 6 7 8 • Morton code: maps n-dimensional points to one-dimensional scalars • Locational code: appends an octant’s level to the Morton code of its left-lower corner

8. Linear quadtree — how it works (cont’d) d’s left-lower corner (2, 2) y 8 binary form (010, 010) 7 h m 6 5 interleave the bits to obtain Morton code 4 e g b l j 3 d f 010 010 2 a c i k 1 0 00 11 00 x 0 1 2 3 4 5 6 7 8 append the level of d 001100_11

9. m x h m i j k l a b c d e f g Linear quadtree — applications m h e g b l j d f a c i k • an addressing scheme that clusters nearby octants • finding an octant without knowing its locational code • the order imposed by the locational code is the same as the preorder traversal of leafs in octree

10. Auto navigation — how it works Navigation octree • guided by an application function • an in-memory pointer-based octree • dynamically grows in the depth-first order • leaf octants are pruned and flushed to disk in preorder (increasing locational code order) • appends the octant to the database to avoid database search : octants not yet processed (in memory) : non-leaf octants being decomposed (in memory) : leaf octants (flushed to database)

11. Local balancing — how it works Operational steps partition the whole domain into equal-size blocks conduct internal balancing to enforce 2-to-1 constraint within each block (in memory a resident blocking array) perform boundary balancing to resolve interactions between adjacent blocks

12. Correctness of local balancing Claim: Interactions between adjacent blocks arealways absorbedby boundary octantsandwill not be propagated into the blocks Proof: See the paper

13. Evaluation — questions • Is the etree method feasible? • How does the running time vary with the physical memory size? • What is the impact of auto navigation? • What is the impact of local balancing?

14. Evaluation — methodology • We implemented an etree-based mesh generator to generate a family of finite element meshes for San Fernando valley earthquake wave propagation simulations

15. Evaluation — setup • All experiments are conducted on a PIII 1GHz machine running Linux 2.4.17. • The machine’s physical memory for the experiments ranges from 128MB to 880MB • Before each experiment, two 1.5 GB files are sequential scanned to ensure that the operating system’s buffer cache is flushed

16. Evaluation — etree feasibility All experiments are performed with 128MB physical memory Etree-based mesh generator running time and throughput • Generating a mesh with 13.6 million elements and of size 4.3GB in 2.6 hours seems reasonable • The overall throughput increases with mesh size

17. Evaluation — impact of memory size • Memory size does not have a significant impact on the running time • The etree method is not relying on the operating systems internal caching mechanism to achieve its performance

18. Evaluation — impact of auto navigation • Reducing B-tree buffer size does not increase the construction time • Auto navigation is not sensitive to B-tree buffer size

19. Evaluation — impact of local balancing • Achieves a speed-up factor ranging from 8 (SF1) to 28 (SF10) • Benefits from the one-time scan of the database and the efficient array-based neighbor finding algorithm

20. Related work • General octree algorithms: Samet 90 • Octree mesh: Shepard & Geoges 91, Bern et al. 90, Young et al. 91, Wang99 • Out-of-core octree method: Salmon 97 • Linear quadtree: Gargantini 82, Morton 66 • Space filling curve: Orenstein 84, Orenstein 86, Faloutsos & Roseman 89 • Large dataset processing: Freitag & Loy 99, Seamons & Winslett 96, Ferreira et al. 99, Kurc et al. 01, Choudhary et al. 99 Parashar & Browne 97

21. Conclusion and future work • Experiment results suggest that the etree method can generate large octree meshes on memory-limited machines in a reasonable amount of time • Incorporating existing database techniques (linear quadtree and B-tree) with new algorithms (auto navigation and local balancing) in a unified design scheme (the etree) can deliver new capability • We are porting the etree method to commercial database systems such as IBM DB2