Production Visualization Tools for Adaptive Mesh Refinement Data

1. Production Visualization Tools for Adaptive Mesh Refinement Data Gunther H. WeberHank Childs, Kathleen Bonnell, Jeremy Meredith, Mark Miller, Brad Whitlock, E. Wes Bethel www.vacet.org

2. Mathematics • PI: Phil Colella (LBNL), SciDAC Applied Partial Differential Equations Center • Accomplishment(s) • Software engineering to “bring product to market”. Performance improvements, interface enhancements, file readers, visual data exploration techniques. • Science Impact • Direct cost savings: APDEC no longer uses its own resources (e.g., FTEs) to develop, maintain, and support AMR visualization software. • Ability to perform AMR visualization on large, time-varying data, and using parallel platforms. • Benefits propagate to all APDEC stakeholders.

3. Increasing use of adaptive mesh refinement (AMR) simulations Various DOE groups develop and use AMR codes Differential Equations Center for Technologies SciDAC LBNL Center for Computational Sciences and Engineering Mainstream visualization tools lack AMR support AMR code development team invest significant effort in development of their own tools “Undesired” effort that consumes/diverts resources VACET Goal: Provide replacement for custom tools Eliminate Need for Custom AMR Vis Tools

4. Basis of Our Efforts: VisIt • Richly featured visualization and analysis tool for large data sets • Data-parallel client server model, distribution on per patch-basis • Use of meta-data / contracts to reduce amount of processed data • Built for five use cases: • Data exploration • Visual debugging • Quantitative analysis • Presentation graphics • Comparative analysis • Ideal basis for specialized AMR visualization tool replacement [Hydrogen flame, LBNL CCSE] [Radial velocity in supernova simulation, LBNL CCSE/Stony Brook University]

5. VisIt Already Handles AMR Data • Supported as “first-class” data type • Handled via “ghost-cells”: Coarse cells that are refined are marked “ghost” in the lower level • Work on rectilinear grids and skip ghost cells or “remove” results produced in ghost cells later on

6. Transitioning ANAG/APDEC SciDAC to VisIt • Customer project manager serves as “interface” between APDEC and VACET • Interact constantly with group members to assess changing needs • Implement missing capabilities in VisIt (e.g., visual spreadsheets) • Ease transition to new tool (e.g., custom macros) • Help in using new capabilities not available in ChomboVis (e.g., expressions, animations) • Implement additional new capabilities required (e.g., mapped grids, crack-free isosurfaces) www.vacet.org

7. Visual Spreadsheets • Significance: • Essential debugging tool for code development teams • Support inspecting values in individual cells • Available in ChomboVis (ANAG) and AMRVis (CCSE) • Development: • Implemented ChomboVis version of visual spreadsheets • Linked it to VisIt’s “pick” capability • Mark picked points in 2D/3D visualization view (existing VisIt capability) and spreadsheet to indicate correlation

8. Visual Spreadsheets

9. Important, e.g., in climate simulations Implementation simplified by VisIt capabilities conn_cmfe Macros (added for APDEC) No existing file format -> Workaround with two files, since Chombo files either node or cell centered VisIt macros tying it all together (e.g., AddPlot callback) Mapped Grids

10. Mapped Grids – Additional Images

11. First Success: VisIt Replaces ChomboVis! www.vacet.org

12. VisIt At CCSE • Some AMRVis capabilites more difficult to implement in VisIt (such as their “flavor” of Spreadsheets) • Due to limited resources: VisIt not a complete AMRVis substitute, yet • BUT: • Facilitate VisIt adoption via postdocs in group and collaborators • VisIt established as important tool in group, e.g., many presentation/poster/publication images were generated with VisIt www.vacet.org

13. Lots of “grunt” work “behind the scenes” • Macros to ease transition of Applied Numerical Algorithms Group to VisIt (one click for common operations) • Show only root level, show only explicit materials • Invoke VisIt from gdb • Performance enhancements, specifically for rendering patch boundaries • Ability to select data using logical indices • Basic support for embedded boundaries • Deployment of VisIt development version to ANAG machines • Mouse wheel support • Automatic expressions in Chombo files • Bind clip and slice operator via plane tool • Countless bug fixes • … much more basic infrastructure to support these functions

14. Ongoing Research: AMR-aware Streamlines Stay in Level 0 “Descend” into AMR hierarchy

15. Recent Research: Crack-free Isosurfaces • Multiple requests from science stakeholders for crack-free isosurfaces • Needed for analysis • Started integration in VisIt based on prior work • Research focus: • Efficient parallel implementation www.vacet.org

16. Science Impact • Astrophysics/cosmology • ETH Zürich (Chombo/CHARM) • Stony Brook (BoxLib/Maestro) • Reactive Flow/Combustion • CCSE (BoxLib) • Particle acclerators • Berkeley Lab Laser Accelerator (Chombo) • Porous flow/carbon sequestration • Energy Frontier Research Center for Nanoscale Control of Geologic CO2 (Chombo) • CCSE (BoxLib) • Climate • BISICLES (Chombo) • Fusion Science • Edge Simulation Laboratory (Chombo) • Princeton Plasma Physics Laboratory (Chombo) • Aerospace Engineering • NASA (Chombo) • Land Water Simulations • River, Estuary and Land Model (Chombo) www.vacet.org

17. Future Work (1/2) • Will continue to adapt VisIt to ANAG/APDEC’s and CCSE needs • Further user interface changes • Important: Users need to make efficient use of their time • Substantial time investment necessary to get things right • Functionality improvements • Improved ghost data support / generation • Native Precision • Maintenance of software infrastructure

18. Future Work (2/2) • Invest in critical, missing visualization algorithms • Improved streamline generation • Crack-free isosurfaces • Improved embedded boundary support, … • Current list of desired functions likely to fully utilize available VACET resources • Anticipate further function requests as VisIt becomes the main tool for APDEC and its collaborators • CCSE requirements likely to add significant additional burden