1 / 30

Outline

Towards Rapid GeoScience Model Development Bill Appelbe Victorian Partnership for Advanced Computing Director, Software Development - ACcESS bill@vpac.org May 5th , 2002. Outline. Scientific Software – Why is the software not keeping pace with the models?

dmccoll
Télécharger la présentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards Rapid GeoScience Model Development Bill AppelbeVictorian Partnership for Advanced ComputingDirector, Software Development - ACcESSbill@vpac.orgMay 5th , 2002

  2. Outline • Scientific Software – • Why is the software not keeping pace with the models? • How can we change the way we build scientific software? • Goals for scientific software development • The Snark Project • Philosophy, toolkits, off-the-shelf stuff • Snark-itechture • Snark Finite Element Particle in Cell (PIC) models • Snark Particle Models (SPH) • Model Integration • Other Collaborations • Timelines, deliverables, availability

  3. Scientific Software – Requirements for New Models • Increasing demands on scientific software • Must be 3D • 2D just not realistic • 3D requires massive increase in computational requirements • 10,000x10,000 2D grid (108) is 1012 in 3D! • Therefore must be scaleable-parallel • Parallel programming is hard! • Only MPI-parallel programs scale to hundreds of processors • But MPI programs must be crafted by hand

  4. Scientific Software – Requirements for Geoscience • Increasing demands to be able to rapidly • Quickly test new models • Incorporate different rheological models • Requires changing the solver, not just the data • Incorporate new observational data • Models must be linked and interoperate

  5. Scientific Software – What's Wrong With This Picture • Scientific Software • Largely been built "from scratch" in Fortran and C • Each team or project builds their own specialized programs (codes) • To meet their own specific models • Codes start small, but soon grow by "feature-creep" • Evolve to "hero-codes" – maintenance and extension require heroic efforts, eventually all thrown away • "Cut and paste" programming • Little or no reuse or sharing of modules • Limited to simple, low-level libraries (e.g. BLAS) • Little or no change in productivity in decades

  6. What Can We Learn from Commercial Software? • Commercial software development • An order-of-magnitude increase in productivity since 1990, by • Using Generators, Frameworks, extensible libraries • Component-based technology • For example, JBuilder • Why has scientific software development not kept pace?

  7. What Inhibits Productivityin Scientific Software? • Focus on publications and grants • No incentive to build reusable software • No publication or grant credit for doing it • Lack of expertise in modern software engineering among computational scientists • Until recently, no effective large-scale reusable components

  8. What Are We Doing About It in Australia? • Form a major national earth sciences consortium – ACcESS • All the major computational earth science research groups involved • ANU, UQ, Monash U, U of Melbourne, CSIRO, VPAC, ... • Get long-term Federal funding • Mostly for software • A fair amount of compute grunt

  9. Goals for Geoscience Software • Reduce custom code by an order of magnitude • Thus reduce the time to build and test new models from months to days • Scaleable parallel programs • Create a framework for combined particle and grid codes • Model interoperability

  10. Goals for Geoscience Software - Can it be done? YES! • At VPAC, we have prototypes demonstrating all the above • First versions should be released by mid-year

  11. The Snark Project - Goals • Create a framework for geoscience software • Building on top of "state of the art" tools, components and libraries • Rebuild and re-engineer existing geoscience models into this framework • Integrate particle and mesh based models

  12. Snark – Philosophy, toolkits, off-the-shelf stuff • "Productivity is measured by the number of lines of code NOT written" • Careful survey and testing of existing scientific software tools & components • See vpac.org/snark • The core component technology that Snark uses is PETSc • A powerful extensible library of solvers and support software from Argonne

  13. Using PETSc • Strong support group and documentation • Large library of linear and non-linear solvers, grid support functions, etc. • Scaleable parallelism built-in! • Continuously being extended • We are adding particle libraries • The programmer can focus on "setting up the problem", not the algorithm details • The only drawback is the "learning curve" • See ... http://www-fp.mcs.anl.gov/petsc/

  14. Snark – Other tools • Off the shelf tools are available for many ancillary functions • Graphics, viewing data sets • Vis5D, VU, .. • Grid refinement/triangulation • LaGRIT, Triangle • Domain decomposition • Parmetis • All these tools just hook in to PETSc!

  15. Automatic Mesh Generation – Triangle

  16. Snark –Top-Level Architecture • Almost all grid and particle models fit into one top-level architecture! • There is an isomorphism between irregular grids and particle ensembles Grid Point  Particle • Grid tools that can • partition and refine irregular grids • generate grid communication, etc work equally well for particle ensembles

  17. Snark –Top-Level Architecture Initialize boundary conditions, grids/particle ensembles Partition across processors Derived from constitutive laws Update matrix Update particle properties PETSc Library Solve matrix Solve particle properties Timestepper & IO

  18. The Snark Engine –Detailed Architecture MainRoutine Appl. Init. LaGriT/GMV ParMETIS FEM & Shape Fun. Ax = b setup & loading PIC Prometheus or FEAP PETSc Unstr. Mesh VPAC code Vis5D or VU Other code Visualization Post-Processing Evaluations TAO

  19. Snark – Particle in Cell Models • Based on the Ellipsis PIC code • A "state of the art" code for creeping viscoelastic fluids • Uses a novel Particle in Cell model • Coupled Eulerian multigrid solver to Lagrangian particles • Used to model problems in earth sciences, e.g. mantle convection and crustal deformation • Limitations of Ellipsis • A classic "hero-code" • 30,000 lines of code, several variants • Not parallel, very difficult to adapt and maintain for new constitutive laws • 2D and 3D versions, but only for regular rectangular grids • 3D version has taken person-years to adapt

  20. Ellipsis • Very efficient for rheological simulations on large timescales

  21. Snark – Particle in Cell Models • The Snark version of ellipsis • 3,000 lines of code • An order of magnitude reduction • Does both viscoelastic and thermally driven convection • Fully parallel and scaleable • A quick Snark video

  22. Snark – Particle in Cell Models (cont) • Current limitations of the Snark version of Ellipsis • Particle tracking implemented • But not integration/update of matrix from particle states • Coming soon! • Only 2D • But 3D version for regular rectangular grids fairly trivial • 3D and 2D supported by PETSc • Estimate delivery June/July of prototype • 3D irregular grid with arbitrary topology by end of 2002 • Programmer must hand code matrix setup/update for constitutive laws and boundary conditions • Medium-term solution will evolve to setup/update generators from operator specifications

  23. Snark –The Mesh/Particle Continuum • There is a clear continuum between mesh and particle based codes • PETSc’s support of particle codes is in its infancy • To move towards unified particle/mesh codes, we are putting a “pure particle” code into Snark • SPH, Smoothed Particle Hydrodynamics • Used extensively in astrophysics • Ideally suited to: rock fracture, multi-phase mixed fluids • The code framework is readily extensible to incorporate other particle models such as DEM or LMSheath

  24. Snark –The Mesh/Particle Continuum • 3D parallel version of SPH built, being extended to • Allow specification of arbitrary boundary conditions • Simulation of Tsunamis and dam fracture • Handle multi-fluids • Student working on simulation of pyroclastic flows • Integrate with PETSc framework of PIC codes • Extend to DEM codes • A quick video or two

  25. Collaboration • Surface process modeling (ANU) • Reconstruction software (Monash) • LSM (UQ) • Griddles • Linking applications across the grid • With NO source language changes

  26. Couple to Other Models • The Surface models (ANU, UoM);

  27. Couple to Other Models • PlatyPlusPlus (ACRC, Monash U)

  28. Couple to Other Models • LSM (QUAKES, UQ)

  29. Timelines, Deliverables, Availability • Snark V1 will release midyear • There is a website with copious documentation –vpac.org/snark • Programmer must hand code matrix setup/update for constitutive laws and boundary conditions • Medium-term goal is to enable constitutive laws to be input as operator specifications • Key project • Be able to model the evolution of the Tibetan plateau

  30. Thank You! VPAC's HPC Facility – 128 processor Compaq AlphaServer SC

More Related