1 / 14

The LANL POP Team B. Malone, R. Smith, M. Maltrud, P. Jones, J. Davis, J. Baumgardner,

The Parallel Ocean Program (POP) and Coupled Climate Modeling with Particular Emphasis on Performance Tools and Whatever Else Comes To Mind. The LANL POP Team B. Malone, R. Smith, M. Maltrud, P. Jones, J. Davis, J. Baumgardner, J. Dukowicz (ret). POP History. Bryan-Cox z-level model

aizza
Télécharger la présentation

The LANL POP Team B. Malone, R. Smith, M. Maltrud, P. Jones, J. Davis, J. Baumgardner,

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Parallel Ocean Program (POP) and Coupled Climate Modeling with Particular Emphasis on Performance Tools and Whatever Else Comes To Mind The LANL POP Team B. Malone, R. Smith, M. Maltrud, P. Jones, J. Davis, J. Baumgardner, J. Dukowicz (ret)

  2. POP History • Bryan-Cox z-level model • Advanced under DOE CCPP • Designed for parallel computers • Data-parallel, MPI, OpenMP/MPI • Introduced features like free surface, pressure-averaging, dipole grid,etc. • Added state-of-the-art parameterizations like GM, KPP • Publicly available (version 1.4.3) http://climate.acl.lanl.gov/models/pop

  3. POP Simulations • High Resolution Simulations • Global 0.28, 0.1 • North Atlantic 0.4, 0.2, 0.1, 1/15

  4. North Atlantic Series 0.1 0.2 0.4

  5. POP Simulations • High Resolution Simulations • Global 0.28, 0.1 • North Atlantic 0.4, 0.2, 0.1, 1/15 • Coupled Simulations • NCAR CCSM • POP/CICE • CSU Geodesic

  6. CCSM Coupled Model Land LSM NCAR/others Atmosphere CCM NCAR 7 States 10 Fluxes 6 States 6 Fluxes Once hour per per Flux Coupler NCAR/DOE hour Once 6 States 6 Fluxes 7 States 9 Fluxes 4 States 3 Fluxes 6 States 13 Fluxes day per Once per Once hour 6 Fluxes 11 States 10 Fluxes Ocean POP LANL Ice CICE LANL

  7. Performance Issues • Memory Bandwidth • Operators (5pt, 9pt) • Array Syntax • No “kernel” • Message Passing • Latency with Implicit (PCG) Solver • Load Imbalance

  8. POP OpenMP/MPI Hybrid • Sub-block decomposition • Domain decomposed into blocks sized for cache (or vector) • Land blocks eliminated • Remaining blocks distributed in load-balanced manner using a rake algorithm • Priorities can be set to maintain some locality during rake • Many blocks on each node provide OpenMP parallelism • Block loops at high level to amortize OpenMP overhead • Different block distribution used for barotropic solver to optimize for communication rather than load balance

  9. All Evidence Points to…

  10. …Phil’s Feeble Brain • Run time vs. compile time • Load imbalance • Really, really, really, really, really, stupid implementation of global reductions

  11. Actual Performance • 0.1 Global

  12. Actual Performance

  13. CCSM Coupled Model Land LSM NCAR/others Atmosphere CCM NCAR 7 States 10 Fluxes 6 States 6 Fluxes Once hour per per Flux Coupler NCAR/DOE hour Once 6 States 6 Fluxes 7 States 9 Fluxes 4 States 3 Fluxes 6 States 13 Fluxes day per Once per Once hour 6 Fluxes 11 States 10 Fluxes Ocean POP LANL Ice CICE LANL

  14. Performance Tools • Load Balance • Visual Maps (XPVM, VAMPIR, etc) • Timing • Per processor/per thread • Routine/Line • Memory Bandwidth • Cache Misses, Loads/Stores • Automated

More Related