1 / 24

The Developmental Testbed Center: Historical Perspective and Benefits to NOAA Steve Koch

The Developmental Testbed Center: Historical Perspective and Benefits to NOAA Steve Koch DTC Deputy Director Director, NOAA/ESRL/Global Systems Division. Fundamental Purpose of DTC.

rusty
Télécharger la présentation

The Developmental Testbed Center: Historical Perspective and Benefits to NOAA Steve Koch

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Developmental Testbed Center: Historical Perspective and Benefits to NOAA Steve Koch DTC Deputy Director Director, NOAA/ESRL/Global Systems Division

  2. Fundamental Purpose of DTC • To serve as a bridge between research and operations to facilitate the activities of both halves of the NWP Community in pursuit of their own objectives: • Research community gets a functionally equivalent operational environment to test and evaluate new NWP methods in retrospective extended period tests using advanced tools • Operational community benefits from DTC testing and evaluation of strengths and weaknesses of new NWP advances prior to consideration for operational implementation DTC Operational Codes

  3. Benefits to NOAA NOAA will use the DTC as the primary gateway through which promising well-tested NWP science and technology originating in the research community will be selected for further development and evaluation by NCEP/EMC EMC resources will be used much more efficiently since DTC will filter out less promising technologies and greatly reduce the time needed for the better ones to become ready for operational implementation DTC activity links with Weather and Water Goal’s STI program, with future support to be provided primarily through the Environmental Modeling Program

  4. Agency priorities for DTC NOAA: Top priority of DTC is “to manage the WRF community code and support the WRF modeling system to the research community… the DTC is the critical link to reaping the benefits of the WRF Program.” April 28, 2006 WRF ExOB AFWA: Top priority of DTC is “to manage the WRF baseline. DTC is the key component/enabler for establishing, maintaining, and supporting the ‘Official’ (ARW- and NMM-WRF cores) baseline of the community’s end-to-end WRF ‘Reference-grade’ code.” June 5, 2006 SOW

  5. Operational Community Provide guidance for next generation Operational Configurations Research Community Provide baselines for evaluating impacts of new techniques Aid in selecting configurations for research projects Testing and Evaluation Goals

  6. WRF Test Plan: Getting to the NCEP IOC • DTC demonstrated the capabilities of the candidate dynamical cores to qualify them for a 6-member WRF IOC ensemble system to run daily in High Resolution Window (HRW) domains: • NMM + ARW dynamical WRF cores with NCEP + NCAR physics • NMM and ARW cores with switched physics packages • Model variants using bred Initial / Boundary Conditions • Ensemble improves accuracy over a single deterministic forecast and provides for a measure of uncertainty.

  7. Impact of the WRF Test Plan on NCEP • DTC testing and evaluation demonstrated to NCEP that the 6 WRF members tested were qualified to run as an ensemble system • 21 Sept 2004: IOCimplemented with only 2 members (the two dynamical cores - ARW and NMM - without physics swapping) • 6 Dec 2005: Six WRF members added to NCEP Short-Range Ensemble Forecast (SREF) included physics swapping between the cores and 2 members from NCEP breeding system • New WRF-based SREF outperformed the current one in all ensemble aspects including mean, spread and probability distributions

  8. DTC Winter Forecast Experiment (DWFE) 15 January - 31 March 2005

  9. DTC Winter Forecast Experiment Goal: Provide real-time model guidance for winter weather over a large domain using 2 WRF configurations with explicit convection Objectives NWS: Expose forecasters to WRF at very high (5-km) resolution prior to the first scheduled operational implementation at NCEP. NCEP: Evaluate WRF in HRW at 8-10 km resolution relative to WRF run at 4-5 km over CONUS with explicit microphysics only. NCAR: Determine whether encouraging results from warm season 4-km WRF runs carry over to winter events out to 48h. • Real-time displays: • NCAR: DTC website • GSD: FX-Net • GSD: 2D AWIPS fields-NMM only • Objective & subjective evaluations by DTC and NWS, respectively

  10. Provided valuable hands-on experience to forecasters with high-resolution WRF model output over CONUS NCEP added simulated reflectivity to diagnostic fields available to WFOs Results helped NCEP decide to upgrade resolution of HRW domains (NMM 8 to 5.1 km / ARW 10 to 5.8 km) DTC demonstrated importance of statistical tests of significance when comparing forecast skill from multiple models NMM 5 km real-time runs were continued into Spring at request of forecast offices (NMM5-CONUS) DWFE Impact on NWS

  11. ARW 3-h forecast Composite Reflectivity Lake-effect snowbands Lake-effect snowbands Narrow Cold-Frontal Rainband Narrow Cold-Frontal Rainband 0600 UTC Radar Mosaic Observed Composite Reflectivity at 03Z

  12. Stationary Mesoscale Wave-Precip Systems Verifying 5-km Surface Mesoanalysis for 0300 UTC WRF-ARW 00Z 22Jan05 F27 995 988

  13. DWFE Verification Activities RTVS verification of 6h precipitation forecasts over the DWFE domain WRF-NMM WRF-ARW Eta-12

  14. WRF-Rapid Refresh “Core Testing” • Key questions still remain: • What are the implications of choosing a single dynamic core for WRF? • Do the potential cost savings overcome the loss of model diversity for ensemble predictions? • Previous DTC experiments provided some idea of how the two WRF cores compare in terms of forecast skill, but no tests have been totally “clean” so far – this requires each core to be run with the same physics package, initial and boundary conditions, horizontal and vertical resolution, & run over the same domain • This was done with the WRF-Rapid Refresh “core test” • WRF-RR is the first operational WRF model at NCEP for which the DTC can perform such tests and evaluations prior to implementation • Never before has such a clean dynamic core test been performed • 2 tests completed: using NMM physics in both cores and RUC-like physics • 13-km WRF run to 24h even though RUC only extends to 12h

  15. WRF Core Test: Rapid Refresh (RR) Goals Conduct controlled experiments carefully comparing the 2 WRF dynamic cores.  Provide datasets to support GSD’s dynamic core recommendation to NCEP for WRF-RR Impacts 1st “clean” WRF dynamic core comparison ARW selected for initial implementation of WRF-RR

  16. WRF Core Test Rapid Refresh Setup

  17. 2-sigma = 95% CI Red = Physics 1 = NMM set Blue = Physics 2 = RUC-like set

  18. 2-sigma = 95% CI Red = Physics 1 = NMM set Blue = Physics 2 = RUC-like set

  19. Objective Assess whether the medium range (60h) skill of the two dynamic cores is similar enough to allow the direct transfer of research results obtained with one core to the other. Approach Use Rapid Refresh Core Test as basic template for test design Use current code for all components of end-to-end system Relevance to community! Maximize support from developers WRF Core Test - Extended

  20. 60-h bias-corrected RMSE (NMM in red; ARW in blue; ARW-NMM in green) • RH errors increase with height to 500 hPa • Core differences are not statistically significant • Wind errors are maximum at 250 hPa • NMM has best forecast at 850 and 700 hPa • Temperature errors show max at 850 and 200 hPa • Core differences are not statistically significant

  21. 2-m temperature bias 100 hPa temperature bias Evolution in Time • Most variables and levels, such as surface temperature, do not show an increase in ARW – NMM differences in time out to 60h. • However, core differences do grow in time for a few variables and levels, such as temperature at 100 hPa.

  22. Extended Core Test • ARW and NMM results are not substantially different • Inter-core differences do not grow in time • Potential for transferring research done with one core to the other

  23. IBM vs Linux Comparison

  24. WRF Test Plan NMM ported to WRF Framework (increased number of options available to community) DWFE Problem related to initialization of small lakes during winter season brought to attention of developers WRF-RR Core Test Significant increase in physics options available for NMM Extended Core Test More consistency between pre-processing options available for 2 dynamic cores Pre-release and system testing Bug fixes for both dynamic cores Improved interoperability (more microphysics & PBL schemes available for NMM) Impact of DTC T & E WRF Community Code

More Related