DTC Mission

1. Bridging the Valley of Death:Defining a process for transitioning promising new mesoscale innovations from research to operations Jamie K. Wolff1, Louisa B. Nance1, Brad S. Ferrier2, Clifford F. Mass3, Barbara G. Brown1 and Ying-Hwa Kuo1 Joint 46th CMOS/25th WAF/21st NWP AMS Conference May 29, 2012 1National Center for Atmospheric Research/Research Applications Laboratory and Developmental Testbed Center 2National Centers for Environmental Prediction/Environmental Modeling Center 3University of Washington

2. DTC Mission • The fundamental purpose of the DTC is to facilitate the interaction & transition of NWP technology between research & operations. DTC facilitates: • O2R transition by making the operational NWP systems available to the research community & providing community user support • R2O transition by performing testing & evaluation of new NWP innovations in a functionally similar operational environment over an extended period • Interaction between research & operational NWP communities through the organization of community workshops/meetings on important topics of interest to the NWP community & hosting a DTC Visitor Program DTC strives to be an effective and efficient community facility for the transition of innovations in NWP between research and operations.

3. Testing Protocol Motivation • Wide range of NWP science innovations under development in the research community • Testing protocol imperative to advance new innovations through the research to operations (R2O) process efficiently and effectively. • Three stage process: • Proving ground for research community • Comprehensive T&E performed by the DTC • Pre-Implementation testing at Operational Centers

4. Testing Protocol – Stage IProving ground for research community • Code development; Initial stage of testing • Mesoscale Model Evaluation Testbed (MMET) • Communicate results to the DTC; Nominate for Stage II testing • Contribution of new technique into WRF repository encouraged • Work with WRF Developers Committee (contact: wrfhelp@ucar.edu) • Apply for DTC Visitor Program support (see: http://www.dtcenter.org/visitors) = –

5. Mesoscale Model Evaluation Testbed (MMET) • Why: Assist the research community in efficiently demonstrating the merits of a new development • Provide a common framework for testing; allow for direct comparisons • What: Mechanism to assist research community with initial stage of testing • Provide model input and observational datasets to utilize for testing • Establish and publicize baseline results for select operational models • Where: Hosted by the DTC; served through Repository for Archiving, Managing and Accessing Diverse DAta (RAMADDA) http://dtcenter.org/repository

6. RAMADDA View (http://dtcenter.org/repository)

7. MMET Cases • Initial solicitation of cases from DTC Science Advisory Board Members and Physics Workshop Participants – great response and enthusiasm towards endeavor • Target casesduring initial year • 20090228 – Mid-Atlantic snow storm where North American Mesoscale (NAM) model produced high QPF shifted too far north • 20090311 – High dew pointpredictions by NAM over the upper Midwest and in areas of snow • 20091007–High-Resolution Window (HIRESW) runs underperformed compared to coarser NAM model • 20091217 – “Snowapocalypse ‘09”: NAM produced high QPF over mid-Atlantic, lack of cessation of precipitation associated with decreasing cloud top over eastern North Carolina • 20100428-0504– Historic Tennessee flooding associated with an atmospheric river event • 20110518-26 – Extended period of severe weather outbreak covering much of the mid-west and into the eastern states later in the period • 20111128 – Cutoff low over SW US; NAM had difficulties throughout the winter of breaking down cutoff lows and progressing them eastward • 20120203-05 – Snow storm over Colorado, Nebraska, etc.; NAM predicted too little precipitation in the warm sector and too much snow north of front (persistent bias)

8. “Snowpocalypse 2009” • Nor’easter that became a major snowstorm for the East Coast of the U.S with record-breaking snowfall totals for several major cities, including D.C., Baltimore and Philadelphia. • MMET case study for 20091217 12 UTC out to 84 hours

9. May 21-26, 2011 Tornado Outbreak • Deadly outbreak across the Midwest and Southern regions of the U.S., including the Joplin tornado (May 22) and two “High Risk” days (May 24-25) • Extended time period included in MMET to 20110518-26 in order to capture additional weaker warm-season convection prior to outbreak

10. Testing Protocol – Stage IIComprehensive T&E performed by the DTC • Maintain a neutral position in order to provide a trusted, unbiased assessment • Conduct comprehensive testing for a broad range of weather regimes • Run end-to-end system composed of community codes • Functionally similar to operational environment • Evaluate based on extensive objective verification statistics • Traditional scores • New, relevant verification techniques (e.g., spatial methods) • Statistical significance assessment

11. Testing Protocol – Stage IIIPre-Implementation testing at Operational Centers • Ultimate decision to proceed with pre-implementation testing is made by the Operational Centers and is based on a variety of factors, including: • Forecast performance • Computational requirements • Testing specifics depend on the target production configuration, but may include: • Complex data assimilation testing • Initial condition diversity testing for ensemble members

12. Conclusion • Establish data sets for nine initial MMET cases • Publicize more information on R2O testing protocol and MMET at: http://dtcenter.org/eval/mmet • Contact information for DTC MMET TeamJamie Wolff jwolff@ucar.eduMichelle Harrold harrold@ucar.eduTara Jensen jensen@ucar.eduTricia Slovacekslovacek@ucar.edu Acknowledgements: The DTC is funded by the National Oceanic and Atmospheric Administration, the Air Force Weather Agency, and the National Center for Atmospheric Science (NCAR). NCAR is sponsored by the National Science Foundation.