Developmental Testbed Center : Overview, Status and Future Direction

1. Developmental Testbed Center: Overview, Status and Future Direction Bill Kuo

2. Outline • Brief history of DTC • DTC management structure • Highlights of recent DTC accomplishments • On-going activities for FY2010 • Proposed DTC AOP for FY2011 • Future direction and challenges

3. Fundamental Purpose of DTC • To serve as a bridge between research and operations to facilitate the activities of both halves of the NWP Community • Research: functionally equivalent operational environment to test and evaluate new NWP methods over extended retrospective periods • Operational: benefits from DTC T & E of strengths and weaknesses of new NWP advances prior to consideration for operational implementation NCAR/RAL & NOAA/ESRL

4. DTC Management Structure* 8-Member WRF Executive Oversight Board (existing) AFWA NCAR NCEP OAR ARL FAA NRL OON ARL DTC Executive Committee AFWA NCAR NWS OAR (EC provides agency oversight and policy guidance, and sets high-level priorities.) DTC Director Management Board (Director + Deputy. Dirs.) External Community Science Advisory Board *Blue indicates structure established by proposed DTC Charter

5. Management Structure • Day-by-day management of DTC: Bill Kuo (Director), Steve Koch (ESRL/GSD Director), Barb Brown (NCAR/RAL/JNT Director), Louisa Nance (DTC Assistant Director) • Priorities set by funding agencies with guidance from Science Advisory Board

6. Brief history of DTC • Established in 2003, initially funded by USWRP • Two major functions of DTC: • Providing support for the community codes: • WRF, HWRF, GSI, and MET • Performing testing and evaluation of community codes for research and operation: • WRF test plan, DTC winter forecast experiment, WRF Rapid Refresh core test, Extended core test, High-Resolution Hurricane (HRH) test, and Reference Configuration testing. • DTC Charter was signed in Sep. 2009, formally establishing the governing structure of DTC • For FY2010, DTC has a budget of ~$5M, and ~25 FTE (involving about 45 staff in NCAR/RAL and ESRL/GSD)

7. Past DTC Funding FundingSources: NOAA, NCAR, AFWA, FAA (FY06,07), NSF NOAA+ HFIP AFWA+ 7

8. DTC Activity Time Line

9. DTC Organization & Tasks WRF: WRF modeling system WRF for Hurricanes: HWRF, AHW, HFIP GSI: Grid-point Statistical Interpolation data assimilation system MET: Model Evaluation Tools HMT: Hydrometeorology Testbed collaboration HWT: Hazardous Weather Testbed collaboration DET: DTC Ensemble Testbed NEMS: NOAA Environmental Modeling System • Two major functions of DTC: • Provide support for community systems • Conduct testing and evaluation of community systems for research and operations

10. DTC Tasking and FY10 Funding Task areas Funding sources DTC Director’s office: $623K Visitor program: $222K WRF (Wolff): $698K HWRF (Bernardet): $1,197K GSI (Huang): $622K MET (Fowler): $395K HMT (Tollerud): $323K HWT (Jensen): $148K DET (Toth): $651K NEMS (Carson): $173K NOAA Core: $2,900K HFIP: $328K USWRP: $300K AFWA: $825K NCAR: $300K GSD: $300K NSF: $100K Total budget: $5,053K Collaborations with other testbeds

11. DTC Funding and Constraints • NOAA OAR line item ($2.9M): General guidance from “Program Change Summary (PCS)”. Considerable flexibility. AOP is a “proposal” that substitutes the PCS. • AFWA ($825K): Specific Statement of Work and deliverables. Limited flexibility. (P.M.: Steve Rugg) • USWRF ($300K): Specific Statement of Work and deliverables. Limited flexibility. (P.M.: Marty Ralph) • HFIP ($328K): Specific Statement of Work and deliverables. Limited flexibility. (P.M.: Fred Toepfer, Bob Gall) • NCAR, GSD ($300K each): Flexible • NSF ($100K): dedicated to Visitor Program

12. Community Code Goals • A free and shared resource with distributed development and centralized support • DTC currently supports the following software packages to the community: • WRF: NWP model + pre- and post-processors * * • WRF for Hurricanes (coupled atmosphere and ocean system) * * • Gridpoint Statistical Interpolation (GSI) Data Assimilation System* • Model Evaluation Tools (MET) – Verification package • Provide direct assistance through an email helpdesk for each software package * * • Facilitate community contributions to the respective code repositories • Support community outreach events (tutorials/workshops) * * -in collaboration with MMM* and EMC*

13. GSI R2O Transition Procedure (draft) 1 2 Code development candidate DTC branch Community research Code development candidate DTC branch 3 5 4 DTC trunk EMC trunk EMC branch GSI Review Committee Initial Scientific Review DTC-developer code merging and testing GSI Review Committee code and commitment review DTC->EMC GSI code commitment EMC->DTC GSI repository syncing

14. GSI Transition: O2R and R2O • Past (two years ago?) • Distributed development • Different applications and operational requirement (GFS, RTMA, NAM, RR, AFWA…) • Manual version control • No system documentation • limited system support (for GSI partners and community) • Non-portable system • Current • GSI Review Committee (DTC, NCEP/EMC, NASA/GMAO, NOAA/ESRL, NCAR/MMM-AFWA) • Multiple platform GSI system supported by DTC • SVN version control (Duel repositories with synced trunk and localized development branches) • Completed GSI User’s Guide and website • Annual community release and residential tutorial • Community user support by DTC (gsi_help@ucar.edu) • R2O infrastructure and application

15. HWRF Accomplishments – Past & Future • Community Code • Ongoing support for users of WRF for Hurricanes • Community WRF code baseline for 2011 implementation • HWRF configuration in the general WRF code repository • Tests of community version to show skill equivalent to operational HWRF used in 2008 and 2009 • Testing • HWRF testing infrastructure on njet(on-going) • Run full end-to-end HWRF system(on-going) • Collaborate with EMC & HFIP HWRF developers on testing & evaluation of new capabilities for NCEP operations (on-going)

16. H050 (V2 Baseline) HWR1 (V3.2 Baseline) H56C (2010 HWRF) HWRE (2010 V3.2+) Comparison to qualify WRF community code to be used as a baseline for 2011 From Vijay (EMC)

17. New Thrust in FY2010: DTC Ensemble Testbed (DET) • Ensembles are used increasingly as a basis for forecasts in aviation weather, convective outbreaks, and major snowstorms. • Goals for DTC Ensemble Testbed: • Establish and support community ensemble modeling infrastructure to facilitate research in ensemble prediction • Conduct testing and evaluation of key components of ensemble systems to support research and operation • Collaborate with research community to test and evaluate proto-type ensemble forecasting systems • Collaborate with EMC on the development of next generation SREF, and use it as a benchmark • Serve as a critical element of the National Mesoscale Probability Prediction (the “White Paper”) effort

18. Anticipated DTC Ensemble Testbed (DET) Accomplishments Module 1: Configuration External Sources (HMT, HFIP, HWT, etc...) • Six-module infrastructure designed for DET • Initial development of ensemble configuration & initial perturbation modules • Preliminary DET Testing & Evaluation Plan • DET Ensemble Workshop (18-19 August 2010) - review DET technical design • WRF Ensemble Working Group re-activated to provide technical advice on the DET Module 2: Initial Perturbations Module 3: Model Perturbations Module 4: Statistical Post-Processing Module 5: Product Generation Module 6: Verification

19. New Thrust in FY2010:NEMS model software framework • NOAA has adopted ESMF and developed NOAA Environmental Modeling System (NEMS) framework • All EMC modeling systems are migrating toward NEMS software framework, including NMM-B, SREF, GSI, and HWRF. • DTC must become familiar with the NEMS software framework in order to be an effective conduit between research and operation. • DTC recently hired a DTC software engineer at EMC to work with EMC NEMS team and participate in the transition of WRF modeling system (including ARW and NMM) to the NEMS software framework. • DTC is organizing a model software framework workshop in October 2010 to promote collaboration and communications on model framework

20. Plan for FY2011 • Anticipate level budget with FY2010 (with contributions from partners at roughly the same as those of FY2010) • Proceed with the development of DTC Ensemble Testbed • Participate in the NEMS development • Continue to support WRF, HWRF, MET and GSI to the communities • Allocate DTC resources to the various tasks at similar levels of FY2010 (no major change expected) • No plan to take on new initiatives (unless additional resources become available)

21. Future Direction and Challenges • Model software framework: • WRF community remains on WRF CMI (Common Model Infrastructure) • NCEP moving its modeling systems to NEMS • How do we maintain effective path between O2R and R2O? • Data assimilation: • GSI community repository has an excellent start • Community is migrating toward 4D-Var and hybrid • Need to work with EMC and the community toward a hybrid system that can effectively support research and operations • Need to develop strong linkage between data assimilation and ensemble

22. Future Direction and Challenges: • Hurricanes: • HWRF’s performance has been less than desirable over the past few seasons • HFIP is moving toward multiple model ensemble (e.g., HWRF, AHW, COAMPS, GFDL) • What is DTC’s role in hurricane prediction and HFIP? • Working with EMC to improve the performance of HWRF • Application of DET to support hurricane prediction • What hurricane prediction systems should DTC support to the community?

23. Future Direction and Challenges: • DTC Ensemble Testbed: • The community strongly endorses this effort. However, the community also would like to see this effort accelerated. • Testing of various components of ensemble will require considerable computing resources. • Need to develop stronger linkage with HWT, HMT, and HFIP • Need to provide community support for DET ensemble infrastructure • Performance Measure: • How do you measure the success of DTC? • Some of the impacts are difficult to measure quantitatively

24. Summary • DTC is a community facility dedicated to supporting the transition between research and operations in NWP. • DTC has come a long way since its inception. We now have a well established facility and management structure. • Advances in computing technology and modeling science arecausing rapid changes in NWPin various areas, for example: ensemble, advanced data assimilation, and cloud-resolving global model. • By working closely with research and operational communities, DTC has the potential to make significant contributions to the U.S. atmospheric science community, allowing U.S. to regain its pre-eminent position in NWP.

25. Questions to SAB: • Is DTC taking on too much and spreading too thin? • Is the on-going effort appropriate for the DTC? • Is the resources allocation reasonable? • Is DTC on the right track? • What are your recommendations for DTC’s future direction?

26. THANK YOU!

27. DTC Director’s Office & Visitor Program Louisa Nance

28. Director’s Office Responsibilities • Internal Coordination • Manage and coordinate DTC tasks • Planning, budgeting, execution and reporting • External Communication • Conduct or assist with workshops and tutorials • Interact with DTC partners on collaborative efforts • Maintain the DTC Website • Provide administrative support for DTC EC, MB & SAB meetings • Host the DTC Visitor Program

29. Director’s Office • Management • Bill Kuo – Director (0.5 FTE) • Louisa Nance – Assistant Director (0.5 FTE) • Barb Brown – JNT Director (0.10 FTE) • JNT DTC Manager (0.25 FTE) • Steve Koch – GSD Director (0.10 FTE) • GSD DTC Manager (0.25 FTE) • Administrative Support • Pam Johnson (0.50 FTE) • Travel: $30 K • External committee meeting support: $36 K • Materials & Supplies: $10 K • Computers: $25 K Total FTEs: 2.20 FTEs

30. Major accomplishments - past 6 months • Internal Coordination • DTC staff retreat • Monthly staff / task lead meetings • External Communication • Facilitated AOP 2011 discussions with operational centers • Completed implementation of DTC Management Structure • MB teleconference meeting (May 2010) • 1st face-to-face meeting of DTC Executive Committee (26 August 2010) • Established the DTC Science Advisory Board

31. Next 6 months… • Planned activities - • Coordinate planning activities for AOP 2011 • Coordinate strategic planning activities • Provide support for upcoming workshops and tutorials • Continue making improvements to DTC website • Implement tools to assist with task and staff coordination • Major accomplishments • AOP 2010 report • AOP 2011 document and budget • Draft Strategic Plan for next 5 years

32. DTC Visitor Program – AOP 2010 • Major accomplishments – past 6 months • Proposal to NSF for $100 K funded (double past contributions) • Total budget for visitor projects: $200 K • Distributed Announcement of Opportunity to community • Focus areas based on input from MB • Revamped program to increase support for PIs and added graduate student option • Planned activities – next 6 months • Select projects to be funded based on SAB recommendations and MB approval

33. DTC Visitor Program • Received 22 proposals in response to AO • Type of support • 14 PI • 7 graduate student • 1 PI or graduate student • Topics • Ensembles (1) • Verification (8) • Data assimilation (5) – including data impact studies • Model testing (6) • New model features (2) • Estimate available funds to support 6-7 projects

34. Challenges for DTC Visitor Program How can the DTC make its visitor program more effective? • Current funding level limits the types of projects that can be funded • DTC EC recently recommended expanding the visitor program to include longer-term targeted R2O projects or a grants program • How does the DTC attract the research community to work on projects relevant to operations? • Should educating and training the next generation of scientists be part of this program’s goals? Bottom line: The DTC is looking for input from SAB/MB on defining/refining the goals and scope of the DTC Visitor Program

35. WRF Task:Community Support & Test and Evaluation Jamie Wolff Collaborators: NCEP/EMC NCEP/EMC NCEP/EMC ESRL/GSD ESRL/GSD ESRL/GSD NESL/MMM

36. WRF Task Goals • Community Support • Maintain, support and facilitate contributions to the community-released code repositories, currently including: • WRF * * • Post Processing Software * • Support community outreach events * * -in collaboration with MMM* and EMC* • Testing and Evaluation • Extensively test and evaluate a variety of model configurations that will provide benefits to both the operational and research communities • Maintain verification results of primary configurations for subsequent (major) model versions

37. Community SupportMajor accomplishments - past 6 months • WRF v3.2 release (April) * * • Community outreach and support* * • WRF Workshop: 21 – 25 June 2010 • WRF Tutorial: 26 July – 6 Aug 2010 • wrfhelpemail support • Buy-in to develop a Code Management Plan for a community post processor • AFWA, DTC, EMC, GSD -in collaboration with MMM* and EMC*

38. Testing and EvaluationMajor accomplishments - past 6 months • Published/updated online documentation for past & current WRF T&E activities on the DTC T&E webpage • http://www.dtcenter.org/eval • Established several DTC RCs • http://www.dtcenter.org/config • Broadly distributed information to the user community (through wrf-news and WRF Workshop) • Established baseline for determining impacts of future developments

39. Community Support - next 6 months • Planned activities • Continue on-going community support * * • Enhance WRF Regression testing * • Continue transition from WRF Post Processor (WPP) to Unified Post Processor (UPP) * • Anticipated major accomplishments • UPP Code Management Plan ** * • Community-UPP code repository * • WRF Tutorial * * -in collaboration with MMM*,EMC*, GSD*and AFWA*

40. Testing and Evaluation - next 6 months • Planned activities • Perform evaluation of NAM/GFS Precipitation Forecast Comparison • Retest select RCs with WRF v3.2.1 • Begin manuscript on DTC T&E methodology • Anticipated major accomplishments • Results of the NAM/GFS T&E made available to entire community • Posted on the DTC T&E webpage • Presented at relevant conferences • Updates on retested-RCs posted on DTC RC webpage

41. Community Support - AOP 2011 • Planned Activities • Maintain / Enhance WRF regression testing * • Continue on-going efforts in community support * * • Continue to prepare community-UPP software package for distribution * • Extensively test • Write documentation • Provide community support upon release • Define a plan for how to facilitate interactions between research and operations into the future given change in environment • Anticipated major accomplishments • WRF release (v3.3 April 2011) * * • Community-UPP version 1.0 release (April 2011)* • Tutorials (Jul 2011 and Jan 2012) * * • WRF Users Workshop (Jun 2011) * -in collaboration with MMM* and EMC*

42. Testing and Evaluation - AOP 2011 • Planned activities • Continue expansion of RC testing and evaluation • Retest select designated DTC RCs and establish new, as appropriate • Encourage submission of CCRCs • Gather sponsor & community input on direction of future testing • Host a physics workshop for mesoscale modeling (Spring 2011) • Work on publications for DTC methodology and results from select test activities • Anticipated major accomplishments • New/Updated RC results posted on the DTC RC webpage and presented at relevant conferences • Coordinated efforts on physics improvements promoted within the community • Manuscript(s) published

43. Resource requirements for AOP 2011

44. Challenges • How does DTC remain relevant to both the operational and research communities as NEMS is established? • What will be the path of WRF? • Continue to offer joint tutorials? • Role of DTC in WRF repository maintenance? • Should DTC support (software distribution, documentation, tutorials, help desk) NMM-B to the community? via WRF or NEMS? • With limited funding, how does the DTC conduct enough configuration testing to make an impact on the community as a whole? • How many configurations should DTC test in a given year? • What will make up the “Performance measure” to show improvement? • How can the DTC promote more rapid advancement in physics? • Can DTC help facilitate a coordinated effort among the community to make progress on physics improvements? • Should developers be encouraged to work on a suite rather than individual scheme? • Should the DTC become primarily focused on physics testing? • For example, test a physics suite under WRF then retest under NEMS if it shows promise?

45. WRF for Hurricanes Ligia Bernardet External collaborators: NOAA Environmental Modeling Center NOAA Geophysical Fluid Dynamics Laboratory NOAA Atlantic Oceanographic and Meteorological Laboratory NCAR Mesoscale and Microscale Meteorology Division University of Rhode Island

46. WRF for Hurricanes Goals • Facilitate transfer of research to operations by creating a framework for NCEP and the research community to collaborate • Support the community in using operational hurricane models • Develop and maintain a hurricane testing and evaluation infrastructure at DTC • Perform tests to assure integrity of community code and evaluate new developments for potential operational implementation

47. Major accomplishments - past 6 months • Release of a beta-version of HWRF code to the community: code management, documentation, support to 122 registered users. • Inclusion of HWRF features in community WRF code, creating a community WRF code with 2010 operational capabilities. Successful tests indicating that community code will be adopted as a baseline for 2011 operational HWRF. • Creation of an infrastructure to run HWRF on jet - being used for extended regression tests to control quality of community code.

48. HWRF code management: atmospheric model 04/2011 WRF V3.3 HWRF for operations (2011) Potential Contributions from Community & NOAA Research to HWRF 02/2011 WRF V3.2.1+ 1st HWRF release 09/2010 WRF V3.2.1+ Upgrades 2011 08/2010 WRF V3.2.1 Tutorial, HWRF Beta release 03/2010 WRF V3.2 07/2009 2010 baseline WRF V3.1.1 (…) 05/2004 WRF V2 oper HWRF 2009 oper HWRF 2010 (…)

49. Planned activities - next 6 months • Expand DTC HWRF testing suite by adding capability to read native GFS data. • Qualify DTC HWRF testing suite by repeating a comprehensive test run on NCEP machine. Results will be used to create a Reference Configurationfor the community code. • Expand DTC HWRF testing suite by adding HYCOM. • Run test of community code using HYCOM as HWRF ocean component (instead of POM). • Incorporate new developments in WRF code. • Prepare for April 2011 Tutorial.

50. Anticipated major accomplishments - next 6 months • Completion of planned HWRF tests (including HWRF-HYCOM) and dissemination of results. • Upgraded HWRF community code (with operational 2011 features, 3rd nest, revised tracker etc.) • Preparation of HRH manuscript for publication.