Update on Dropout Team Work and Related COPC Action Items Presented by Dr. Bradley Ballish

1. Update on Dropout Team Work and Related COPC Action Items Presented by Dr. Bradley Ballish Co-Chair JAG/ODAA and Dropout Team* Member 16 November 2010 COPC Meeting AFWA *Including EMC’s J. Alpert, D. Carlis, Y. Ling, X. Su And NCO’s K. Kumar and J. Carr

2. Outline • Recent AC scores with dropouts indicated • COPC Action Item (AI) 2008-1.5 (real-time monitoring) • COPC AI 2010-1.6 (data impact tests) • NCEP activity summary • Dropout related projects • NCEP GFS implementation summary • Summary COPC Dropout Update

3. 00Z 5-day AC Scores in September/October 2010 FNMOC dropout No GFS dropouts and GFS is number 2 in scores! ECM run Scored .73 Multiple dropouts for FNMOC and GFS in SHEM COPC Dropout Update

4. AI 2008-1.5 COPC Action Item 2008-1.5: Develop a monitoring system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real-time and make this real-time system available to OPCs as a daily tool. Status • Project management charter has been signed and work has started • Initial operating capability at NCEP planned for February 2011 • Recommend AI remain open COPC Dropout Update

5. AI 2010-1.6 • COPC Action Item 2010-1.6:The dropout team will provide a final summary report on the impact of observational data types suspected of causing model dropouts and make specific recommendations on data quality control and usage for operational implementation Status • Team has conducted thorough analysis for many dropout occurrences • More work and analysis will be needed well beyond the time frame of typical COPC action items • Impact tests with satellite winds and radiances have been run (see the next four slides) • Satellite radiances generally have good impact • Satellite winds have shown some negative impact in dropout cases with GFS • Speed dependent QC and ECMWF filtering of satellite winds showed positive impact • FNMOC super obs of satellite winds show big positive impact COPC Dropout Update

6. Impact of Conventional and Satellite Radiance Obs on 3-d and 5-d GFS Forecasts puzzle • Retaining satellite radiance data shows small positive impact on NH 3-D and 5-D forecasts • In the SH, conventional and non-radiance satellite observations (PREPB alone) show a large negative impact (8 points) in 5-D forecasts! -- another puzzle • Addition of satellite radiance data to conventional and satellite observations has a positive impact on 5-D forecasts even for dropout cases • AMSUA along with PREPB conventional observations (yellow) show the largest positive impact in the NH and SH experiments and typically are correlated with the results of the CNTRL COPC Dropout Update

7. Satellite Wind QC Experiments CONTROL=0.804 SU’S QC=0.817 CARLIS’ QC=0.815 CONTROL=0.818 SU’S QC=0.824 CARLIS’ QC=0.823 Satellite winds (SATWIND) are a data type of concern at EMC due to case studies that have shown negative impact by using SATWIND’s in the GDAS system. Su’s SATWND QC lowers the gross check asymmetrically, is planned for operations rather than Carlis’ QC, and its implementation is TBD Carlis’ Satwnd QC removes about 50% of the observations by deleting wind speeds that are less than the background guess COPC Dropout Update

8. By Riishojgaard, Jung and Velden COPC Dropout Update Plan for operational SATWIND QC upgrades TBD

9. Notice big impact from Satellite winds COPC Dropout Update FNMOC has big positive impact from satellite winds possibly from super obs

10. AI 2010-1.6 (continued) Recommend closing this AI and continue working on the data usage improvements listed below: • Implement improved satellite wind QC and filtering (TBD) • Aircraft data • Implement NRL track-checking code (Q2 2011)* and then improve • Aircraft temperature bias corrections • Improve waypoint dictionaries for AIREP data (Q1 2011)* • Test and implement aircraft moisture data • Implement surface pressure bias corrections • Use adjoint codes to analyze dropout cases and tune reject-list criterion • Improve timeliness of adding bad data platforms to reject-list (Q1 2011)* • Develop and implement use-list and improve methods of adding bad data platforms to reject-list * Indicates concrete implementation date COPC Dropout Update

11. NCEP Activity Summary • Dropouts • Data QC and data usage • Dropout analysis • GFS implementation summary • GFS QC, data assimilation and model changes COPC Dropout Update

12. Dropout Related Projects Data QC and Data Usage COPC Dropout Update

13. Dropout Related Projects Data QC and Data Usage COPC Dropout Update

14. Dropout Related Projects (cont) Dropout Analysis COPC Dropout Update

15. NCEP GFS Implementation Summary QC, Analysis and Model Changes COPC Dropout Update

16. Summary • Observational data usage continues to be explored as a possible cause of dropouts • GFS model upgrade helped in NH, but less so in SH • GFS performance likely depends on the quality of each observation, not simply a particular class of data • Continue to work on improving and testing QC rules and metadata on a case by case basis • Recommend AI 2010-1.6 to be closed • Operational implementations • FY10 • GFS model July 2010 • Surface dictionary upgrades June 2010 • FY11 • GSI upgrade (Q2 2011)* • NRL Aircraft QC (Q2 2011)* • TBD • Improved satellite wind QC and filtering implementations COPC Dropout Update

17. Background Slides COPC Dropout Update

18. Interagency Participants • NCEP/EMC • Jordan Alpert • Yangrong Ling • DaNa Carlis • NCEP/NCO • Bradley Ballish • Krishna Kumar • Joe Carr • NCEP/HPC • James Cisco • NRL • Rolf Langland • FNMOC • Chuck Skupniewicz • NESDIS • Kathy Kelly • Tom Renkevens • The NCEP dropout team meets weekly with the EMC director, John Derber and sometimes with the JCSDA director and reports quarterly to the NCEP director COPC Dropout Update

19. Latest Dropout Team Findings (continued) • For more information on dropout research, see the reports presented at the AMS meetings: http://ams.confex.com/ams/89annual/techprogram/paper_142644.htm http://ams.confex.com/ams/89annual/techprogram/paper_142649.htm http://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154268.htm http://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154282.htm COPC Dropout Update

20. Adjoint Codes • Based on the code from the NASA/Goddard/GMAO GSI, the dropout team has the GSI analysis adjoint code running • Further work is needed to develop an adjoint of the NCEP global model that with the analysis adjoint will produce estimates of how every observation helps or hurts the forecast skill • We will determine if smart statistics from these promising new tools will give us more precision on how to better use observations, such as: • Satellite radiance data impact estimates by type, channel, surface conditions (water, ice, etc.) and more • Possibly better rules on filtering of satellite winds • Improved reject-list criterion • Possibly better analysis of dropout cases COPC Dropout Update

21. AI 2008-1.5 Official Action Plan (Revised) • NCEP will generate warnings on a restricted website when the GFS global analysis has extreme localized differences from ECMWF analyses on a 1x1 degree grid • NCEP will develop a real-time warning system to alert us when the GFS global model 5-day forecast height correlations with the ECMWF exceed normal limits, which will give an early warning of a possible dropout • NCEP website will have graphics to allow staff to study the divergence in forecasts in real time and show likely analysis problem areas COPC Dropout Update