A Generalized Approach to Microwave Satellite Data Assimilation Quality Control and Preprocessing

1. A Generalized Approach to Microwave Satellite Data Assimilation Quality Control and Preprocessing Kevin Garrett1 andSid Boukabara2 11th JCSDA Science Workshop on Satellite Data Assimilation June 6, 2013 College Park, MD 1: RTI @ NOAA/NESDIS/STAR, JCSDA 2: NOAA/NESDIS/STAR, JCSDA

2. Outline • Goal: To have a flexible, consistent algorithm applicable to a variety of sensors for use as a preprocessor to NWP data assimilation systems which: • provides quality control information about radiance observations • provides dynamic information about scenes (precip, surface conditions) • has consistent error characteristics across all sensors (using retrieved parameters) • is flexible and easily extended to new/future sensors • is mindful of computing resources/overhead/latency • has meaningful, positive impact on analyses and forecasts • Introduction to the MIIDAPS • 1DVAR retrieval process • Potential applications to NWP (GSI) • Current progress • Next steps

3. MIIDAPS OverviewMulti-Instrument Inversion and Data Assimilation Preprocessing System Megha-Tropiques SAPHIR/MADRAS S-NPP ATMS • Inversion Process • Consistent algorithm across all sensors • Uses CRTM for forward and jacobian operators • Use forecast, fast regression or climatology as first guess/background • Assimilation/Retrieval • All parameters retrieved simultaneously • Valid globally over all surface types • Valid in all weather conditions • Retrieved parameters depend on information content from sensor frequencies MetOp-A AMSU/MHS MetOp-B AMSU/MHS MIIDAPS NOAA-18 AMSU/MHS NOAA-19 AMSU/MHS DMSP F16 SSMI/S DMSP F17 SSMI/S DMSP F18 SSMI/S TRMM TMI GPM GMI GCOM-W1 AMSR2 MIIDAPS 1DVAR is based on the Microwave Integrated Retrieval System (MiRS)

4. MIIDAPS Overview Post Processing Example of MIIDAPS retrieval using S-NPP ATMS with vertical cross sections at 170° longitude. Core state variables (products) from MIIDAPS Rain & Graupel Cloud Temperature Water Vapor 170° Using All Channels 100 Layers Latitude 90° TPW Rain Rate 0° -90° Over All Surfaces 170° Emissivity Skin Temperature

5. 1DVAR Retrieval/Assimilation Process 3. X is updated through the Levenberg-Marquardt equation: 2. Retrieval done in reduced (EOF) space Reduce the dimensionality of the covariance matrix from 400x400 to 22x22 (or less depending on sensor) Transform [K] and [B] to EOF space for minimization Solution [X] Reached Observed TBs (processed) Convergence Compare Bias Correction Covariance Matrix [B] No Convergence Compute DX 1. Solution is found by minimizing the cost function: Convergence is determined by non-constrained cost function: Simulated TBs Obs Error [E] K Retrieval mode Climatology CRTM Update State Vector [X] Initial State Vector [X] Assimilation mode Forecast Iterative Processes

6. 1DVAR Retrieval/Assimilation Process State Vector Parameters per Attempt • MIIDAPS allows a maximum of 2 retrieval attempts per observation • 1st attempt assumes no scattering signal in the TBs • 2nd attempt assumes scattering from rain/ice is present in TBs • Maximum of 7 iterations per attempt • Tunable parameters: nattempts, niterations, channels used (optimize efficiency without degrading outputs) Chi-square with out scattering Chi-square with scattering

7. 1DVAR Retrieval/Assimilation Process Emissivity inclusion in the state vector is vital for retrieval/assimilation Over All Surfaces In All Weather Surface preclassifier determines which background and covariances to initialize for retrieval (left) Emissivity sensitivity to rainfall rate for AMSU-A frequencies Seamless transition along surface boundaries Retrieved Rainfall Rate error as a function of retrieved emissivity error Retrieved 23 GHz Emissivity Retrieved TPW

8. Applications to NWP Primary objective for a 1DVAR preprocessor on microwave observations Chi-square based QC Chi-square based QC Emissivity constraint/assimilation Emissivity constraint/assimilation • Focus for Global NWP using GSI • Use chi-sq for QC/filtering • Use CLW/RWP/GWP for detecting cloudy/rainy obs • For filtering or assimilation • Non-precip cloud/precipitating cloud • Use surface emissivity as boundary condition for forward simulations to increase surface channel observations Cloudy/rainy radiance detection Cloudy/rainy radiance detection

9. Applications to NWP Emissivity vital for assimilation of surface sensitive channels in all weather Average emissivity spectra before/after a 3-day rain event in May 2008. 5/4-5/8 shows ~8% change in 23 GHz emissivity. 5/4 NEXRAD NEXRAD NEXRAD 5/5-5/7 5/8 5/10 Retrieved TPW

10. Applications to NWP Primary objective for a 1DVAR preprocessor on microwave observations Temperature 400 mb Temperature 400 mb Rainfall Rate Rainfall Rate TPW TPW • Focus for Regional NWP using GSI +HWRF • Assimilation of sounding data near tropical storm cores • Assimilation of TPW • Assimilation of rainfall rate retrievals

11. Applications to NWP Implementation of the 1DVAR preprocessor BUFR Files ‘read_sensor’ routines “setuprad” (clw, O-B filtering) • Implementation of 1DVAR preprocessing at the Bufferization stage: • Process all radiance observations during time window • CPU time spent outside of assimilation (minimize effect on latency) • Encode 1DVAR output in BUFR as metadata or in unique BUFR file • Increased control for radiance thinning/selection during GSI read process • Maintain ability to use 1DVAR geophysical outputs on optimized set • Implementation of 1DVAR preprocessing BUFR read stage: • Process all radiance observations during time window • Increased control for radiance thinning/selection during GSI read process • Maintain ability to use 1DVAR geophysical outputs on optimized set • Separate 1DVAR interface for each satellite sensor • Read routines must be parallelized • CPU time added to the analysis (how much can be afforded?) • Implementation of 1DVAR in “setuprad” stage: • Process only on thinned set of observations • Maintain ability to use 1DVAR geophysical outputs on optimized set • CPU time used in analysis (how much can be afforded?) • Code is universal for all satellite datasets (single interface to 1DVAR)

12. Current Status • Testing currently underway with implementation in read_atms routine • 1DVAR called for additional QC (based on chisq) • No optimized thinning implemented (every 5 FOVs/Scanlines) • Prelimenary implementation in setuprad routine • Still testing the interface Current operational With additional 1DVAR filter

13. Future Work • Continue with implementation both in setuprad and in the read routines for optimized thinning • Test impact of cloud filters, use of emissivity in number of obs, O-B, O-A, etc. • Extend to other sensors, starting with infrared • Apply to regional HWRF (product assimilation) • Involve other interested JCSDA partners (NCEP, OAR, GMAO, Navy, AFWA, NCAR)