Maturation of Data Assimilation Over the Last Two Decades

1. Maturation of Data Assimilation Over the Last Two Decades John C. Derber Environmental Modeling Center NCEP/NWS/NOAA

2. Roger Daley (1996) ECMWF seminar on data assimilation • “Fifteen years ago, data assimilation was a minor and often neglected sub-discipline of numerical weather prediction. The situation is very different today. Data assimilation in now felt to be important for all climate/environmental monitoring and estimating the ocean state. There has been great advances in both modelling and instrumentation for a variety of atmospheric phenomena and variables, and data assimilation provides the bridge between them.”

3. Optimal Interpolation (1980s) • With the advent of optimal interpolation, analysis schemes transitioned from empirical techniques to theoretically based techniques. • With these techniques, one could begin to use information on: • Observations and observation errors • Short term forecasts and forecast errors • However, for most applications of optimal interpolation, many approximations had to be made.

4. Variational assimilation (1990s and 2000s) J = Jb + Jo + Jc J = (x-xb)TB-1(x-xb) + (H(x)-y0)T(E+F)-1(H(x)-y0) + JC J = Fit to background + Fit to observations + constraints x = Analysis xb = Background B = Background error covariance H = Forward model y0 = Observations E+F = R = Instrument error + Representativeness error JC = Constraint term

5. Variational assimilation • Inclusion of observation operator (H), transforming the analysis variable into the form of the observation operator, which in turn allowed; • Inclusion of radiances and other indirect observations • Definition of analysis variables different than the model variables or observations • Inclusion of forecast model in interpolation operator (4DVAR) • Use of all observations at once, eliminating many approximations/complex codes which were prone to failure • Allowed inclusion of additional constraint terms

6. Variational Assimilation • Background error covariances • Background error variance now approximately ½ radiosonde error variance (ECMWF) • Fully non-separable covariance matrices • Inclusion of constraints within background error • Ongoing research in situation dependent background errors

7. Isotropic Error Correlation in ValleyPlotted Over Utah Topographyobs influence extends into mountains indiscriminately

8. Anisotropic Error Correlation in ValleyPlotted Over Utah Topographyobs influence restricted to areas of similar elevation

9. Anisotropic Error Correlation on Slope Plotted Over Utah Topographyobs influence restricted to areas of similar elevation

10. Anisotropic Error Correlation on Mt Top Plotted Over Utah Topographyobs influence restricted to areas of similar elevation

11. Observations • No large scale data voids. • Number of observations used in assimilation increasing rapidly (but not as rapidly as number of observations). • Both operational and non-operational satellite data being used operationally. • Increased use of data assimilation systems in calibration/validation activities for satellites. • Expected data impact from data producers always overoptimistic.

12. Observations • Adequate fast forward models for observations still major problem, e.g., • Precipitation (satellite/radar) • Clouds (IR and microwave) • Lightning • Biases in forward models/observations greatly impact the usefulness of data

14. Variational AssimilationSurprises • Computational cost for 3DVAR similar to OI (even with all observations used together). • Much of need for Nonlinear Normal Mode Initialization came from data selection. • Direct use of radiances produces significant impacts in both hemispheres. • In Southern Hemisphere, significantly stronger circulations were produced using radiances. • Southern Hemisphere forecast skill has become similar to Northern Hemisphere skill. Since NH much better observed, is SH easier? • Microwave instruments dominate impact.

16. Future? • Extension of data assimilation techniques to: • Additional analysis variables (including cloud water/ice, etc.) • Smaller scales • Tropical disturbances • Land/ice/ocean surfaces • Inclusion of improved bias correction for background and all types of observations • Inclusion of observation specific observational/representativeness errors

17. Future? • Use of situation dependent background errors • Trying to catch up with volume of data from new observing platforms • Improved models and physics within analysis systems • Ensembles? • New systems judged on performance. With data assimilation you must do everything right!