Critical issues of ensemble data assimilation in application to carbon cycle studies

1. Critical issues of ensemble data assimilation in application to carbon cycle studies Dusanka Zupanski And Scott Denning Colorado State UniversityFort Collins, CO 80523-1375 CMDL Workshop on Modeling and Data Analysis of Atmospheric CO2 Observations in North America 29-30 September 2004 ftp://ftp.cira.colostate.edu/Zupanski/presentations ftp://ftp.cira.colostate.edu/Zupanski/manuscripts

2. OUTLINE: • Introduction: EnsDA approaches • Non-linear processes • Model error and parameter estimation • Uncertainty estimates • Correlated observations • Non-Gaussian PDFs • Conclusions and future work Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

3. Probabilistic approach to data assimilation and forecasting orEnsemble Data Assimilation (EnsDA) Provides the following: (1) Optimal solution or state estimate (e. g., optimal CO2 analysis) (2) Optimal estimates of model error and empiricalparameters (3) Uncertainty of the analysis(a component of the analysis error covariance Pa) (4) Uncertainty of the estimated model error and parameters(components of the analysis error covariance Pa) (5) Estimate of forecast uncertainty(the forecast error covariance Pf ) Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

4. DATA ASSIMILATION (ESTIMATION THEORY) Discrete stochastic-dynamic model wk-1 – model error (stochastic forcing) M – non-linear dynamic (NWP) model G – model (matrix) reflecting the state dependence of model error Discrete stochastic observation model ek – measurement + representativeness error H – non-linear observation operator (M  D ) Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

5. DATA ASSIMILATION EQUATIONS: (1) State estimate (optimal solution): MAXIMUM LIKELIHOOD ESTIMATE (VARIATIONAL APPROACH ): MINIMUM VARIANCE ESTIMATE (KALMAN FILTER APPROACH ): (2) Estimate of the uncertainty of the solution: KALMAN FILTER APPROACH ENSEMBLE KALMAN FILTER or EnsDA APPROACH In EnsDA solution is defined in ensemble subspace (reduced rank problem) !

6. PDF(x) PDF(x) x x xmode xmean xmode = xmean Ensemble Data Assimilation (EnsDA) (1) Maximum likelihood approach (involves an iterative minimization of a functional) =>xmode (MLEF, Zupanski 2004) (2) Minimum variance approach (calculates ensemble mean) =>xmean Non-Gaussian Gaussian Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

7. Critical issues: Non-linear processes - Use only non-linear models (tangent-linear, adjoint models are not needed) - Iterative minimization is beneficial for non-linear processes Example: KdVB model (M. Zupanski, 2004) Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

8. Critical issues: Model error and parameter estimation - Estimate and correct all major sources of uncertainty: initial conditions, model error, boundary conditions, empirical parameters - Unified algorithm: EnsDA+state augmentation approach (Zupanski and Zupanski, 2004) Example: KdVB model Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

9. EnsDA experiments with KdVB model (PARAMETER estimation impact) 10 obs 101 obs

10. EnsDA experiments with KdVB model (BIAS estimation impact) Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

11. Critical issues: Uncertainty estimates - Analysis error covariance Pa (analysis uncertainty) - Forecast error covariance Pf (forecast uncertainty) - Both defined in ensemble sub-space KdVB model example: Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

12. Critical issues: Correlated observations Problem: Numerous observations (~108 -109) are being projected onto a small ensemble sub-space (~101 -103) ! Loss of observed information! Remedies: • Process observations one by one (Anderson 2001, Bishop et al. 2001; Hamill et al. 2001). Or • Process observations successively over relatively small local areas (LEKF, Ott et al. 2004). Assumption in both approaches: Observations being processed separately are uncorrelated (independent)! This may not be justified for dense satellite observations. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

13. Critical issues: Correlated observations How does the observed information impact the uncertainty estimate of the optimal solution (analysis error covariance Pa ) ? - square root of analysis error covariance (Nstate x Nens) - square root of forecast error covariance (Nstate x Nens) - impact of observations on the optimal solution (Nens x Nens) The eigenvalue spectrum of (I+A)-1/2 may help understand the impact of observations, and perhaps find a better solution for correlated observations. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

14. RAMS model example If eigenvalues of (I+A)-1/2 spread over the entire interval [0,1], ensemble size (Nens) is appropriate for a given observation number (Nobs) . A safe approach to prevent loss of observed information, assuming independent observations: Nobs Nens. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

15. CSU shallow-water model on geodesic grid (Results from M. Zupanski et al.) Smooth start (in cycle 1) can improve the performance of EnsDA When system can learn from its past, less information from observations is needed ! Analysis error smaller than obs error Milija Zupanski, CIRA/CSU ZupanskiM@CIRA.colostate.edu

16. Non-Gaussian PDFs • Non-linear Atmospheric- Hydrology- Carbon state variables and observations are likely to have non-Gaussian PDFs. • MLEF, as a maximum likelihood estimate, is a suitable tool for examining the impact of different PDFs. • Develop a non-Gaussian PDF framework (M. Zupanski) • - allow for non-Gaussian observation errors • - apply the Bayes theorem for multiple events Milija Zupanski, CIRA/CSU ZupanskiM@CIRA.colostate.edu

17. In case we solved all critical issues, one problem remains:How to define observation error covariance matrix R, if it is not known? NASA’s GEOS column model example R1/2 = e R1/2 = 2e Prescribed observation errors directly impact innovation statistics. Since the observation error covariance R is the only input required by the system, it could be tuned! • CSU EnsDA algorithm is currently being examined in application to • NASA’s GEOS column model in collaboration with: • A. Hou and S. Zhang (NASA/GMAO) • C. Kummerow (CSU/Atmos. Sci.) Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

18. CONCLUSIONS • EnsDA approaches are very promising since they can provide not only optimal estimate of the state, but also the uncertainty of the optimal estimate. • The experience gained so far indicates that the EnsDA approach is suitable for addressing critical issues of data assimilation in Carbon cycle studies. • Model error and parameter estimation are necessary ingredients of a data assimilation algorithm. • Problems involved in Carbon data assimilation require a state-of-the art approach. We anticipate findings from different scientific disciplines (e. g., atmospheric science, ecology, hydrology) to be of mutual benefits. • It is especially important to gain experience with complex coupled models (e. g., RAMS-SiB-CASA), correlated (satellite) observations, and non-Gaussian PDFs in the future. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu

19. Dusanka Zupanski, CIRA/CSU Zupanski@CIRA.colostate.edu