Analyzing input and structural uncertainty of a hydrological model with stochastic, time-dependent parameters

1. Analyzing input and structural uncertainty of a hydrological model with stochastic, time-dependent parameters Peter Reichert Eawag Dübendorf and ETH Zürich,Switzerland

2. Contents • Motivation • Approach • Implementation • Application • Discussion Motivation Approach Implementation Application Discussion

3. Motivation Motivation Approach Implementation Application Discussion Motivation

4. Motivation Motivation Approach Implementation Application Discussion • Environmental modelling is often based on deterministic models that describe substance and organism mass balances in environmental compartments. • Statistical inference with such models is often based on the assumption that the data is independently and identically distributed around the predictions of the deterministic model at „true“ parameter values. • The concept underlying this approach is that the deterministic model describes the „true“ system behaviour and the probability distributions centered at the model predictions the measurement process.

5. Motivation • Empirical evidence often demonstrates the invalidity of these statistical assumptions: • Residuals are often heteroscedastic and autocorrelated. • The residual error is usually (much) larger than the measurement error. • This leads to incorrect results of statistical inference. In particular, parameter and model output uncertainty are usually underestimated. • These obviously wrong results lead to abandoning of the statistical approach and to the development of conceptually poorer techniques in applied sciences. • We are interested in a statistically satisfying approach to this problem. Motivation Approach Implementation Application Discussion

6. Motivation Suggested solution (Kennedy and O‘Hagan, 2001, and many earlier, more case-specific approaches): Extend the model by a discrepancy or bias term. Replace:by:where yM = deterministic model, x = model inputs, q = model parameters, Ey = observation error, B = bias or model discrepancy, YM = random variable representing model results. Motivation Approach Implementation Application Discussion The bias term is usually formulated as a non-parametric statistical description of the model deficits (typically as a Gaussian stochastic process).

7. Motivation Advantage of this approach: Statistical description of model discrepancy improves uncertainty analysis. Disadvantage: Lack of understanding of the cause of the discrepancy makes it still difficult to extrapolate. Motivation Approach Implementation Application Discussion We are interested in a technique that supports identification of the causes and reduction of these discrepancies.

8. Motivation There are three generic causes of failure of the description of nature with a deterministic model plus measurement error: Motivation Approach Implementation Application Discussion • Errors in deterministic model structure. • Errors in model input. • Inadequateness of a deterministic description of systems that contain intrinsic non-deterministic behaviour due to • influence factors not considered in the model, • model simplifications (e.g. aggregation, adaptation, etc.), • chaotic behaviour not represented by the model.

9. Motivation Because of these deficits we cannot expect a deterministic model to describe nature appropriately. Motivation Approach Implementation Application Discussion Pathway for improving models: • Reduce errors in deterministic model structure to improve average behaviour. • Add adequate stochasticity to the model structure to account for random influences. This requires the combination of statistical analyses with scientific judgment. This talk is about support of this process by statistical techniques.

10. Approach Motivation Approach Implementation Application Discussion Approach

11. Approach Questions: • How to make a deterministic, continuous-time model stochastic? • How to distinguish between deterministic and stochastic model deficits? Motivation Approach Implementation Application Discussion • Replacement of differential equations (representing conservation laws) by stochastic differential equations can violate conservation laws and does not address the cause of stochasticity directly. • It seems to be conceptually more satisfying to replace model parameters (such as rate coefficients, etc.) by sto-chastic processes, as stochastic external influence factors usually affect rates and fluxes rather than states directly. The model consists then of an extended set of stochastic differential equations of which some have zero noise.

12. Approach Motivation Approach Implementation Application Discussion

13. Approach Note that the basic idea of this approach is very old. The original formulation was, however, limited to linear or weakly nonlinear, discrete-time systems with slowly varying driving forces (e.g. Beck 1987). The bias term approach is a special case of our approach that consists of an additive output parameter. Motivation Approach Implementation Application Discussion Our suggestion is to • extend this original approach to continuous-time and nonlinear models; • allow for rapidly varying external forces; • embed the procedure into an extended concept of statistical „bias-modelling“ techniques.

14. Implementation Motivation Approach Implementation Application Discussion Implementation

15. Model Deterministc model: Motivation Approach Implementation Application Discussion Consideration of observation error:

16. Model Model with parameter i time-dependent: Motivation Approach Implementation Application Discussion

17. Time Dependent Parameter The time dependent parameter is modelled by a mean-reverting Ornstein Uhlenbeck process: Motivation Approach Implementation Application Discussion This has the advantage that we can use the analytical solution: or, after reparameterization:

18. Inference We combine the estimation of • constant model parameters, , with • state estimation of the time-dependent parameter(s), , and with • the estimation of (some of the) (constant) parameters of the Ornstein-Uhlenbeck process of the time dependent parameter(s), . Motivation Approach Implementation Application Discussion

19. Inference Gibbs sampling for the three different types of parameters. Conditional distributions: Motivation Approach Implementation Application Discussion simulation model (expensive) Ornstein-Uhlenbeck process (cheap) Ornstein-Uhlenbeck process (cheap) simulation model (expensive) Tomassini et al. 2007

20. Inference Metropolis-Hastings sampling for each type of parameter: Motivation Approach Implementation Application Discussion Multivariate normal jump distributions for the parameters qM and qP. This requires one simulation to be performed per suggested new value of qM. The discretized Ornstein-Uhlenbeck parameter, , is split into subintervals for which OU-process realizations conditional on initial and end points are sampled. This requires the number of subintervals simulations per complete new time series of . Tomassini et al. 2007

21. Application Motivation Approach Implementation Application Discussion Application

22. Hydrological Model Simple Hydrological Watershed Model (1): Motivation Approach Implementation Application Discussion Kuczera et al. 2006

23. 3 A 4 6 5 1 B 2 C 7 8 Hydrological Model Simple Hydrological Watershed Model (2): Motivation Approach Implementation Application Discussion 8 model parameters 3 initial conditions 1 standard dev. of obs. err. 3 „modification parameters“ Kuczera et al. 2006

24. Hydrological Model Simple Hydrological Watershed Model (3): Motivation Approach Implementation Application Discussion

25. Model Application • Data set of Abercrombie watershed, New South Wales, Australia (2770 km2), kindly provided by George Kuczera (Kuczera et al. 2006). • Box-Cox transformation applied to model and data to decrease heteroscedasticity of residuals. • Step function input to account for input data in the form of daily sums of precipitation and potential evapotranspiration. • Daily averaged output to account for output data in the form of daily averaged discharge. Motivation Approach Implementation Application Discussion

26. Analysis with Constant Parameters • Estimation of 11 model parameters: 8 rate parameters 3 initial conditions 1 measurement standard deviation • Priors: Independent lognormal distributions for all parameters with the exception of the measurement standard deviation (1/s). • Modification factors (frain, fpet, fQ) kept equal to unity. Motivation Approach Implementation Application Discussion

27. Results for Constant Parameters Motivation Approach Implementation Application Discussion

28. Results for Constant Parameters Motivation Approach Implementation Application Discussion

29. Results for Constant Parameters Motivation Approach Implementation Application Discussion

30. Results for Constant Parameters Motivation Approach Implementation Application Discussion The results show the typical deficiencies of deterministic models: • Residuals are heteroscedastic and autocorrelated. • The standard deviation of the residuals is larger than the measurement error (increasing from 0.24 m3/s at a discharge of zero to 30 m3/s at 100 m3/s). • Model predictions are overconfident. In addition: ground water level trend seems unrealistic.

31. DeficiencyAnalysis Motivation Approach Implementation Application Discussion

32. Deficiency Analysis / Step 1 Step 1: Estimation of time-dependent parameters • Estimation of 11 time-dependent parameters: 8 rate parameters 3 modification factors (frain, fpet, fQ) • Ornstein-Uhlenbeck process applied to the log of each parameter sequentially. Hyperparameters: t =1d, s =0.2 (22%) fixed, only estimation of initial value and mean (0 for log frain, fpet, fQ). • Constant parameters as before. Motivation Approach Implementation Application Discussion

33. Deficiency Analysis / Step 2 Step 2: Analyzing Degree of Bias Reduction • As quality of fit is insufficient (residual standard deviation larger than measurement error), quality of fit is a primary indicator of bias (when being careful with regard to overfitting). • Reduction of autocorrelation can be checked as a secondary criterion (it is likely to be accompanied by reduction of residual standard deviation). Motivation Approach Implementation Application Discussion

34. Deficiency Analysis / Step 2 Improvement of fit: Nash-Sutcliffe indices: frain0.90 ks0.84 fQ0.67 sF0.63 fpet0.60 kr0.57 ket 0.54 qlat,max 0.54 kdp 0.53 kgw,max 0.52 kbf 0.52 base 0.51 Motivation Approach Implementation Application Discussion Assessment: • Input (frain) and output (fQ) modifications. • Potential for soil / runoff model (ks, SF) improvements. • Some potential for river and evaporation improvements. Random or deterministic?

35. Deficiency Analysis / Step 3 Step 3: Identification of Potential Dependences • Despite doing an exploratory analysis of the values of time dependent parameters on all model states and inputs, no significant dependences could be found. • This is an indication that it may be difficult to improve the deterministic model, or that the improvement will be restricted to a small number of data points. Motivation Approach Implementation Application Discussion

36. Deficiency Analysis / Step 4 Step 4: Improvement of Deterministic Model : Motivation Approach Implementation Application Discussion Extension 1: Modification of runoff flux: Extension 2: Modification of sat. area funct.: Extentsion 1 has two, extension 2 three additional model parameters.

37. Deficiency Analysis / Step 4 Model Extensions: Motivation Approach Implementation Application Discussion

38. Deficiency Analysis / Step 4 Previous results: Nash-Sutcliffe indices: frain0.90 ks0.84 fQ0.67 sF0.63 fpet0.60 kr0.57 ket 0.54 qlat,max 0.54 kdp 0.53 kgw,max 0.52 kbf 0.52 base 0.51 Motivation Approach Implementation Application Discussion Extended models: Nash-Sutcliffe indices: ext. 1 0.73 ext. 2 0.51

39. Deficiency Analysis / Step 4 Original Model: Motivation Approach Implementation Application Discussion Modified Model:

40. Deficiency Analysis / Step 4 Original Model: Motivation Approach Implementation Application Discussion Modified Model:

41. Deficiency Analysis / Step 4 Original Model: Motivation Approach Implementation Application Discussion Modified Model:

42. Deficiency Analysis / Step 4 Conclusions of Step 4 • The significant increase in the Nash-Sutcliffe index is caused by the elimination of a small number of outliers. • All other deficiencies remain. • This is the reason why the improvement could not have been detected in the exploratory analysis. • It seems questionable that the remaining deficiencies could be significantly reduced by improvements of the deterministic model. Motivation Approach Implementation Application Discussion

43. Deficiency Analysis / Step 5 Step 5: Addition of Stochasticity to the Model Major sources of indeterminism: • Spatial aggregation: Aggregation of distributed reservoirs in a much smaller number of reservoirs in the model leads to the same model results for different „states of nature“ (that lead to different results in nature). • Rainfall uncertainty:Spatial heterogeneity of rainfall intensity is not well captured by point rainfall measurements. Motivation Approach Implementation Application Discussion

44. Deficiency Analysis / Step 5 It seems reasonable to summarize these sources of indeterminism by a stochastic rain modification factor frain. To quantify input uncertainty (combined with aggregation error) we need an informative prior for the measurement error. We choose sQ,trans ~ N(0.5,0.05).0.5 corresponds to a standard deviation in original units increasing from 0.1 m3/s at a discharge of zero to 12.6 m3/s at a discharge of 100 m3/s. The standard deviation of the Ornstein-Uhlenbeck process for log frain is now estimated from the data. Motivation Approach Implementation Application Discussion

45. Deficiency Analysis / Step 5 Time-dependent parameter frain: Motivation Approach Implementation Application Discussion

46. Deficiency Analysis / Step 5 Motivation Approach Implementation Application Discussion

47. Deficiency Analysis / Step 5 Original Model: Motivation Approach Implementation Application Discussion Modified Model with Time-Dependent Parameter frain:

48. Deficiency Analysis / Step 5 Original Model: Motivation Approach Implementation Application Discussion Modified Model with Time-Dependent Parameter frain:

49. Deficiency Analysis / Step 5 Original Model: Motivation Approach Implementation Application Discussion Modified Model with Time-Dependent Parameter frain:

50. Discussion Motivation Approach Implementation Application Discussion Discussion