Estimation of Surface Pressure & Wind Observation Errors from ERA-20C Observation Feedback

1. Estimation of Surface Pressure & Wind Observation Errors fromERA-20C Observation Feedback Paul Poli

2. Outline • Why we care about observation errors • Handling of • Gross errors • Systematic errors • Random errors • And lessons learnt from ERA-20C • Conclusions Poli, Observation errors in ERA-20C, EMS 2013

3. Reanalysisaims to objectively reconstruct a history of the past “Observations-only” climatology “outliers” Reanalysis Benefits 1) Immediate: improve upon estimates made at the time, by bringing together & recovering measurements from various sources, benefit from state-of-the-art analysis methods 2) Longer-term: expose discrepancies= issues to be resolved, to improve the agreement between observations and models, understand uncertainties “Model only” integration Continuous, as realistic as the model and forcings Poli, Observation errors in ERA-20C, EMS 2013

4. Most global (re)analysis methods rely (more or less) on least-squares statistical linear estimation Observation matrix True state Observations Observation errors Observations are unbiased Observations error covariances are known Mapping the information from the observations to estimate the system state happens through the observation matrix and the observation error covariance matrix Poli, Observation errors in ERA-20C, EMS 2013 (e.g.,Talagrand, 1997)

5. In practice • We use sequential or variational schemes, which assume a prior estimate • Obtained by propagating / forwarding in time with a (imperfect) forecast model another prior (imperfect) estimate typically valid at an earlier time • Makes the problem solvable (whole atmospheric state >>106 variables) with limited observations • This apparent simplification further complicates things… • … because we then need to estimate errors in these first-guess states • These errors aren’t separable analytically in their sources (between observation, model, forcings) because the information propagates in time and space and across the geophysical variables • So… the theory very quickly becomes much more complicated • In any case, whatever the analysis method, the presence of S (sometimes written R) in the earlier slide always remain: Uncertainties of past (atmospheric) observations are essential when deriving quantitative estimates of past (atmospheric) states. Poli, Observation errors in ERA-20C, EMS 2013

6. Illustration with actual data: surface pressure observations on 4-5 June 1900 Poli, Observation errors in ERA-20C, EMS 2013

7. Errors, mistakes, biases… • Mistakes and errors are not the same in our jargon • Mistakes or obvious outliers (whatever this means) are nicely referred to as “gross errors” • Another, also somehow philosophical separation, is also adopted between • The systematic parts of the errors • Generally called observation biases • Although what is systematic in some restricted population may appear as random in a larger population • The random parts of the errors • The standard deviations of the observation errors being abusively called ‘observation errors’ • We will make no attempt to estimate the observation error distributions • Although we will generally assume that they are ~Gaussian Poli, Observation errors in ERA-20C, EMS 2013

8. ERA-20C summary slide • ECMWF reanalysis, covering years 1899-2009 • Assimilating ISPD 3.2.6 and ICOADS 2.5.1 observations • surface pressure (ocean, land) • near-surface winds (ocean) • Data assimilation system • Ensemble of 10 members • 24-hour 4DVAR • Self-adjusting background errors over time and space • Products 3-hourly, 91 model levels, ~125 km horiz. resolution • Forced by HadISST2.1.0.0 ensemble • Data to be copied on public data server within a few months,along with observation feedback (containing obs. input) [hPa] Poli, Observation errors in ERA-20C, EMS 2013

9. 4 Quality Control steps to (hopefully) remove all occurrences of gross errors in ERA-20C • Blacklist quality control; rejects observations: • From sources other than ISPD 3.2.6 or ICOADS 2.5.1 • Reporting variables other than surface pressure (or geopotential) and near-surface wind • Reported exactly at location exactly 0◦N 0◦E (except for buoy observations because there is a buoy from the Pirata network there) • Of wind above the land surfaces or near the coastlines (such observations could be assigned land surface roughness) • Of wind reported too far above the model surface height (such observations are then located in closed seas in mountainous regions) • Of surface pressure reported too much below the model surface are rejected • Redundancy quality control; rejects observations: • Of surface geopotentialif there is a corresponding surface pressure observation • Of pressure reported at sea-level if there is a corresponding report at station level • Background or first-guess quality control; rejects observations: • With departures from the first-guess (+3 to +27-hour forecast initialised from previous analysis) larger than a threshold: 18 * ( so2 + sb2 ) 1/2 • Since sb(background error) varies over time and space, the rejection threshold also varies • For surface pressure: regional maxima (minima) of 310 hPa (20 hPa) in 1901, 175 hPa (17 hPa) in 2008. • For wind observations, the rejection thresholds are also extremely permissive, but an added criterion removes observations reporting or in regions of wind speed greater than 35 m/s. • Overall, less than 1% of the observations are rejected by the first-guess check. • Variationalquality control (Anderssonand Järvinen, 1999), using the Huber norm (Tavolato and Isaksen, 2010); rejects observations: • That cannot be fitted within reasonable limits by the first minimisation • About 2% (14%) of the surface pressure (wind) observations are typically rejected, more in 1900, less in 2000. This rate increases as the observation time is located towards the end of the analysis window Poli, Observation errors in ERA-20C, EMS 2013

10. Impact of observation quality controls  Hardly any rejections for ‘other causes’ (first-guess check nearly always passed) Poli, Observation errors in ERA-20C, EMS 2013

11. End March 1954, Southern Hemisphere 30 March 1954, 00 UTC 31 March 1954, 00 UTC 31 March 1954, 03 UTC Southern Hemisphere 30 March 1954, 06 UTC 31 March 1954, 09 UTC 31 March 1954, 12 UTC Poli, Observation errors in ERA-20C, EMS 2013

12. Hourly data from this station indicate surface pressures ~800 hPa, at 51m above msl Date time obsvalue (Pa) O-B (Pa) O-A (Pa) (…) 19540330 10000 82040.000000 -18152.878906 -18208.525391 19540330 20000 82010.000000 -18265.691406 -18324.767578 19540330 30000 81950.000000 -18398.414062 -18456.244141 19540330 40000 81930.000000 -18483.009766 -18535.792969 19540330 50000 81880.000000 -18590.810547 -18646.716797 19540330 60000 81850.000000 -18675.292969 -18737.505859 19540330 70000 81860.000000 -18713.806641 -18790.753906 19540330 80000 81790.000000 -18828.126953 -18927.904297 19540330 90000 81760.000000 -18906.607422 -19023.998047 19540330 100000 81700.000000 -19157.095703 -19037.751953 19540330 110000 81710.000000 -19189.607422 -19114.912109 19540330 120000 81640.000000 -19249.113281 -19148.837891 19540330 130000 81610.000000 -19230.953125 -19113.726562 19540330 140000 81470.000000 -19317.041016 -19218.109375 19540330 150000 81470.000000 -19261.933594 -19192.337891 19540330 160000 81410.000000 -19259.660156 -19195.234375 19540330 170000 81340.000000 -19253.533203 -19209.138672 19540330 180000 81300.000000 -19224.179688 -19227.476562 19540330 190000 81210.000000 -19272.863281 -19283.312500 19540330 200000 81250.000000 -19214.863281 -19180.957031 19540330 210000 81240.000000 -19199.351562 -19124.273438 19540330 220000 81100.000000 -19295.736328 -19209.656250 (…) Poli, Observation errors in ERA-20C, EMS 2013

13. Some information about this island Information and photo from http://www.sthelena.se “In 1961 a dramatic volcanic eruption forced the evacuation of the entire island. They were taken to what we glibly refer to as 'civilisation'. Almost all chose to return to the island when the eruption was over.” Poli, Observation errors in ERA-20C, EMS 2013

14. Background constraint Observation constraint Estimate the 3D atmospheric fields and the bias parameters Observation operator and observation bias model simulate the observations Observation biases In ERA-20C, we use a variational bias correction method for surface pressure. The observation biases are estimated within the analysis, using prior bias estimates: For eachanalysis, construct a costfunction and findits minimum: Background error covariances Observation error covariances Poli, Observation errors in ERA-20C, EMS 2013

15. Maps of surface pressure observation bias estimates 1906 2006 Poli, Observation errors in ERA-20C, EMS 2013

16. Time-series of surface pressure observation bias estimates Reminder: the number of observations increases by a factor ~50 in the time-frame above Poli, Observation errors in ERA-20C, EMS 2013

17. Overall observation bias estimates become closer to zero over time: interpretation • Assuming these estimates are correct • Are the observations now better calibrated, with more recurrent procedures? • In particular, for the ships, are the methods to correct for the height of the barometer more accurate? • If these estimates of observation bias contain in fact model bias: • Is the model bias (constant over the years, but could be time and space-dependent) more corrected by a greater number of observations in the assimilation? Poli, Observation errors in ERA-20C, EMS 2013

18. Model bias problem in ERA-20C Mean surface pressure analysis increments In 1900 Only in the region 10S-10N [hPa] • Model time-step of 60 minutes was employed instead of 30 minutes (CPU cost constraints) • This time-step is insufficient to produce the right phase in the representation of atmospheric tides • As a result, systematic increments are produced at the same time of day (which can be seen here by looking here around the equator (10S-10N) as a function of longitude) Poli, Observation errors in ERA-20C, EMS 2013

19. Estimating the observation error (standard deviation) • From Desrozierset al. (QJRMS, 2005) • Under hypotheses of optimality in the analysis and separation of scales between background and observation • Also, if we haven’t fully removed the observation biases, they may creep into the results • There are other methods • Usually also assuming separation of scales Poli, Observation errors in ERA-20C, EMS 2013

20. Observations in the ERA-20C feedback Hereafter, we ignore report types with fewer than 100 obs. per year Poli, Observation errors in ERA-20C, EMS 2013

21. Surface pressure observation error estimates Poli, Observation errors in ERA-20C, EMS 2013

22. Interpretation of surface pressure observation error estimates • Land stations: • Observation quality appears to improve over the century, from 1.6 hPa in 1900 to about 0.8 hPa in the years 2000s (assumed estimate was 1.1 hPa) • This improvement is in two steps: • 1.6 1.0 hPa pre-World War II, • 1.0 hPa0.8 hPa in the years 2000s • Possibly a mixture of factors: generally improved instrumentation and site installations, more regular calibration practices and facilities, and also probably more accurate reporting of the observation time and automatic reporting with fewer errors in reading the instruments (?). • Buoys: • Of worse quality in the early years of introduction (in the 1970s); the location error may play a large role in this total error estimate. • In the recent times, of similar quality as from land stations • Ship: • Observation quality appears to improve over the century, from 2 hPa in 1900 to about 1.2 hPa in recent years (assumed estimate 1.5 hPa) • Ocean bottle and Conductivity Temperature Depth (CTD), eXpandable CTD (XCTD), Mechanical or digital or micro Bathythermograph (MBT), Expandable Bathythermograph (XBT): • Estimates based on small yearly samples • Nevertheless, estimates are substantially larger than for other report types and would need to be understood in future repeats of similar reanalysis. • Tropical cyclone bogus • About near-constant estimates around 4–5 hPa • Could reflect representativeness error & the poor horizontal resolution of ERA-20C. • Meteorological vessels: • In the 1980s, their quality approaches that of land stations • Better known position of the ship, more regular observing times closer to the schedule time, more regularly recalibrated instrumentation, more trained onboardpersonnel(?) Poli, Observation errors in ERA-20C, EMS 2013

23. Automatic stations (from the ERA-Interim feedback) • Land SYNOP • Automatic land SYNOP only Estimated from ERA-Interim Assumed in ERA-Interim Poli, Observation errors in ERA-20C, EMS 2013

24. Pressure observation error estimates with respect to station altitude 4000m • Estimates for values reported at station level show increased errors in altitudes • Representativeness issues in mountains • Estimates for values reported at sea level (using some formula*) show much increased height dependence • Reduction to sea-level introduces errors 3000m 2000m 1000m 0m * • Pressure reduction formulae have been discussed for a while (WMO, 1954; 1964; 1968). • As of early 2013, the World Meteorological Organisation recommends a single practice only for stations below 750 m altitude (WMO Commission for Instruments and Methods of Observation Expert Team on Standardisation, 2012). • For stations located at altitudes above 750 m, the reduction methods are still a matter of regional choices. The resulting differences reach a few hPa for high-altitude stations. Poli, Observation errors in ERA-20C, EMS 2013

25. Wind component observation error estimates Poli, Observation errors in ERA-20C, EMS 2013

26. Interpretation of wind component observation error estimates • Ship: • Around 2.2 m/s per component • Quality appears to increase between WWI & II • Quality appears to degrade from the 1970s • Could this be related to the greater size of the ships? (i.e. not measuring what we think we are using – not ’10-meter’ wind anymore) • Buoys: • Would appear to have the ‘best’ quality • Could be because their measurement height is the closest to expected? Poli, Observation errors in ERA-20C, EMS 2013

27. Interpretation of wind component observation error estimates • General remark about observation representativeness error • Above ocean, the error structures caused by non-representation of fine-scale wind, even if transient in principle, may look the same for all the ships and buoys because the background is produced by the same (low-resolution) model • Consequently may the diagnostic used here assign representativeness part of the observation error to the background error? • In such case, the estimate computed here would thus be an under-estimation • Some ideas to go further • Estimate wind speed and wind direction errors, instead of zonal and meridional component estimates • Retrace how wind estimation methods evolved over the years Poli, Observation errors in ERA-20C, EMS 2013

28. How reliable are those estimates? • Quick answer: We can’t really tell right away! • Let’s look at regional disparities For ship reporting observations of mean sea level pressure, from ICOADS 2.5.1, after assimilation in ERA-20C Poli, Observation errors in ERA-20C, EMS 2013

29. How useful are these revised (larger) observation error estimates? Surface pressure at Montreal, Quebec Observations from ISPD 3.2.6, collection #3004 (Canadian Stations Environment Canada ) ERA-20C Observations with uncertainties (some could not be fitted – they are VARQC rejected) Analysis, with uncertainties Background forecast, with uncertainties in the model and its forcings (HadISST2.1.0.0 ensemble) Increased sigo Same observations but with LARGER uncertainties No obs. rejected Analysis, presents LARGER uncertainties Background forecast, Initial state (and subsequent ones) presents LARGER uncertainties Poli, ERA-20C Data Assimilation System, EMS 2013

30. Conclusions • Nothing is perfect… … The observations … Our procedures to detect gross errors … Our estimations of observation biases … Our assumptions about observation errors • A reanalysis provides the means to realize all of this, in 2 steps: • Creation of a unique archive of collocated observations and model data (DONE) • Exploitation of the observation feedback archive so as to improve procedures, estimates, understanding (TODO) • Today we saw a few examples; this talk shall motivate further investigations from researchers and users Poli, Observation errors in ERA-20C, EMS 2013