What is the half-life of a cloud forecast?

1. What is the half-life of a cloud forecast? Robin Hogan Ewan O’Connor University of Reading, UK

2. Cloud fraction “Bony diagrams” Winter (Oct-Mar) Summer (Apr-Sep) ECMWF model Chilbolton

3. How good is a forecast? • Most model comparisons evaluate the cloud climatology • What about individual forecasts? • Standard measure shows forecast “half-life” of ~8 days (left) • But virtually insensitive to clouds! ECMWF 500-hPa geopotential anomaly correlation • Overview of talk • Which skill scores have the most desirable properties? • How does skill depend on spatial scale, lead time etc? • If it has an inverse-exponential decay with forecast lead time, what is the “half-life” of the forecast?

5. Raw (1 hr) resolution 1 year from Murgtal DWD COSMO model Joint PDFs of cloud fraction b a d c • 6-hr averaging …or use a simple contingency table

6. Desirable properties of skill scores • Equitable: all random forecasts score zero • This is essential! • Note that forecasting the right climatology versus height but with no other skill should also score zero • Proper: not possible to “hedge your bets” • Some scores reward under- or over-prediction (e.g. hit rate) • Jolliffe and Stephenson: not possible to be equitable and proper! • Independence of how often cloud occurs • Almost all scores asymptote to 0 or 1 for vanishingly rare events • Dependence on 10x10 joint PDF, not just 2x2 table • Difference between cloud fraction of 0.9 and 1 is as important for radiation as a difference between 0 and 0.1 • Linearity: so that can fit an inverse exponential • Some scores (Yule’s Q) “saturate” at the high-skill end

7. Three quite good scores • 1. Log of odds ratio: LOR=ln(ad/bc) • Good “properness” properties • Unbounded: a perfect forecast scores infinity! Generalized skill score = (x-xrandom)/(xperfect-xrandom) • Where “x” is any number derived from the joint PDF • Resulting scores vary linearly from random=0 to perfect=1 • 2. Heidke skill score: x=a+d • Monotonically related to the Equitable Threat Score, but more linear • 3. Linear Brier score: x=mean absolute difference • Sensitive to cloud fraction errors in model for all values of cloud fraction

8. Both scores well fitted by S=S0exp(-t/t0) Half life=ln(2)t0 Met Office NAE has higher scores than DWD COSMO But apparently a shorter half life (~2.7 days versus ~4.1 days) Obviously need longer lead-time forecasts to check this! Score versus lead time, Murgtal 2007

9. DWD COSMO versus hours averaged • Skill and lead time both increase with the number of hours over which cloud fraction is averaged • Larger-scale features are easier to forecast

10. Statistics poorer for larger number of hours averaged Log of odds ratio and Heidke skill score are sensitive to cloud fraction threshold Linear Brier score considers all cloud fractions so more robust Met Office versus hours averaged

11. Summary • Half-life of a cloud forecast is between 2.5 and 5 days • Relatively insensitive to skill score (provided a good one is used) • Compare to ~8 days for ECMWF 500-hPa geopotential height forecast • Skill at forecasting cloud increases somewhat for larger scale features • Important to assess the merits of various skill scores • At least 5 criteria to judge against, and none are good on all • Plenty of bad ones to use (hit rate, false-alarm rate etc)! • Worth trying Stephensons’s “Extreme Dependency Score”, which is good for very rare events • Wish list • Obtain Met Office cloud forecasts beyond a lead time of 3 days • Compare skill of the Met Office model at different model resolutions, but averaged to the same scale • Can we see what skill comes from global model at boundaries, what comes from mesoscale data assimilation etc?

13. Contingency tables Model cloud Model clear-sky Comparison with Met Office model over Chilbolton October 2003 Observed cloud Observed clear-sky

14. Simple skill score:Hit Rate • Misleading: fewer cloud events so “skill” is only in predicting clear skies • Models which underestimate cloud will do better than they should Met Office short range forecast • Hit Rate: fraction of forecasts correct = (A+D)/(A+B+C+D) • Consider all Cabauw data, 1-9 km • Increase in cloud fraction threshold causes apparent increase in skill. Météo France old cloud scheme

15. More sophisticated scores • Yule’s Q =(-1)/(+1) where the odds ratio =AD/BC. • Advantage: little dependence on frequency of cloud • Equitable threat score =(A-E)/(A+B+C-E) where E removes those hits that occurred by chance. • For both scores, 1 = perfect forecast, 0 = random forecast • From now on use Equitable threat score with threshold of 0.1.

16. Monthly skill versus time • Measure of the skill of forecasting cloud fraction>0.05 • Comparing models using similar forecast lead time • Compared with the persistence forecast (yesterday’s measurements) • Lower skill in summer convective events

17. Skill versus lead time • Unsurprisingly UK model most accurate in UK, German model most accurate in Germany! • Half-life of cloud forecast ~2 days • More challenging test than 500-hPa geopotential (half-life ~8 days)