1 / 24

Huiling Yuan 1 , Xiang Su 1 , Yuejian Zhu 2 , Yan Luo 2, 3 , Yuan Wang 1

Evaluation of TIGGE ensemble predictions of Northern Hemisphere summer precipitation during 2008-2012. Huiling Yuan 1 , Xiang Su 1 , Yuejian Zhu 2 , Yan Luo 2, 3 , Yuan Wang 1

williamswalter
Télécharger la présentation

Huiling Yuan 1 , Xiang Su 1 , Yuejian Zhu 2 , Yan Luo 2, 3 , Yuan Wang 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of TIGGE ensemble predictions of Northern Hemisphere summer precipitation during 2008-2012 Huiling Yuan1, Xiang Su1, Yuejian Zhu2, Yan Luo2, 3, Yuan Wang1 1. Key Laboratory of Mesoscale Severe Weather/Ministry of Education and School of Atmospheric Sciences, Nanjing University, China 2. Environmental Modeling Center/NCEP/NWS/NOAA, College Park, Maryland, USA 3. I.M. Systems Group, Inc., College Park, Maryland, USA WWOSC 2014, Montreal, Canada, 21 August 2014

  2. Outline • Objectives • The TIGGE Ensemble Prediction Systems (EPSs) data • Evaluation results of TIGGE quantitative precipitation forecasts (QPFs) and probabilistic QPF (PQPFs) • Summary the THORPEX Interactive Grand Global Ensemble (TIGGE)

  3. Objectives • Evaluate the QPF and PQPF performance of TIGGE EPSs • Assess the performance change before and after major EPS upgrade Su et al. 2014, JGR

  4. Evaluation of TIGGE QPFs and PQPFs ECMWF portal http://tigge-portal.ecmwf.int/ • Study period: 2008-2012summer(June-August) • Spatial coverage: Northern Hemisphere (NH) tropics (0-20°N) and midlatitude (20-49°N) • Accumulation period: 24-h precipitation (12 UTC-12 UTC) • Forecast lead time: 1-9days • Horizontal resolution: 1° ×1° (interpolation) • Observation data: TRMM3B42 V7 (gauge adjustment)

  5. Configurations of six TIGGE EPSs 1. The CMC EPS was upgraded to version 2.0.2 on 17 August 2011. 2. The ECMWF EPS used a horizontal resolution of N200 (~0.45º) for 0-10 day forecasts and N128 (~0.7º) for 10-15 day forecasts before 26 January 2010. EVO-SVINI was used as the initial perturbation method before 24 Jun 2010. The SPBS method has been added on 9 November 2010. 3. The JMA EPS began to use the SPPT method on 17 December 2010. 4. The NCEP EPS was upgraded to version 8.0 and began to use the STTP method on 23 February 2010. In 14 February 2012, the NCEP EPS was upgraded to version 9.0. 5. The UKMO EPS used a horizontal resolution of 1.25º×0.83º before 9 March 2010. Referenced to the CMA EPS (frozen), the impacts of major model upgrades on the forecast performance are examined for other five EPSs.

  6. Verification methods of QPFs and PQPFs • Area-weighted scores consider the latitude discrepancies • Continuous scores: RMSE, Spatial correlation (SC), CRPSS, and spread-skill relationship • Dichotomous scores: Bias, ETS, POD, FAR, BSS, attributes diagram (reliabilitycurve and three decomposed terms of BS), ROCarea and potential economic value (PEV) where：wi=cos(lat) xi, yiare forecast and observation samples, Nis the number of samples Su et al. 2014, JGR

  7. Precipitation climatology (day +3) CMA NCEP JMA CMC JMA day 1 ECMWF TRMM UKMO mm/day Cause: JMA employs moist SVs over the entire tropics and perturbs the specific humidity with a large amplitude (Yamaguchi and Majumdar, 2010) JMA day +1 ensemble mean QPFshave large moist biases in the NH tropics

  8. Forecast error (RMSE) Ensemble is better than control, especially in longer lead times EC ensemble best EC control is better than CMA ensemble in short lead times JMA control in NH midlatitude best Control: dotted (…..) Ensemble mean: solid (——)

  9. RMSE and frequency The control QPFs of JMA have the smallest RMSE in the NH midlatitude Cause: JMAunderestimates heavy rain It is not appropriate to use only RMSE to evaluate QPFs

  10. Discrimination diagrams • The discrimination ability decreases with the lead time • EPSs have weak ability to discriminate heavy events • In the NH tropics, CMA shows little discrimination ability among different rain events

  11. Dichotomous scores of ensemble mean QPFs • ECMWFbest • CMAvery poor in the NH tropics

  12. Spread-skill relationship • The day +1 ensemble spread of JMA is the largest in the NH tropics • CMC has the largest spread and it grows with the lead time: NH midlatitude: level with the ensemble mean error NH tropics: slightly overdispersive Spread: dash (- - -), RMSE: solid (——)

  13. PQPF error: CRPSS • ECMWFbest • In the NH tropics, the day +1 CMA is even poorer than the day +9 ECMWF • Skill of CMC rapidly drops from day +2

  14. PQPF skill: BSSand ROCarea • Light rain: CMCbest • Heavy rain: CMC and ECMWFbetter • CMAis very poor in the NH tropics

  15. Attributes diagram (reliability curve, BSS, and BS terms) NH Midlatitute

  16. Potential economic value • Prob. thresholds of CMCare most reliable • ECMWF has the highest PEV

  17. Performance changes due to major EPS upgrade Changes of spread, Spread/RMSE ratio are significant for CMC and ECMWF Change of RMSE is only significant for CMC (increase) Spread: grey, RMSE: black

  18. Performance changes due to major EPS upgrade Changes of spread, Spread/RMSE ratio are significant for UKMO, NCEP, JMA Change of RMSE is only significant for UKMO (decrease) Spread: grey, RMSE: black

  19. Changes due to major EPS upgrade • Use the frozen CMA as the reference to eliminate the interannual variability • CMC greatly increases spread and spread/RMSE ratio • Similar performance changes for other lead times Changes due to major EPS upgrade of day +3 spread, RMSE, and Spread/RMSE ratio in the NH midlatitude (significant change with 95% confidence interval)

  20. Skill changes After EPS major upgrade: • CMC: decreased skill • ECMWF, UKMO,NCEP:improved • JMA: no significant change

  21. CMC EPS upgrade Wednesday February 13, 2013 • Major upgrade to the Global Ensemble Prediction System (GEPS) version 3.0.0 at the Canadian Meteorological Centre “Changes installed uniquely into the forecast component include: adjustments to how physics tendencies are perturbed for convective precipitation; the physics tendencies perturbations are applied at every level except the very last one; addition of diffusion into the advection procedure; perturbation of the bulk drag coefficient in the orographic blocking scheme; and fine tuning of the adjustment factor alpha of the stochastic kinetic energy backscattering scheme.” http://collaboration.cmc.ec.gc.ca/cmc/CMOI/product_guide/docs/changes_e.html#20130213_geps

  22. Summary Evaluation of the QPFs and PQPFs from six TIGGE EPSs in the NH midlatitude and tropics during the boreal summers of 2008-2012: • Ensemble mean QPFs: CMA: large systematic biases, poor performance in the NH tropics ECMWF: less errors and best skill JMA: unusually large moist biases of day +1 QPFs in the NH tropics • PQPFs: CMC: relatively good for light precipitation and short lead times, increased spread and larger errors for longer lead times; better reliability and reliable probabilistic thresholds in PEV ECMWF: best skill, except for light precipitation; best discrimination ability and highest potential economic benefit NCEP and UKMO: most sharp

  23. Summary • The model upgrade in EPS cannot always guarantee skill improvements • The enlarged ensemble spread of CMC forecasts after the upgrade increases the QPF errors • Uncertainties and quality of verification data • How to fairly evaluate an EPS is essential for the development and upgrade of the EPSs Comprehensive evaluation with multiple verification metrics • Provide general guidance for the postprocessing of the EPSs Reliability and discrimination ability

  24. Thank you E-mail: yuanhl@nju.edu.cn

More Related