1 / 13

A Superior Alternative to the Modified Heidke Skill Score for Verification of Categorical Versions of CPC Outlooks

A Superior Alternative to the Modified Heidke Skill Score for Verification of Categorical Versions of CPC Outlooks. Bob Livezey Climate Services Division/OCWWS/NWS 28 th Climate Diagnostics and Prediction Workshop Reno, October 20, 2003. Outline. Introduction

ziv
Télécharger la présentation

A Superior Alternative to the Modified Heidke Skill Score for Verification of Categorical Versions of CPC Outlooks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Superior Alternative to the Modified Heidke Skill Score for Verification of Categorical Versions of CPC Outlooks Bob Livezey Climate Services Division/OCWWS/NWS 28th Climate Diagnostics and Prediction Workshop Reno, October 20, 2003

  2. Outline • Introduction 2. Contingency Tables & Notation • Common Scores & Score Attributes • Gandin & Murphy Equitable Scores • Gerrity Scores • Recommendations

  3. Contingency Tables and Notation Table 4.2. Contingency table giving pij in percent (total sample size n=788) for U.S. mean temperature forecasts for June through August 1983-90. pij: Joint relative frequencies pi:: Observed relative frequencies qi: Forecast relative frequencies pi*: Prescribed relative observed frequencies (climatology)

  4. Some Simple Categorical Skill Scores: Heidke, CPC Heidke, and Pierce

  5. Some Simple Categorical Skill Scores: Heidke, CPC Heidke, and Pierce Skill scores for U.S. mean temperature forecasts in three categories for February through April and June through August 1983-90.

  6. Desirable Attributes of Scores • Equitable; • Equitable without dependence on the forecast distribution; • Rewards for correct forecasts inversely proportional to their event frequencies; • Penalties for incorrect forecasts directly proportional to their event frequencies; • Penalties for incorrect ordinal forecasts with equal event frequencies proportional to degree of miss; • Consistent with an underlying linear association and insensitive to type or number of categories used. Note: 2 &3 imply that all information in the contingency table is taken into account.

  7. Figure 4.1

  8. Gandin and Murphy Equitable Scores

  9. Gerrity Scores

  10. Figure 4.4

  11. Recommendations • CPC use the Gerrity score for ordinal multi-categorical verification • Forecast history is digitized so skill history can be constructed • Clueless audience remains clueless • Score now equitably accounts for all facets of forecast performance • CPC use actual frequencies • CPC routinely determine confidence limits of scores • Reference Jolliffe and Stephenson (2003; Wiley)

More Related