50 likes | 156 Vues
This document discusses the essential development of a toolkit aimed at advancing diagnostic verification capabilities beyond current standards (e.g., NCEP). It addresses three key questions regarding the computation of confidence intervals and hypothesis tests in the presence of spatial and temporal correlations, evaluation methods for extremes, and the implementation of various diagnostic approaches across multiple tiers. Emphasizing appropriate handling of autocorrelations, the need for clarity in defining extremes, and the importance of using a tiered method approach, this initiative seeks to improve operational precision.
E N D
Diagnostic verification and extremes: 1st Breakout • Discussed the need for toolkit to build beyond current capabilities (e.g., NCEP) • Identified (and began to address) 3 major questions: • How should confidence intervals and hypothesis tests be computed, especially when there are spatial and temporal correlations? • What methods should be included for evaluating extremes? • What diagnostic approaches should be included initially; in a 2nd tier; in a 3rd tier
Confidence intervals and hypothesis tests • Need to appropriately take into account autocorrelations • Reduce sample by eliminating cases • Block re-sampling (Candille results indicate spatial correlation may have more impact than temporal, at least for upper air) • Identify situations when parametric approaches are “ok” • Bootstrapping approaches are computer-intensive and require lots of data storage • May not always be practical in operational settings
Methods for evaluation of extremes • Need to distinguish (in our minds) between extremes and high impact weather • User should define thresholds for extremes • May be based on quantiles of sample distribution • Could use extreme value theory to help with this (e.g., return level methods) • Extreme dependency score is appropriate in many cases • Also compute standard scores: Yule’s Q; odds ratio; ORSS; ETS, etc.
Diagnostic methods • Goal: Identify different tiers of methods/capabilities that will be implemented over time, starting with Tier 1 in 1st release • Initial discussion: Stratification • Friday discussion: Specific methods
Stratification • Tier 1: Based on meta-data, including time-of-day, season, location, etc. • Tier 2: Based on other information from the model, such as temperature, wind direction, etc. • Tier 3: Based on feature such as location or strength of jet core; cyclone track; etc.