Extracting probabilistic severe weather guidance from convection-allowing model forecasts

1. Extracting probabilistic severe weather guidance from convection-allowing model forecasts Ryan Sobash 4 December 2009 Convection/NWP Seminar Series

2. Identification of severe convection in high-resolution models Today’s state-of-the-art NWP models are run at resolutions capable of explicitly representing convection [< 4km]. Although these models do not explicitly predict the phenomena responsible for severe reports, the potential exists to produce guidance for these hazards if a relationship exists between the “intensity” of model convection and observed severe weather reports. If this relationship is robust, high-res model forecasts could be used to produce automated forecast guidance of convective hazards, just as traditional guidance is produced with environmental parameters.

3. Previous work – Identifying supercells • Several studies have investigated the issue of mining output of operational models to identify severe convection, primarily supercells. • Elmore et al. (2002) identified supercells (using a w-zeta correlation) in an operational high-res ensemble to provide guidance to forecasters on storm intensity/longevity. Shaded: UH > 25 m2 s-2 Kain et al. (2008) • Kain et al. (2008) identified rotating storms in real-time model output using an integrated helicity parameter (updraft helicity - UH) during Spring 2005 to obtain statistics on number/coverage of modeled supercells. • Kain et al. (2008) preferred the UH parameter over a correlation parameter (supercell detection index - SDI). Both produced similar results.

4. Previous work – Identifying supercells • The updraft helicity parameter is an measure of the vertical component of helicity associated with an updraft: • The quantity is integrated between 2 km and 5 km AGL to identify rotating updrafts in the lower-to-middle troposphere. It is computed on the model grid using a mid-point approximation. • Restricted to positive values (i.e. cyclonically rotating updrafts). Could potentially identify shear areas not associated with supercells?

5. Previous work – Spring Experiment 2008 • During the 2008 NSSL/SPC Spring Experiment, severe convection was identified in a convection-allowing ensemble to produce guidance for forecasters. • Along with UH, other fields associated with severe convection were identified in the model output. • > large updraft helicity • > strong low-level wind speed • > moderately strong low-level winds co-located with linear reflectivity segments • The grid points that met any of these criteria were flagged and accumulated over a 24 hour period (using 13-36 hour model forecasts). • These grid points are interpreted as the locations of “surrogate” severe weather reports.

6. Previous work – Spring Experiment 2008 • A Gaussian smoother was applied following the procedure outlined in Brooks et. al. (1998) to produce a “practically perfect” Convective Outlook given a distribution reports. • Subjective interpretations indicated this technique routinely captured the areas of severe reports.

7. Current Study • An investigation of this guidance concept was undertaken. Although an ensemble was used during SE2008, a deterministic model run was used in this work. • NSSL-WRF Configuration: • > WRF-ARW V2.2 • > Initialization time: 00 UTC • > Forecast length: 36 hours • > Horizontal resolution: 4 km • > Physics: MYJ BL, WSM6 MP • http://www.nssl.noaa.gov/wrf Domain

8. Current Study • 5 fields were chosen to identify convection in the NSSL-WRF output: UH: updraft helicity (computed between 2 km and 5 km) [m2 s-2] • UU: 10 m wind speed [m s-1] • RF: 1 km AGL simulated reflectivity [dBz] • UP: max. column updraft (below 400 hPa) [m s-1] • DN: max. column downdraft (below 400 hPa) [m s-1] • To capture intra-hourly convective-scale variations, the maximum value of each field within an hour was recorded. • Grid points where a field exceeds a severe “threshold” were flagged and are referred to as surrogate severe reports. Focus on 13-36 hour forecasts.

9. Current Study • The thresholds were chosen subjectively during SE2008. To provide a more systematic examination, a range of thresholds were selected from each field’s frequency distribution during SE2008, near the original subjective thresholds. Specific thresholds based on percentiles of the distribution. UH:33 > 103 m2 s-2 UU: 19 > 24 m s-1 RF: 53 > 56 dBz UP: 20 > 27 m s-1 DN: -5 > -7 m s-1

10. Current Study • Each day’s surrogate reports are placed on an 80 km grid and smoothed using a Gaussian weighting function. Since the Gaussian weights sum to 1, the result can be interpreted as a probabilistic forecast. • The final product is a surrogate severe probability forecast (SSPF). • This post-processing technique was performed by Theis et. al. (2005) on precipitation forecasts to introduce a simple measure of spatial uncertainty into deterministic high-resolution forecasts.

11. Case: 29 May 2008 UH UU RF DN UP OBS

12. Case: 29 May 2008 UH UU RF UP OBS DN

13. Research Questions Does the SSPF provide skillful probabilistic guidance? Can the SSPF provide useful guidance to SPC forecasters? ? • What fields produce the most skillful guidance? • For a given field, which thresholds produce the most skillful guidance? • What forms of guidance are most appropriate? • How reliable are the probabilistic forecasts?

14. SSPF Verification • ROC (relative operating characteristic) curves • > assesses ability of forecasts to discriminate between different outcomes – is conditioned on observations • > plot of probability of detection (POD) vs. probability of false detection (aka false alarm rate – FAR) • > diagonal is POD=POFD (chance of forecasting event is equal to chance of forecasting a false alarm) • > commonly summarized with ROC curve area • Reliability diagrams • > assesses ability of forecasts to produce reliable probabilities – is conditioned on the forecasts • > ideally, want the forecasts of 30% to occur 30% of the time • A skillful probabilistic forecasting system produces large ROC curve areas and is highly reliable. Images courtesy Austrailian Bureau of Meteorology Verification website

15. SSPF Verification SSPF-UH ROC curves for SE2008 ROC curve areas Decreasing threshold

16. SSPF Verification SSPF-UU ROC curves for SE2008 ROC curve areas Decreasing threshold

17. SSPF Verification SSPF-RF ROC curves for SE2008 ROC curve areas Decreasing threshold

18. SSPF Verification SSPF-UP ROC curves for SE2008 ROC curve areas Decreasing threshold

19. SSPF Verification SSPF-DN ROC curves for SE2008 ROC curve areas Decreasing threshold

20. SSPF Verification SSPF-UH Reliability diagram for SE2008 No Skill Increasing threshold Climatology

21. SSPF Verification SSPF-UU Reliability diagram for SE2008 No Skill Increasing threshold Climatology

22. SSPF Verification SSPF-RF Reliability diagram for SE2008 No Skill Increasing threshold Climatology

23. SSPF Verification SSPF-UP Reliability diagram for SE2008 No Skill Increasing threshold Climatology

24. SSPF Verification SSPF-DN Reliability diagram for SE2008 No Skill Increasing threshold Climatology

25. SSPF Verification • Largest ROC curve areas using lowest thresholds for UH, UP, DN. • ROC curve areas appear to asymptote approaching these thresholds. • UH naturally reliable at mid-range (~50 m2s-2) thresholds, other fields tend to overforecast probabilities at all thresholds. • Probabilistic forecasts which have large ROC curve areas, but are insufficiently reliable can be calibrated to improve reliability.

26. SSPF Verification • SSPF forecasts were produced with constant sigma = 120 km. Can reliability of the forecasts be improved by changing this smoothing parameter? sigma = 120 km sigma = 160 km sigma = 240 km Too cold… Too hot… Just right?

27. Increase sigma SSPF Verification SSPF-UH Reliability for SE2008 sigma=80 km sigma=160 km sigma=240 km

28. SSPF Verification SSPF-UU Reliability for SE2008 sigma=80 km sigma=160 km sigma=240 km

29. SSPF Verification SSPF-RF Reliability for SE2008 sigma=80 km sigma=160 km sigma=240 km

30. SSPF Verification SSPF-UP Reliability for SE2008 sigma=80 km sigma=160 km sigma=240 km

31. SSPF Verification SSPF-DN Reliability for SE2008 sigma=80 km sigma=160 km sigma=240 km

32. SSPF Verification Summary • UH is the best predictor as indicated by ROC area, reliability diagrams. • The best performing SSPF is UH with sigma ~ 200 km, using a threshold ~ 35 m2 s-2. • Calibration of other fields is more challenging… • - Changes in sigma have desired effect for some forecast probabilities, but not for others. • - Still have an overforecasting problem for UU, RF, UP, DN. • UH is uniquely suited to identify severe convection. • Increasing sigma decreases potential for higher probabilities. For rare-event forecasting, this may not be an issue.

33. Future work • Work is underway to verify forecast over a 1.5 year period to determine if these findings are applicable under all seasons and regions. • Discriminate between severe weather type. • Applying the SSPF procedure to an ensemble of forecasts. Will help improve upon this proof-of-concept.

34. Research Questions Does the SSPF provide skillful probabilistic guidance? Yes. Can the SSPF provide useful guidance to SPC forecasters? Potentially. Additional calibration needed. Could be useful as a starting point in the analysis of model data. ? Day1 Surrogate Convective Outlook 29/1200Z – 30/1200Z Model: 20080529/00Z NSSL-WRF