Forecast Skill and Major Forecast Failures over the Northeastern Pacific and Western North America

1. Forecast Skill and Major Forecast Failures over the Northeastern Pacific and Western North America Lynn McMurdie and Cliff Mass University of Washington

2. Why the North Pacific Ocean? • The Pacific is one of the largest areas of sparse insitu observations in the world • Uncertainty over the Pacific has a large impact on predictability over the North American continent and beyond • There are often large initialization errors and short-term forecasts over the northern Pacific ocean. • One symptom of such problems is that short-term forecast skill on the western side of North America is worse than in other areas.

3. Eta 24-h 03 March 00UT 1999 Eta 48-h 03 March 00UT 1999 An example of a short-term forecast error

4. And the public and media have noticed these failures….

5. Eugene Register Guard Seattle Times

6. Some applicable research in the literature • Langland et al. (2002) – poor forecast of Jan 2000 storm on East Coast due in part to sensitivity over the Pacific. • Bosart et al. (2002) – lack of convection over the midwest not represented in forecasts due to poorly initialized trough along west coast • Schultz et al. (2005) – 70% of troughs arriving on the West Coast were underforecast, a portion of which continued to effect short-term forecasts across the North American continent. • McMurdie and Mass (2004) – documented forecast failures over the eastern Pacific.

7. This talk will … • Demonstrate that large initial condition and short-term forecast errors still occur over the eastern Pacific and downstream • Present a feature-based approach to monitoring errors in this region • Discuss implications for THORPEX

8. How frequent are large numerical forecast errors? • Approach: compare buoys/coastal observations of sea level pressure (SLP) to NCEP’s Eta and GFS 00, 24, 36 and 48 hr forecasts. • Error = Forecast SLP – Observed SLP • At each station, calculated average and absolute error and the standard deviation using winter (Oct – Mar) data. • Large Error = |Error| > (average error + 2 * SD)

9. Station Locations Tatoosh Is. Cape Arago

10. Large Errors Inter-annual variability 24 h Tatoosh Is., WA Cape Arago, OR

11. 48-h Errors 48h errors much larger and more frequent than 24-h errors

12. GFSvs. Eta 24-h errors NCEP GFS better than Eta on average

13. 48-h errors GFS over forecasts Eta under forecasts

14. GFS has more accurate SLP initializations and forecasts than Eta over the Northeast Pacific • For 00-h forecasts (initial conditions), GFS has smaller mean absolute error (MAE) and standard deviation (SD) than Eta at all 17 stations • For 24- and 36-h forecasts, GFS has smaller MAE and SD than Eta at 13/17 buoy and coastal stations • For 48-h forecasts, GFS has smaller MAE and SD than Eta at 12/17 stations.

15. Forecast Verification: The Need for Feature-Based Evaluation • Examining statistics at observing sites is not sufficient for understanding the problems. • Must also track features to gain an understanding of the deficiencies. • Case studies of major failures should reveal important information.

16. GFS What are these large forecast errors associated with?

17. How frequently do large forecast errors of synoptic events occur? Event = large error at 2 or more adjacent stations for 2 or more forecasts periods Data shown for Eta model only Number of Events/Season associated with Lows/Troughs/Highs Season Total Low Trough High 1999 – 2000 21 12 7 2 2000 – 2001 19 12 4 2 2001 – 2002 16 14 2 0 2002 – 2003 16 11 5 0 (from McMurdie and Mass 2004)

18. Of the forecast failures associated with lows, what are the central pressure and cyclone position errors? Ave SLP error = 3.4 mb SD = 8.7 mb Absolute error = 7.5 mb Ave position error = 453.8 km SD = 260 km

19. Recent examples of major forecast errors • February 2002 • October 2003 • February 2004 • November 2004 • April 2005 • May 2005

20. L996 L1004 L1008 L1010 An example of a recent high-impact, poorly forecast storm 7 – 8 February 2002 Cyclone • Power outages, large trees uprooted in Eugene, OR • Powerful, rapidly developing storm with strong winds (70 kts) • Very poor short-term numerical guidance

21. ETA AVN UKMO NOGAPS 48-hr Forecasts Valid 00 UTC 8 February 2002

22. AVN ETA UKMO NOGAPS 24-Hr Forecasts Valid 00 UTC 8 February 2002

23. Difference between UKMO and Eta 850 mb Temperature K Valid 00 UTC 7 February 2002 L1010 Solid = UKMO, Dashed = ETA, Shades, blue = differences

24. 20 Oct 03

25. Flood of 20 October 2003

26. 00hr GFS 24hr GFS 48hr GFS 00hr + 48hr GFS

27. GFS Forecasts of 12-hr Precipitation 36h Forecast 24h Forecast 48h Forecast 12h Forecast

28. February 04

29. 00UTC 16 Feb 04 GFS 00-hr Forecast

30. 00 UTC 16 Feb 04 24-hr GFS

31. 00 UTC 16 Feb 04 48 hr GFS

32. Apr 05

33. 24-hr forecast GFS Position error ~ 420km

34. Large Short Term Forecast Errors Still Occur • Number of slp errors > 10 mb continues to be 10 – 15 per winter (despite the ridge this year) • Vast majority of large errors due to mispositioned or under (or over) forecast low centers (see McMurdie and Mass, 2004) • For Feb 02 case, forecast errors were likely due to initial condition errors (McMurdie and Mass, 2004)

35. Some Unanswered Questions • What are the origins of these short-term forecast errors – initial condition/data assimilation errors, model errors? • Are there particular flow patterns (or regimes) where short-term (or longer term) forecasts are less accurate (e.g., E-T transitions)? • How do model sensitivity structures compare for major forecast failure cases? How do they project on obvious initialization problems? How do adjoint-based and ensemble-based sensitivities compare for such cases?

36. Unanswered Questions continued • What are the downstream implications for medium to long- range forecasts when initial condition errors are large over the Pacific? To what degree are downstream errors mitigated by greater data density over North America?

37. Implications for THORPEX • Major forecast failures still occur, even at the short-time ranges. So there is still work to be done! • It is important to monitor the quality of model initializations and forecasts to know how well we are doing and where the failures are. • Both statistical and feature-based approaches are needed to gain a full understanding of model failures. • Case studies can provide important insights into forecast failures

38. The END

39. Recent trends in forecast accuracy From Simmons and Hollingsworth (2002) Hemispheric r.m.s. error of SLP Increased skill of 3-5 Day forecast skill of SLP (and 500 hghts) especially last 10 years. Unable to discern forecast skill of storm systems in particular locations from these statistics

40. Brief Outline • Show statistics of short term errors along West Coast • Highlight several examples of major forecast failures • Briefly discuss the effect of uncertainty over the Pacific on longer term forecasts

41. Adjoint Sensitivity wrt 850 mb Temperature Area of forecast error projected onto sensitivities Courtesy of Brian Ancell

42. 2004 17 Nov

43. 2004 17 Nov

44. 6-hr forecast Eta

45. 6-hr Forecast GFS