1 / 52

Observations on WAFS Gridded Forecasts

WAFS Workshop on the Use and Visualization of Gridded SIGWX Forecasts 14-15 September 2009. Observations on WAFS Gridded Forecasts. Overview. Introduction Compatibility of forecasts of the two WAFCs Quality issues: seasonal aspects Quality issues: CB diurnal variation

michel
Télécharger la présentation

Observations on WAFS Gridded Forecasts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WAFS Workshop on the Use and Visualization of Gridded SIGWX Forecasts 14-15 September 2009 Observations on WAFS Gridded Forecasts

  2. Overview • Introduction • Compatibility of forecasts of the two WAFCs • Quality issues: seasonal aspects • Quality issues: CB diurnal variation • Quality issues: CB associated with TC • Verification issues • Calibration issues • User survey • Conclusions

  3. Introduction • WAFSOPSG/4: WAFC Provider States, in coordination with WAFSOPSG members from IATA, IFALPA and WMO, to undertake systematic comparisons of trial gridded forecasts of icing, turbulence and CB clouds of the two models, highlighting characteristics of areas with different values. • The results should lead to the alignment of algorithms used by the two WAFCs.

  4. Introduction • To facilitate comparison of the trial gridded forecasts, HK, China set up website (http://wafs-grid-fc.weather.gov.hk) to display gridded values (without smoothing/interpolation). • The observations were reported in CNS/MET SG/13 meeting in July 2009 and were agreed to be referred to the WAFSOPSG/5 meeting. (Earlier observations reported in CNS/MET SG/12 meeting in 2008)

  5. Compatibility of forecasts of the two WAFCs UK US Diff • Difference map shows absolute difference in the two forecasts (little agreement between the two forecasts if difference map appears to be the union of them). T+24 h Max CAT Potential at FL340 based on 12 UTC 2 July 2009 )

  6. Compatibility of forecasts of the two WAFCs • CB horizontal extent - CB coverage of UK more extensive - Forecast values of US reaching 50-75% (OCNL following Annex 3 spec) over tropics, larger than those given in the UK forecast • ICAO height at CB top -UK still gives more extensive CB areas - UK over-forecast CB for vast areas in the Pacific Ocean • Mean in-cloud turbulence potential (FL180) - UK and US forecast values with similar range - UK forecast shows small values almost everywhere

  7. Compatibility of forecasts of the two WAFCs • Max icing potential (FL180) - Significant differences in the spatial coverage over the tropical and higher latitude regions - WAFS Science Coordination Meeting 2009: “percentage overlap of all (icing) forecasts between UK and US ~ 20%” • Max CAT potential (FL340) - UK gives much less extensive CAT areas (except over certain spots)

  8. Compatibility of forecasts of the two WAFCs • Difference maps in most cases appear to be the union of the forecasts from the two centres, indicating little agreement

  9. Latest CB forecasts of UK MTSAT-1R infrared satellite imagery, with deep convection indicated in red, at 08 UTC on 14 Sep 2009 CB top height valid 12 UTC 14 Sep 2009 CB horiz extent valid 12 UTC 14 Sep 2009

  10. Quality issues: seasonal aspectsmean in-cloud turbulence at FL 180 T+24 h Gridded Forecasts (based on 00 UTC 6 Jan 2009) US UK For both winter & summer: large values are forecast by UK for high latitudes in both hemispheres. They are not seen in the US forecasts. T+24 h Gridded Forecasts (based on 00 UTC 2 Jul 2009)

  11. Quality issues: seasonal aspectsmax icing potential at FL 180 • At high latitudes in winter: large values • (> 85%) forecast by UK, but much smaller values (mostly • < 30%) forecast by US • Scientific paper on icing climatology:non-occurrence of icing at FL180 in the wintertime north of 25N T+24 h Gridded Forecasts (based on 00 UTC 6 Jan 2009) UK US T+24 h Gridded Forecasts (based on 00 UTC 2 Jul 2009)

  12. Quality issues: seasonal aspects max CAT potential at FL340 – 3 days in Jan 2009 • large values are forecast by UK for a number of isolated locations • stationary “hot spots” associated with mountainous regions

  13. Quality issues: seasonal aspects • Some common issues can be identified when data for different seasons are monitored (e.g. in-cloud turb, icing) • Some issues (e.g. terrain-related) may show up at different locations in different seasons • The above observations are not exhaustive • It is imperative for the WAFCs to: • conduct verification for all seasons • examine model climatological biases

  14. Quality issues: seasonal aspectsClimatology of differences between two WAFCs (May – Aug 2009) CB horiz extent CB top height Mean in-cloud turb Max icing Max CAT ? ?

  15. Quality issues: CB diurnal variation ICAO height at CB top(UK) T+24h gridded forecasts Based on 00 UTC 17 August 2008 T+24h gridded forecasts Based on 06 UTC 17 August 2008 Solar heating

  16. Quality issues: CB diurnal variation ICAO height at CB top(US) T+24h gridded forecasts Based on 06 UTC 17 August 2008 T+24h gridded forecasts Based on 00 UTC 17 August 2008 Solar heating Again the UK forecasts are giving much more extensive CB areas compared to the US forecasts

  17. Quality issues: CB diurnal variation • Actual weather as shown by the satellite images • Weather was generally fine over southern China throughout the period • Both UK and US over-forecast CB occurrence over this region at 06 UTC (14 H local time), probably due to difficulties of the models in handling CB development induced by daytime solar heating MTSAT-1R infrared satellite imagery, with deep convection indicated in red, at 23:30 UTC on 17 August 2008 MTSAT-1R infrared satellite imagery, with deep convection indicated in red, at 05:30 UTC on 18 August 2008

  18. Quality issues: CB diurnal variation • 2009 WAFS Science Coordination Meeting acknowledged that further work would need to be undertaken by the WAFCs for improving the CB gridded forecasts, including to “look at diurnal variations of forecasts (manual and automatic), particularly in tropics”

  19. Quality issues: CB associated with TC UK US Based on 06 UTC 19 June 2009 Rather difficult to identify the area associated with TC without referring to the satellite imagery Satellite imagery clearly shows deep convection associated with tropical cyclone Linfa Need TC (and other SIGWX) information in future visualization models

  20. Verification issues • APANPIRG called for systematic SIGWX verification in 2006 • WAFSOPSG/4: Evaluation of the SIGWX forecasts proposed by APANPIRG would be sufficiently addressed in the systematic comparison of WAFS SIGWX forecasts with the gridded forecasts

  21. 2009 Science Coordination Meeting + POD = probability of detection, or hit rate = hit / (hit + miss). • ^ FAR = false alarm rate = false alarm /(false alarm + correct rejection) . For rare events, correct rejection is very large! @ No indication whether max or mean turb potential was considered, and whether or not forecasts for all flight levels were verified. # DEVG = Derived equivalent vertical gust * Not a product of the CB gridded forecasts

  22. Verification issues • In-cloud turbulence – systematic verification yet to be done. • Icing – verification based on another icing product (CLIP) as ground truth, with “trace icing” criterion Kay et al (2009) on CLIP: “It is important to note that our algorithm only states where icing conditions have the potential to exist rather than stating exactly where within the clouds exactly the icing does or does not exist. ... Therefore our algorithm should be considered as a biased representation of icing conditions in the atmosphere. In the absence of a true measure of icing conditions the amount of bias cannot be known”. The choice of the ground truth for the verification, as well as the verification methoditself, must be agreed by WAFSOPSG. “Trace icing”  calibration issue.

  23. Verification issues • CAT – verification based on DEVG >= 2 m/s from aircraft data, with numerous thresholds, rather than ICAO adopted turbulence metrics, i.e. MOD/SEV in PIREP or EDR thresholds. Numerous thresholds  calibration issue. • CB – verification based on sferics data, with numerous thresholds, rather than observations of CB (e.g. from satellite). Also lack of information on how CB forecast parameters are derived, and use of smoothing algorithm by UK.

  24. Calibration issues • Need to be mindful about users’ perception and expectation based on their experience with the existing SIGWX forecasts • Thresholds for verifying and visualizing gridded forecasts need to be determined in consultation with users

  25. User survey • HKO conducted a user survey on the WAFS trial gridded forecasts in Jul to Aug 2009, 15 returns from IFALPA, pilots, airline management and dispatchers: • 80% consider gridded products are NOT suitable for operational use in flight planning and flight documentation; • 93% considered gridded products CANNOT replace the SWH and SWM charts. • Specific user comments are summarized below.

  26. User comments

  27. User comments

  28. User comments

  29. User comments

  30. Conclusions • Suitability of WAFS gridded forecasts of icing, turbulence and CB for operational use : • Issues with compatibility, quality, verification, and calibration need to be satisfactorily addressed before they could be considered suitable for operational use. • Systematic verification: WAFS output performance indicators (including methodology, thresholds, choice of ground truth) for verifying the gridded forecasts should be systematically developed and agreed. • Calibration: the thresholds for verifying and visualizing the gridded forecast should be calibrated for compatibility with users’ expectation/perception. Pilots have problems with the use of “potential” for icing and turbulence.

  31. Conclusions • Suitability of WAFS gridded forecasts of icing, turbulence and CB to replace the current SIGWX forecasts (SWH and SWM): • Not at this stage (not for operational use) • Visualization issues (including thresholds to be adopted, forecast irregularities e.g. CB top/base heights) need to be resolved • High “at-a-glance” visualization models should be agreed by users • SIGWX features other than icing, turbulence and CB required by users should be included, especially if they are safety-related (e.g. TC, VA)

  32. Thank you

  33. Supplementary information

  34. Observations reported in CNS/MET SG/12, 2008CB horizontal extent valid 00 UTC 23 Jun (based on 00 UTC 22 June 2008) UK US CB coverage is more extensive with generally larger forecast values in the UK forecast compared with that of US. Furthermore, small values of CB horizontal extent are given almost everywhere in the UK forecast Back

  35. Observations reported in CNS/MET SG/12, 2008 ICAO height at CB top UK US valid 00 UTC 23 Jun (based on 00 UTC 22 June 2008) UK forecast generally gives more extensive but lower heights of CB top (generally below 37kft) compared with that of US (above 43 kft) Back

  36. Observations reported in CNS/MET SG/12, 2008 Mean in-cloud turbulence potential UK US valid 00 UTC 23 Jun at FL180 (based on 00 UTC 22 June 2008) UK forecast generally gives wider range and higher turbulence potential values compared with that of US. On the other hand, small values of mean in-cloud turbulence potential are given almost everywhere in the US forecast Back

  37. Observations reported in CNS/MET SG/12, 2008 Maximum icing potential UK US valid 00 UTC 23 Jun at FL180 (based on 00 UTC 22 June 2008) At FL180: UK forecast giving generally higher icing potential over the higher latitude regions; US forecast giving generally higher icing potential over the tropical region and the UK forecast giving generally higher icing potential over the higher latitude regions Back

  38. Observations reported in CNS/MET SG/12, 2008 Maximum CAT potential UK US valid 00 UTC 23 Jun at FL340 (based on 00 UTC 22 June 2008) At FL340: UK forecast generally gives much less extensive CAT areas compared with that of US Back

  39. Back

  40. Bernstein, B.C. and Christine LeBot, 2009: An Inferred Climatology of Icing Conditions Aloft, Including Supercooled Large Drops Part II: Europe, Asia and the Globe, Journal of Applied Meteorology and Climatology, American Meteorological Society Back

  41. CB horizontal extent at FL340 T+24 h Gridded Forecasts (based on 12 UTC 2 July 2009) UK US Diff Back

  42. Back

  43. ICAO height at CB top T+24 h Gridded Forecasts (based on 12 UTC 2 July 2009) UK US Diff Back

  44. ICAO height at CB top T+24 h Gridded Forecasts (based on 12 UTC 2 July 2009) UK US MTSAT-1R IR sat pic With deep convection indicated in red At 11:30 UTC 3 July 2009 Back

  45. Back

  46. Mean in-cloud turbulence potential at FL 180 T+24 h Gridded Forecasts (based on 12 UTC 2 July 2009) UK US Diff Back

  47. Back

  48. Maximum icing potential at FL 180 T+24 h Gridded Forecasts (based on 12 UTC 2 July 2009) UK US Diff Back

  49. Back

  50. Maximum CAT potential at FL 340 T+24 h Gridded Forecasts (based on 12 UTC 2 July 2009) UK US Diff Back

More Related