1 / 51

Forecast text editing rates and analysis

Jaime Clark and Bianca Gibson Brianna Laugher Tennessee Leeuwenburg December 2012. Forecast text editing rates and analysis. www.cawcr.gov.au. Graphical Forecast Editor. GFE Text formatters. GFE Text formatters - editing. GFE Forecasting. Guidance. Grid Editing & Smart Tools.

tori
Télécharger la présentation

Forecast text editing rates and analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jaime Clark and Bianca Gibson Brianna Laugher Tennessee Leeuwenburg December 2012 Forecast text editing rates and analysis www.cawcr.gov.au

  2. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Graphical Forecast Editor

  3. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Text formatters

  4. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Text formatters - editing

  5. GFE Forecasting Guidance GridEditing & Smart Tools Text Formatting Forecast Production The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology

  6. GFE Forecasting Text Formatting Forecast Production Grids original editedtext gfemetrics The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology

  7. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics

  8. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics

  9. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics

  10. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology GFE Metrics http://tardis.bom.gov.au/gfemetrics Web front-end to “FDB” database created/maintained by ITB Initially developed by Brianna, further developed by Anthony Briggs STATS/SUMMARY Can view graphs/tables of total edits: by time, by state, by formatter VIEW Can view any edits made to any GFE forecasts Can query edits by state, date, formatter, period, issuance, locale Can search for edited/unedited/either containing particular words TOOLS Can view graphs/tables of most used tools: by time, by state

  11. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Metric? We initially hoped that improvements to the formatters would be reflected in lower edit rates... this appears not to be the case... but it's hard to tell...

  12. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Dimensions of edits <—————— REPRESENTATIVENESS ——————> Is the generated text consistent with grid contents? “Wrong” results (genuine error) vs stylistic changes (subjective) “Possible shower” -> “Mostly dry” <—————— POLICY ——————> Is the edited text consistent with Services policy? Policy may need resolving with Services “Fine”, “moderating”, “freshening” <—————— MISSING DATA ——————> Is the info required to produce the edited text present in the GFE? Formatter missing rules - “Humid”, “Hot”, sea breeze GFE missing grids – climatology, uncertainty! Other info in forecaster brain (local expertise) Should fixASAP Can't fix(for everyone) Won't fix Can potentially fix

  13. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Limitations Formatters are mainly just a group of phrases – Sky, Wx, Wind, Temp, Seas, Swell. Effort is directed at improving phrases – but there's no way to check edit rates of phrases Hard to get a sense of any general trend when a query returns hundreds of results Little/no time to devote to exploring regional differences

  14. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Monash Talented Students program Science-Industry Student Research Placement Program “short-term, research-based industry placements”Rationale (i) facilitating industry experience for upper undergraduate students; (ii) building collaborations in work integrated learning between Monash and industry partners; (iii) furthering science and its broader communication.

  15. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology About Bianca Second year in undergraduate Computer Science

  16. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Analysing edits by formatter Per forecast period Per state Percentages

  17. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Marine edits by forecast period - percentage

  18. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Marine edits by forecast period - totals

  19. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Precis edits by forecast period - percentages

  20. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Town, metro and district edits by forecast period - percentage

  21. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology NSW edits by formatter - percentage

  22. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology NSW edits by formatter - totals

  23. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology SA edits by formatter - percentage

  24. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology TAS edits by formatter - percentage

  25. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology VIC edits by formatter - percentage

  26. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology WA edits by formatter - percentage

  27. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Two-Town comparisons Hypothesis: Bigger towns (capital cities) get more attention than smaller towns Does “the gap” represent editing that would be done if forecasters had infinite time? Or does it represent stylistic editing that could be done without?

  28. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Two-Town comparisons - VIC

  29. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Two-Town comparisons - TAS

  30. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparisons Hypothesis: initial edits very low, increase as forecasters learn to use the GFE and have more time for text editing “Month 0” is the month the GFE went live in that region VIC: 5/2009 NSW: 9/2010 TAS: 6/2011 SA: 11/2011 WA: 10/2012 Each state has a different version of the GFE and different weather conditions The dot and percentage on the far right represents edits in 11/2012

  31. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison – all formatters (incl. warning)

  32. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison - Precis

  33. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison - Metropolitan

  34. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Go-Live comparison - marine

  35. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology About Jaime 4th year in double undergraduate degree: Science Major - Pure Mathematics Minor - Developmental Psychology Minor – Computational Mathematics Computer Science Major - Intelligent Systems Major – Software Engineering 5 year course

  36. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Edits per formatter by phrase type edited How do we do this? Formatter Unedited forecast Edited forecast Break into sentences Examining key known words within the sentence to determine type Identify the sentence type Unedited categorized sentences Edited categorized sentences Compare categories: Identify if a change is present

  37. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Town 01-NOV-2012 --- 30-NOV-2012 :: TOWN_FORECAST

  38. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - District 01-NOV-2012 --- 30-NOV-2012 :: DISTRICT_FORECAST

  39. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Metropolitan 01-NOV-2012 --- 30-NOV-2012 :: METROPOLITAN_FORECAST

  40. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Coastal 01-NOV-2012 --- 30-NOV-2012 :: COASTAL_FORECAST

  41. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited – Local Waters 01-NOV-2012 --- 30-NOV-2012 :: LOCAL_WATERS_FORECAST

  42. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Phrase type edited - Marine 01-NOV-2012 --- 30-NOV-2012 :: MARINE_FORECAST

  43. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Precis: Investigating additions/removals How do we do this? Formatter Unedited forecast Edited forecast Identify if the given key is present Unedited keys identified Edited keys identified Compare keys: Identify if a key has been added/removed

  44. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Ratio to un-edited Text

  45. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Morning – strongly removed 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT

  46. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Afternoon – strongly added 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT

  47. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Late – strongly added 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT

  48. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Developing – strongly removed 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT

  49. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Clearing – strongly removed 01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT

  50. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Towns - Haze/Humid? 01-FEB-2012 --- 29-FEB-2012 :: PRECIS_TEXT The forecaster trick to get humid into the forecast: • Add haze into the grids. • Generate forecasts. • Remove haze from the grids. • Complete a search and replace all: haze to humid.

More Related