1 / 35

Update on the Collaborative Radar Acquisition Field Test (CRAFT): Planning for the Next Steps

Update on the Collaborative Radar Acquisition Field Test (CRAFT): Planning for the Next Steps. Kelvin K. Droegemeier University of Oklahoma Presented to NEXRAD Technical Advisory Committee 3 December 2001. NCDC. The Collaborative Radar Acquisition Field Test (CRAFT).

perdy
Télécharger la présentation

Update on the Collaborative Radar Acquisition Field Test (CRAFT): Planning for the Next Steps

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Update on the Collaborative Radar Acquisition Field Test (CRAFT): Planning for the Next Steps Kelvin K. DroegemeierUniversity of OklahomaPresented toNEXRAD Technical Advisory Committee3 December 2001 NCDC

  2. The Collaborative Radar Acquisition Field Test (CRAFT) • A Collaborative project to demonstrate the real time compression and Internet-based transmission of WSR-88D base (Level II) data • Motivated by • Storm-scale NWP research at CAPS (now for WRF) • Decision support system development (NSSL) • Desire to improve long-term archive rate (NCDC) • Philosophy: Leverage existing infrastructures (Internet), public-domain software (Unidata LDM), and low-cost services (56K phone lines)

  3. Participants • CRAFT was established by the OU Center for Analysis and Prediction of Storms (CAPS) in collaboration with the • UCAR Unidata Program Office • NOAA WSR-88D Operational Support Facility (now Radar Operations Center) • University of Washington • National Severe Storms Laboratory • Now coordinated by CAPS, NSSL, ROC, Unidata, and NCDC

  4. Current Technical Strategy Optional Router ($2000) Linux PC + LDM + Compression ($1000) At the radar site Dedicated 56K line or ($2000 - $6000/year) or NWS LAN LDM Server WSR-88D Internet or Abilene RIDDS Users Repeater Hub

  5. Technical Highlights • BZIP2 off-the-shelf compression algorithm • maximum 16:1 in clear air; 8:1 in the worst case; average of about 12:1 • 56K lines found to be inadequate in some cases (squall lines) – maybe 10% of time • Exceed available bandwidth by 10-20% • Can create latencies of 10s of minutes • Up to 26% improvement in compression possible in BZIP2, but requires changing code at user end • A pre-processing algorithm (NSSL) provides 10% improvement – no changes in BZIP required • 56K line overrun may not be an issue – used for initial testing because of cost and security • New scan strategies and availability of cheaper, faster links (even cable modems) may render 56K lines moot

  6. Radars Now Delivering Real Time Level II Data to NCDC/OU Seattle Denver Norfolk Cedar City Las Vegas Raleigh Tulsa Morehead City Flagstaff Huntsville Amarillo Oklahoma City Wilmington Fort Smith Columbia Phoenix Yuma Charleston Lubbock Tucson Dallas/Fort Worth 21

  7. Old WSR-88D Level II Data Processing at NCDC Courtesy Steve DelGreco (NCDC)

  8. At NCDC Today HDSS LDM Processing/Recompression Abilene T3 Mass Storage Compressed Level II Data From 21 Radars WSR-88D via FTP Modified from Steve DelGreco (NCDC)

  9. WSR-88D Level II Archival at NCDC % Rcpts #Sites Level II NWS 64.9 120 Level II DoD 35.7 26 Level II FAA 9.5 12 Total 88D Network55.7158

  10. WSR-88D Level II Archival at NCDC % Rcpts #Sites Level II NWS 64.9 120 Level II DoD 35.7 26 Level II FAA 9.5 12 Total 88D Network55.7158 CRAFT EFFICIENCY IS ABOVE 95%

  11. A Notable NCDC Success Story • University of Iowa requested 19.4 gigabytes (uncompressed) of Level II data • The NCDC was able to retrieve it from the HDSS, compress it, and stage it to FTP in 2 hours and 5 minutes • Data was then moved over the Internet in a matter of minutes • The old 8 mm tape system would have required • 58 tapes • 9 workstations each running for 26 hours • A total of 232 hours • Improvement is more than a factor of 100 in time, not including personnel costs

  12. PreliminaryLatency Data

  13. Networking R&D • ESDIM is funding • Analysis of latency, bandwidth, etc • Network simulation (reliability, redundancy) • Additional analysis being conducted in collaboration with the private sector

  14. Additions by End of CY 2001 21+ 11 Seattle Rapid City Detroit Buffalo Cleveland Chicago Cheyenne N. Indiana Pittsburgh Denver Cincinnati Goodland Norfolk Cedar City Las Vegas Raleigh Tulsa Morehead City Flagstaff Huntsville Amarillo Oklahoma City Wilmington Fort Smith Columbia Phoenix Yuma Charleston Lubbock Tucson Dallas/Fort Worth Melbourne

  15. Additions by Spring, 2002 21+ 11 + 6 Seattle Rapid City Detroit Buffalo Cleveland Chicago Cheyenne N. Indiana Pittsburgh Denver Cincinnati Goodland Kansas City Norfolk Cedar City Dodge City Wichita Springfield Las Vegas Raleigh Tulsa Morehead City Flagstaff Huntsville Amarillo Oklahoma City Wilmington Fort Smith Columbia Phoenix Yuma Charleston Little Rock Lubbock Tucson Dallas/Fort Worth Shreveport IHOP Field Program Plus CAPS FAAResearch Melbourne

  16. Military Radars 21+11+6+2 Seattle Rapid City Detroit Buffalo Cleveland Chicago Cheyenne N. Indiana Pittsburgh Denver Cincinnati Goodland Kansas City Norfolk Cedar City Dodge City Wichita Springfield Las Vegas Raleigh Enid Tulsa Morehead City Flagstaff Huntsville Amarillo Oklahoma City Altus Wilmington Fort Smith Columbia Phoenix Yuma Charleston Little Rock Lubbock Tucson Dallas/Fort Worth Shreveport Melbourne

  17. FAA-MIT/LL Additions FY 2002 40+14=54 Seattle Rapid City Albany Boston Detroit Buffalo Binghamton Milwaukee New York City Cleveland Chicago Cheyenne Central PA N. Indiana Pittsburgh Philadelphia Quad Cities Central Ill Baltimore Denver Cincinnati Goodland Kansas City Charleston Norfolk Cedar City St. Louis Dodge City Louisville Wichita Springfield Paducah Las Vegas Raleigh Enid Tulsa Morehead City Flagstaff Huntsville Amarillo Oklahoma City Altus Wilmington Fort Smith Columbia Phoenix Yuma Charleston Little Rock Lubbock Tucson Dallas/Fort Worth Shreveport Melbourne

  18. Current Technical Strategy Optional Router ($2000) Linux PC + LDM + Compression ($1000) At the radar site Dedicated 56K line or ($2000 - $6000/year) or NWS LAN Server WSR-88D Internet or Abilene RIDDS Users Repeater Hub

  19. Interim BDDS Optional Router ($2000) Linux PC + LDM + Compression ($1000) At the radar site Dedicated 56K lines or or NWS LAN Server WSR-88D Internet or Abilene BDDS Users Repeater Hub

  20. Final BDDS At the radar site Dedicated 56K lines or or NWS LAN Server WSR-88D Internet or Abilene BDDS+LDM/Compression Users Repeater Hub

  21. How Does This Experiment Become an Operational System?

  22. Abilene NetworkJanuary 1999 Abilene Router Node Abilene Access Node Operational January 1999 Planned 1999 Seattle Cleveland New York Sacramento Denver Indianapolis Kansas City Los Angeles Atlanta Houston

  23. One Possible Strategy via phone linesor commodityInternet NOAA Facility Silver Spring

  24. One Possible Strategy via phone linesor commodityInternet Silver Spring Seattle NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility Norman Atlanta NOAA Facility NOAA Facility Miami Boulder

  25. One Possible Strategy via phone linesor commodityInternet Silver Spring Seattle NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility Norman Atlanta NOAA Facility NOAA Facility Miami Boulder “Super Sites”: Each LDM on Abilene “bus” Carries all 88D data -- redundancy

  26. NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility NOAA Facility NOAA Facility

  27. One Possible Strategy Commodity Internet Commodity Internet NOAA Facility NOAA Facility Abilene Network Commodity Internet Commodity Internet NOAA Facility NOAA Facility NOAA Facility NOAA Facility Commodity Internet Commodity Internet

  28. Private Sector Academia NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility NOAA Facility NOAA Facility

  29. Private Sector Private Sector Private Sector Private Sector Private Sector Private Sector Academia Academia Academia Academia Academia Academia NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility NOAA Facility NOAA Facility

  30. Private Sector Private Sector Private Sector Private Sector Private Sector Private Sector Academia Academia Academia Academia Academia Academia NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility NOAA Facility NOAA Facility

  31. Private Sector Private Sector Private Sector Private Sector Private Sector Private Sector Academia Academia Academia Academia Academia Academia NOAA Facility NOAA Facility Abilene Network NOAA Facility NOAA Facility NOAA Facility NOAA Facility

  32. Meeting all of the Needs 88D 88D 88D 88D 88D Abilene-based network of (aggregating & relaying) LDMs, each carrying all 88D data, with redundant interconnections Private 1 Private 3 Academia 1 NOAA 1 NOAA 2 Private 3 Private 4 Academia 2 NOAA 3 Courtesy D. Fulker, UCAR

  33. Meeting all of the Needs 88D 88D 88D 88D 88D Abilene-based network of (aggregating & relaying) LDMs, each carrying all 88D data, with redundant interconnections Private 1 Private 3 Academia 1 NOAA 1 NOAA 2 Private 3 Private 4 Academia 2 NOAA 3 Courtesy D. Fulker, UCAR

  34. Benefits of This Concept • NOAA runs its operational ingest system but links to private sector and academic systems, each providing redundancy for the other • Entire community benefits – builds upon the collaborative spirit of CRAFT • Configuration completely scalable • Leverages existing infrastructure (LDM sites, Abilene, WFO connection to commodity Internet) • Significant capacity for future growth (dual-pol, phased array)

  35. Longer Term: Addition of FAA Radars Weber (2000)

More Related