NOAA NWS CIO Report APSDEU-9 Tokyo, Feb 2009

1. NOAA NWS CIO ReportAPSDEU-9 Tokyo, Feb 2009 • Office of the Chief Information Officer • NOAA’s National Weather Service

2. NWSTG Functional Overview • The NWSTG is: • the central communication facility of the NWS; • the primary acquisition and distribution center for NWS data and products; • the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements; • A major data exchange hub for NOAA and other agency data and products.

3. NWSTG Functional Overview The NWSTG includes • WMO Regional Telecommunication Hub (RTH) Washington • ICAO OPMET Databank • the ASOS Operations and Monitoring Center • Operational oversight of U.S. federal automated surface observing systems • the AWIPS Network Control Facility • AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement • The NCF is the central communications hub and technical support center for AWIPS

4. Data Input to the NWSTG • MPLS • IP / Sockets • X.25 • Asynchronous • FTP – http://weather.gov/tg/ftpingest.html • Email – http://weather.gov/tg/emailingest.html • Web - http://weather.gov/tg/bullguid.html

5. GTS ISCS EMWIN Internet GMDSS NWS Telecom. Gateway NWWS NWR LDAD Family Of Services (FOS) Interagency Connections AWIPS SBN / NOAAPORT Dissemination Systems International National

6. Dissemination and Distribution Other Agencies NESDIS Commercial Specialized Satellite GOES Satellite Legend: Imagery Customers EMWIN NOAAPORT • Commercial Weather Services EMWIN • Research Institutions NWWS • FAA, etc. FOS WMO (> 1 min) ISCS SBN/NOAAPORT Designated GOES NCEP Product Suite Local ICAO Observations Customers Telecommunications Operations Center NWWS NCEP NWSTG (10 sec) Media & Other LDAD Customers NCF WAN Data Servers Private line Internet-Based Field Dissemination Dissemination Offices WSR-88D GTS EMWIN NWR Public

8. Replacement / BackupTG Description NWS users • Nat’l Centers for Environ. Prediction • NWS Regional Offices • Domestic/int’l observation & forecast offices • AWIPS Worldwide users • ICAO / WMO • Govt. agencies • Family of services • Internet users • Foreign countries • Emergency mgrs NWS Telecom Gateway • Located in Silver Spring • New message switching system allows future upgrades • 2x upgradeability • Much improved response time • Redundancy ensures uninterrupted service • Full configuration management 120GB/46 600 GB 80GB/38 200 GB Geographically separated backup system 90GB/46 375 GB 700GB/38 2000 GB Daily Throughput/# of circuits Legacy over Replacement

9. RTH Washington • Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent • Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day • On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS) • GTS delivers tsunami data and warnings to connected MTN centers within two minutes

10. Replacement NWSTG • Full functional replacement of existing capabilities • Expanded capacity and capability • Input data growth from 200 to 800 GB/day • Output data growth from 800 to 2400 GB/day • Transition to new technology • Middleware for internal transport • Network-centric systems interconnectivity • Central switching engine with relational database • SAN and NAS storage solutions • Highly scaleable architecture • Hardware refresh

11. RTG/BTG Performance Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds Routine Message Latency (averaged monthly) 60 seconds Daily Traffic Volume (averaged monthly) 1.2TB NWSTG Performance Results (Oct, 2008) Performance Category Threshold Actual System Availability 99.90% 100% Warning Message Latency 10 seconds < 1 second Routine Message Latency 60 seconds < 1 second Daily Traffic Volume 1.2TB 1.3TB (plus .2TB in last year)

12. Government Networking Requirements - NOAANet • IP-based networking solution • Any-to-any connectivity • High degree of bandwidth scalability • Optimum redundancy and survivability • IP convergence (i.e., voice, video and data over IP) • High-end performance • Network security remains paramount, particularly in light of today’s socio-political threats • Segmentation from the public Internet • Minimizes risk of security or privacy breaches

13. Logical Design

14. Primary / BackupNWSTG Access

16. Major Software Components Core software components of the NWSTG replaced in 2006 • Data switching software • Core software rewritten and implemented in IBM WebSphere Message Broker • Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services • Data transport infrastructure (middleware) • Shared data stores replaced with IBM MQSeries Middleware • Applications send data to each other without concern for target location • Data delivery guaranteed even during failures • Data storage infrastructure (relational database) • All data stores moved into Sybase relational database

17. Internal Redundancy Internal redundancy implemented for critical processes • Active – Warm Standby • Standby system running at all times, system disks moved to standby system during failover • EXAMPLES: Individual front-end processors, application servers • Active – Hot Standby • Data replicated to Hot Standby system in realtime, automatic failover • EXAMPLE: Sybase relational database servers • Active - Active • Multiple systems active in parallel, data shared/routed between systems groups • EXAMPLES – Switching system, HTTP/FTP server groups

18. Architecture Overview

19. Architecture Overview

20. System Extensibility Virtualized server hardware platform selected for extensibility • Virtualization • System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame • Individual resources selected to create virtual servers • Major storage implemented in Storage Area Network • Virtual disks allocated to virtual servers • Extensibility • Unused resources can be allocated to virtual servers, as needed • Underused resources can be removed from a virtual server • Virtual disks can be extended or reduced as storage requirements change

21. System Extensibility Virtualized server hardware platform selected for extensibility • Advanced Virtualization • Network and SAN I/O interfaces can be shared through virtual I/O • CPUs can be carved into fractional components down to 1/10th of a CPU • Advanced Extensibility • Resources (CPU, Memory) included within system that were not purchased • If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available

22. System Extensibility (Currently migrating to P595 systems)

23. RTH Washington WIS approach RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements • Production and Replicated core systems • All functions not essential to mission critical operations (i.e. switching) execute against replicant • SOA Approach to design • Integral file, message and application switching • RDBS core design • ~ 2000 destinations defined • ~ 400K defined products • ~ 10M entries (rows) in switching table • Many tables used for managing bulletin, file and report storage including dissemination and services metadata.

24. RTH Washington WIS approach View Data Access & Retrieval DAR service as the only truly new service provision • Plan to lay this service on to existed architecture • Will break up DAR functions • DAR catalog creation & update • discovery & access servicing • DAR will not execute on primary production system (at least in early stages of implementation

25. RTH Washington WIS approach Develop metadata catalog first in conjunction with national & regional partners • Regional WIGOS Development Project • RA-IV Integrated Atmosphere Observing System • With initial emphasis on RADAR observations • Will leverage knowledge of partner expertise • Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners. • Work will be made available to all Members • Will collaborate with WIS Project Team, WMO ETs and developers in other Regions

26. SFC-LAND Information Bases (QCed Datasets) SFC-MARINE U/A-IN SITU MADIS Collection, QC, and Distribution U/A-REMOTE SENSING Quality Control Information (Data QC Flags) And Metadata SATELLITE GRIDS METADATA Meteorological Assimilation Data Ingest System (MADIS) Transition to Operations PM 12/12/05

27. Meteorological Surface METAR Airways Maritime Modernized NWS Cooperative Observer UrbaNet Integrated Mesonet State DOT Mesonets AWS RSAS Surface Grids NOAA Profiler Network Hydrological Surface Automated Aircraft Multi-Agency Profiler Cooperative Agency Profiler Radiosonde Radiometer Satellite Wind NOAA GOES Products Satellite Radiances and Soundings NOAA POES Snow MADIS Ingests and Performs QC on Meteorological Observational Data

28. Gateway Customers (NCDC, NCF, GTS etc) NOAA Customers TOC Environment ? Storage (SAN) FTP LDM Non-NOAA Observations Gateway ? Individual Data Products Integrated Datasets External ISP Web Server ? MADIS-T Data Processing LDM Web Server LDM Internet FTP Automated QC processing Tunnel

29. Weather Impacts on Society

30. Data, Products, and Services to: Greatly reduce loss of life and injury Enable communities to mitigate property loss well in advance of threatening conditions Alert economic sectorsto environmental riskswith sufficient lead timeto limit or avoid impacts Where We’re Headed

31. Integrated Surface Observing System (ISOS) ASOS Cooperative Observers (COOP) COOP-Modernization National Mesonet Where We’re Headed

32. Climate Services Enhance products and services with climate change information Local outlooks for temperature, precipitation, El Niño/La Niña Seamless Climate, Water, and Weather products and services Where We’re Headed Reduce losses from wildfires Save lives and reduce property damages caused by major climate anomalies including drought Meet new needs with end-to-end suite of climate products and services

33. Water Resource Services Provide high-resolution water quantity, quality, and soil moisture forecasts Emergency and resource managers mitigate losses for conditions ranging from droughts to floods Where We’re Headed Enables NOAA to meet our Nation's growing needs for water forecasts Provides forecasts for consumption resources Provides important resource protection capabilities

34. Ecosystem Impact Information Provide forecasts of weather, water, and climate impacts for management decisions Management decisions reflect relationships among humans, nonhuman species, and the environments in which they live Where We’re Headed Benefits through innovative approaches to spill preparedness, response, damage assessments and restoration NOAA contributes approximately $75 million annual to the U.S. economy.

35. Forecast Uncertainty Information Integral and essential part of all forecasts Enterprise-wide partnership to generate and communicate forecast uncertainty to decision makers and public Expressed in terms of probabilities Where We’re Headed Users decide whether to take action and appropriate level of response Thresholds unique to decision maker – based on mission risk

36. Future Activities – RTH/GISC Washington • Continuing tech refresh • Differences between TG & BTG • Not affecting functional backup • Is a maintenance issue • Next round of Continuity of Operations Planning (COOP) • Backup strategy – hot, warm. cold?

37. Future Activities • GTS – WIS evolution • MTN & GISC core network? • HAZNET VPN • GISC planning and Implementation • WIGOS integral with WIS • Our national strategy is integral with the regional strategy • WDP - proof of concept prior to wider architecture implementation A flexible services oriented framework based on a regular requirements review process responding to and feeding into the strategic planning process, at national, regional and WMO levels

38. Data pull Data push WIS DATA-COMMUNICATIONS FUNCTIONS AND SERVICES How do we serve the People? DAR WIS HAZ-NET VPN GTS IGDDS WIS/GTS: for time and operation-critical data & products WIS/IGDDS: for space-based data & products WIS/DAR: data discovery, access and retrieval

39. Food for thought Services Delivery Data Processing Data Exchange Observations Danger of fracturing or stove piping GEO, WMO & other data sharing initiatives

40. Challenges To peer into the future and have a vision of future needs The volume of observations from satellites, RADARs, mesonets, and non-traditional domains continues to grow well beyond levels at which we are able to keep up with. NWP presents a two fold problem to the data exchange community. Insatiable Hunger - Profuse Output What demands will climate services and multi-domain models bring?

41. Challenges I want to scare you, at least a little bit Our community is struggling to handle these demands We are often so busy planning for today’s needs that future requirements are a best a casual afterthought. As a result we stay behind curve , always playing the catch-up game. We need to embrace new technologies and develop adaptive, agile exchange architectures Services Orientation SCALABILITY EXPANDABILITY EXTENSIBILITY