XML Via SOAP & Local Degrib Applications

1. XML Via SOAP & Local Degrib Applications Arthur A. Taylor (arthur.taylor@noaa.gov) http://www.weather.gov/mdl/degrib/ Meteorological Development Laboratory NWS NDFD Technical Workshop 2006November 2, 2006

2. Why Use XML • Existing technology • Extensible Markup Language is widely used (W3C specification) • De Facto standard for internet data exchange • Infrastructure exists (validators, XML Schema, etc.) • Facilitates additional processing • XML tools (XSLT, SAX, DOM) facilitate value added applications • Web application developers familiar with tools • Schema precisely defines a language • Reaches new users

3. Data Broker (UDDI) WSDL WSDL Internet SOAP Request SOAP Request Data Consumer (Client) Data Provider (Server) Data SOAP Response SOAP Response XML on the Move Communicating Process Discovery (UDDI) Description (WSDL/Schema) Messaging (SOAP) Transport (HTTP)

4. The SOAP server • The experimental (becoming operational Dec. 4) XML via ‘Simple Object Access Protocol’ server provided by the NWS does the following: • Once an hour: • Download the NDFD GRIB2 files • Convert the GRIB2 files to data cubes • Upon request from the SOAP server: • PHP code parses the request • degrib_DP probes the data cubes • PHP code reformats it into XML • Note: Sometime after Dec. 4, all formatting will be done by degrib C code.

5. SOAP Server Lessons Learned • “If you build it, they will come”

6. Motivation • Why add DWML capability to degrib? • Alleviate pressure on the XML server • allows whole sale users to use the GRIB2 files to create XML, using their own machines, for as many points and as often as they desire. • Speed up the server • Migrates from a scripted language (PHP) to a compiled language (C) • Provide the source code for the logic behind DWML • (c:\ndfd\degrib18\src\dwmllib\xmlparse.c) • Encourage the use of DWML

7. Overview • To create NDFD DWML on a PC we need to • Download the NDFD GRIB2 to our PC • Probe the data at a given location • To enhance the speed of probing, we can also • Convert the data to a data cube • Probe the data cube at a given location

8. Attaining software • To get the degrib program, go to: • http://www.weather.gov/mdl/degrib/register.php • For MS-Windows, use “degrib-install.exe” • By default this installs to c:\ndfd\degrib18 • For UNIX, use “degrib-src.tar.gz” • By default this expand to “/degrib”. To compile it: • cd /degrib/src • make –f makefile.* (* is replaced by your operating system)

9. Get Data • Option 1: Use “tkdegrib” or “tcldegrib web.tcl” • Advantage: In addition to downloading the data, it checks for data corruption, by performing an inventory. From a command Prompt: • cd c:\ndfd\degrib18\bin • tcldegrib web.tcl –dataSet ndfd –sector midatlan –day d17 –variable all • Option 2: Use your own “URL get” program • To determine the locations of the NDFD GRIB2 files: • http://www.weather.gov/mdl/degrib/dataloc.php • http://weather.noaa.gov/pub/SL.us008001/ST.opnl/DF.gr2/DC.ndfd/AR.conus/

10. Probe Data • For more details on probe options, see: • http://www.weather.gov/mdl/degrib/tutorial.php#outputXML • http://www.weather.gov/mdl/degrib/txtview.php?file=degrib.txt&dir=base • From a command prompt • cd c:\ndfd\degrib18\data\ndfd\midatlan • ..\..\..\bin\degrib . –P –pnt 38.98,-77.07 –XML 1 –geoData c:\ndfd\degrib18\geodata • The –geoData option supplies a directory to look in for information about the point’s timezone • For multiple points, either provide multiple –pnt options, or use the “-pntFile” option

11. Flavors of DWML • -XML 1 (or time-series) creates the time series product • User can choose to format any combination of the following: maxt, mint, pop12, temp, wdir, wspd, td, sky, qpf, snow, wx, waveh, apt, rhm • -XML 1 -Icon 1 will return just the icon information. • Requires: temp, wspd, sky, wx • Formats: icons • -XML 2 (or glance) is the glance product • Requires: maxt, mint, temp, wspd, sky, wx • Formats: maxt, mint, sky, wx, icons • -XML 3 (or 12-hourly) is the 12 hour summary product • Requires: maxt, mint, pop12, wdir, wspd, sky, wx • Formats: maxt, mint, pop12, wx, icons • -XML 4 (or 24-hourly) is the 24 hour summary product • Requires: maxt, mint, pop12, wdir, wspd, sky, wx • Formats: maxt, mint, pop12, wx, icons

12. Convert to data cube • The data cube consists of uncompressed data, which allows faster access than the GRIB2 files • Method 1 (use c:\ndfd\degrib18\bin\MakeData.tcl): • cd c:\ndfd\degrib18\bin • tcldegrib MakeData.tcl c:\ndfd\degrib18\data\ndfd\midatlan c:\ndfd\degrib18\cube\ndfd\midatlan • cd ..\cube\ndfd\midatlan • Method 2: Manual method • cd c:\ndfd\degrib18\data\ndfd\midatlan • ..\..\..\bin\degrib maxt.bin –Data –Cube –Index maxt.ind –msg all • Repeat for all GRIB (*.bin) files.

13. Probe the data cube • From a command prompt either: • ..\..\..\bin\degrib . –DP –pnt 38.98,-77.07 –XML 1 –geoData c:\ndfd\degrib18\geodata • ..\..\..\bin\degrib_DP . –DP –pnt 38.98,-77.07 –XML 1 –geoData c:\ndfd\degrib18\geodata • -DP is for probing a data cube, while -P is for a GRIB file • degrib_DP is compiled with just the data cube probing code so it is smaller than degrib

14. Credits • I’d like to acknowledge: • John Schattel for developing DWML • Paul Hershberg for converting the PHP code to C.