1 / 12

GES DISC OPeNDAP Overview

GES DISC OPeNDAP Overview. Aura Data Systems Working Group James Johnson Sept. 27, 2010. OPeNDAP Introduction. OPeNDAP: Open-source Project for a Network Data Access Protocol Evolved from DODS (Distributed Oceanographic Data System)

EllenMixel
Télécharger la présentation

GES DISC OPeNDAP Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GES DISC OPeNDAP Overview Aura Data Systems Working Group James Johnson Sept. 27, 2010

  2. OPeNDAP Introduction • OPeNDAP: Open-source Project for a Network Data Access Protocol • Evolved from DODS (Distributed Oceanographic Data System) • A framework that simplifies all aspects of scientific data networking • Protocol based on HTTP • Current specification is DAP 2.0; a NASA community standard (see http://www.esdswg.org/spg/rfc/ese-rfc-004) • Standards describe • encapsulating structured data • annotating data with attributes • adding semantics that describe the data • Formats netCDF(3/4), HDF(4/5 + EOS), binary & user defined • Free software (http://opendap.org) Aura Data Systems Working Group

  3. OPeNDAP Basics • OPeNDAP uses the client-server model • Server (Hyrax 4.0) consists of a set of CGI scripts specific to the format of the dataset it serves • Server fetches a subset of data from the dataset, converts it to a binary format, packages it with some descriptive information, and sends it to the client. • Client sends a data request to a server • Request is via HTTP using an enhanced URL • Client translates the data sent by the server into the specified data format used by the application Aura Data Systems Working Group

  4. OPeNDAP enhanced URL Parts of an OPeNDAP URL (without a constraint expression) >dncview http://dods.gso.uri.edu/cgi-bin/nph-nc/data/fnoc1.nc.das ^ ^ ^ ^ ^ ^ ^ | | | | | | | Program | | | | | | Protocol-- | | | | | Machine Name----- | | | | Server------------------------------------ | | | Directory---------------------------------------- | | Filename---------------------------------------------- | URL Suffix----------------------------------------------------| Aura Data Systems Working Group

  5. OPeNDAP URL constraint • A constraint expression is appended to the target URL following a question mark, as in the following examples: • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?temp • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?temp[1,100,5] • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?u&lat>15.0 • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?cast.02<15.0 • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?station&station.temp<15.0 • A constraint expression consists of two parts: a projection and a selection, separated by an ampersand (&). Either part may contain several sub-expressions. Either part may be present, or both. proj_1,proj_2,...,proj_n&sel_1&sel_2&...&sel_m Aura Data Systems Working Group

  6. OPeNDAP Example Clients • Below is a partial list of OPeNDAP Clients: • IDL • Matlab • GrADS • Ferret • IDV • Panoply • CDAT • Pydods • ncview • Generic Web Browser Aura Data Systems Working Group

  7. OPeNDAP GES DISC Datasets • GES DISC serves data following through OPeNDAP: • OMI Level 3 (v3 OMAEROe, OMDOAO3e, OMNO2e, OMTO3e, OMAERUVd, OMTO3d, OMUVBd, OMLER) • OMI Level 2G (v3 OMAEROG, OMAERUVG, OMCLDO2G, OMCLDRRG, OMDOAO3G, OMHCHOG, OMNO2G, OMSO2G, OMTO3G, OMUVBG) • MLS Level 2 (v2 & v1 standard products) • HIRDLS Level 2 • AIRS Level 3 (v5 daily, 8-day, monthly) • MODIS Aqua & Terra Level 3 (v5.1 MYD08, MOD08) • TOMS Level 3 (v8 Nimbus7, Meteor3 and EP) Aura Data Systems Working Group

  8. OPeNDAP with Web Browser List of MLS data files, with response links Goto http://acdisc.sci.gsfc.nasa.gov/opendap/ Then click on HDF-EOS5, Aura_MLS_Level2, product ( ML2H2O.002) and year (2010) dds response lists variables with type and sizes das also includes the attributes ddx is like das but in XML info is in a friendly text layout html is the web interface to subset data Aura Data Systems Working Group

  9. OPeNDAP with Web Browser (cont.) Select variables and optionally subset by dimension index. Then get data via ASCII, NetCDF(3) or Binary (DAP) format URL = http://acdisc.sci.gsfc.nasa.gov/opendap /HDF-EOS5/Aura_MLS_Level2/ML2H2O.002/2010/ MLS-Aura_L2GP-H2O_v02-23-c01_2010d263.he5.ascii? L2gpPrecision[1329:1:1329][6:1:39],L2gpValue[1329:1:1329][6:1:39], lat[1329:1:1329],lon[1329:1:1329],lev[6:1:39] Aura Data Systems Working Group

  10. OPeNDAP IDL client IDL> dlm_register, '/usr/local/lib/libidl_opendap.dlm' IDL> dlm_load, 'opendap‘ IDL> url = 'http://acdisc.sci.gsfc.nasa.gov/opendap/HDF-EOS5/Aura_MLS_Level2/ML2CLO.002/2010/MLS-Aura_L2GP-ClO_v02-23-c01_2010d263.he5’ IDL> status = opendap_get(url, das, mode=‘das’) IDL> help, /structure, das ** Structure <8f288b4>, 16 tags, length=708, data length=708, refs=1: NC_GLOBAL STRUCT -> <Anonymous> Array[1] CONVERGENCE STRUCT -> <Anonymous> Array[1] L2GPPRECISION STRUCT -> <Anonymous> Array[1] L2GPVALUE STRUCT -> <Anonymous> Array[1] QUALITY STRUCT -> <Anonymous> Array[1] STATUS STRUCT -> <Anonymous> Array[1] CHUNKNUMBER STRUCT -> <Anonymous> Array[1] LAT STRUCT -> <Anonymous> Array[1] LINEOFSIGHTANGLE STRUCT -> <Anonymous> Array[1] LOCALSOLARTIME STRUCT -> <Anonymous> Array[1] LON STRUCT -> <Anonymous> Array[1] ORBITGEODETICANGLE STRUCT -> <Anonymous> Array[1] LEV STRUCT -> <Anonymous> Array[1] SOLARZENITHANGLE STRUCT -> <Anonymous> Array[1] TIME STRUCT -> <Anonymous> Array[1] DODS__HDFEOS_INFORMATION_COREMETADATA_0 STRUCT -> <Anonymous> Array[1] IDL> help, /structure, das.l2gpvalue ** Structure <91d8164>, 6 tags, length=56, data length=56, refs=2: _FILLVALUE FLOAT -999.990 TITLE STRING ‘H2O' UNITS STRING 'vmr' MISSING_VALUE FLOAT -999.990 UNIQUEFIELDDEFINITION STRING ‘HIRDLS-MLS-TES-Shared' COORDINATES STRING 'lat lev‘ Aura Data Systems Working Group

  11. OPeNDAP IDL client (cont.) IDL> constraint = 'L2gpValue[1329][6:39],L2gpPrecision[1329][6:39],lat[1329],lon[1329],lev[6:39]’ IDL> stat = opendap_get(url, data, ce=constraint) IDL> help, /structure, data ** Structure <9248a9c>, 5 tags, length=296, data length=296, refs=1: L2GPPRECISION STRUCT -> <Anonymous> Array[1] L2GPVALUE STRUCT -> <Anonymous> Array[1] LAT STRUCT -> <Anonymous> Array[1] LON STRUCT -> <Anonymous> Array[1] LEV STRUCT -> <Anonymous> Array[1] IDL> help, /structure, data.l2gpvalue ** Structure <92156f4>, 2 tags, length=92, data length=92, refs=2: L2GPVALUE FLOAT Array[35] DIMENSION_NAMES STRING Array[2] IDL> print,data.l2gpvalue.dimension_names NLEVELS NTIMES IDL> print, data.lat.lat, data.lon.lon, data.lev.lev 40.0019 -103.995 316.228 261.016 215.443 177.828 146.780 121.153 100.000 82.5404 68.1292 56.2341 46.4159 38.3119 31.6228 26.1016 21.5443 14.6780 10.0000 6.81292 4.64159 3.16228 2.15443 1.46780 1.00000 0.681292 0.464159 0.316228 0.215443 0.146780 0.100000 0.0464159 0.0215443 0.0100000 0.00464159 0.00215443 IDL> plot_io, data.l2gpvalue.l2gpvalue*1e6, data.lev.lev, xrange=[1,100], yrange = [1000,0.01], $ title = 'ML2H2O.002 2010 day 263 at lat '+strtrim(string(data.lat.lat),2)+' lon '+strtrim(string(data.lon.lon),2), $ xtitle = ‘H2O [ppmv]', ytitle = 'Pressure [hPa]' Aura Data Systems Working Group

  12. Panoply Client Example Aura Data Systems Working Group

More Related