1 / 24

In Situ Data, ncBrowse and OPeNDAP

In Situ Data, ncBrowse and OPeNDAP. Presented by Donald Denbo. ncBrowse: Donald Denbo, John Osborne Climate Data Portal: Donald Denbo, Jason Fabritz, Bernard Kilonsky, John Osborne, Charles Sun, Willa Zhu, Nancy Soreide.

orvala
Télécharger la présentation

In Situ Data, ncBrowse and OPeNDAP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. In Situ Data, ncBrowse and OPeNDAP Presented byDonald Denbo ncBrowse: Donald Denbo, John Osborne Climate Data Portal: Donald Denbo, Jason Fabritz, Bernard Kilonsky, John Osborne, Charles Sun, Willa Zhu, Nancy Soreide http://www.epic.noaa.gov/cdp & http://www.epic.noaa.gov/java/ncBrowse

  2. Outline • Accessing distributed in situ data • How Climate Data Portal does it • Basic client - server messages • ncBrowse - OPeNDAP • netCDF conventions • Attribute - variable mapping. • Recommendations • Develop OPeNDAP conventions framework • Develop in situ Java-DODS API that implements in situ convention. OPeNDAP Technical Working Conference

  3. Climate Data Portalaccessing distributed in situ data • Links data from several NOAA ocean observing systems • Data access is accomplished in 3 steps. • LDAP Directory Service is used to find relevant servers. • Coarse Latitude/Longitude/Depth/Time selection • Refine selection of individual stations using ndEdit • Selected data can be downloaded and saved locally or displayed. • Client-Server communications are specified using CORBA IDL, defining both server and client APIs. OPeNDAP Technical Working Conference

  4. LDAP Directory Service • Search on geographic region, observed quantities or keyword • Automatically populated from data servers. • Metadata instantly known to LDAP directories. • Access control for experimental data products. OPeNDAP Technical Working Conference

  5. DatasetSelection OPeNDAP Technical Working Conference

  6. Coarse Selection - lat, long, time box • Coarse selection in lat/long/time box • Important to reduce the number of stations returned from very large datasets. OPeNDAP Technical Working Conference

  7. Fine Selection with ndEdit– view location in time band • Interactive, linked windows • View any 2D slice • xy, xt, xz, yt, yz, yt • Investigate space-time distribution • Select data • in any View Select time rangeSee datain range X-Y map view Y-Ttime range X-Y map view Y-Ttime range OPeNDAP Technical Working Conference

  8. Fine Selection with ndEdit – select along an isobath Section Tool - select stations in a time band- along a depth contour OPeNDAP Technical Working Conference

  9. Fine Selection with ndEdit – select along an isobath Selected stations (in a narrow time band) OPeNDAP Technical Working Conference

  10. API Requirements to Support Multi-Tiered Data Selection • Determine what datasets are available from a server. This can be accomplished by: • Directory Service. LDAP or THREDDS • Query the server for a dataset list • Select a dataset and given a coarse selection get a list of stations and appropriate metadata. • Retrieve stations from a list of selected stations. OPeNDAP Technical Working Conference

  11. Idealized messages between components

  12. ncBrowse • A general purpose, interactive tool for graphical browsing of data and attributes in netCDF files. • OPeNDAP Data Access • OPeNDAP datasets compatible with the Java-netCDF API. • OPeNDAP datasets with more complex data models than netCDF supports, while challenging, is currently in development • Significantly enhancement with 3-D visulaizations using the VisAD library. OPeNDAP Technical Working Conference

  13. ncBrowseOPeNDAP Issues • Client software most frequently use the netCDF API interface to OPeNDAP, thus minimizing the software modifications necessary. • The netCDF API cannot directly and/or transparently access more sophisticated OPeNDAP data types such as Structure and Sequence. OPeNDAP Technical Working Conference

  14. ncBrowseOPeNDAP Goals • Provide ncBrowse users access to the growing collection of datasets available via OPeNDAP. • Access the dataset descriptor (DDS) and data attribute (DAS) structures. • Combine the DAS and DDS to provide easy access to variables and their metadata. • Provide an easy-to-use interface to the OPeNDAP Sequence and Structure objects. OPeNDAP Technical Working Conference

  15. Combined DDS DAS

  16. OPeNDAP - ExampleConnection to “CHRIS” - wind stress OPeNDAP Technical Working Conference

  17. OPeNDAP - ExampleWind Stress from NOAAs Buoys OPeNDAP Technical Working Conference

  18. OPeNDAP - ExampleNSCAT - Mean Wind Field OPeNDAP Technical Working Conference

  19. ncBrowseissues • Combining DAS and DDS difficult because: • DAS and DDS structures don’t match • DAS and DDS namespaces are not always compatible • Not always possible to match attributes to variables with an automated system. • netCDF conventions • No guaranteed way to determine that netCDF conventions apply. • In Java, data structures for DAS and DDS are substantially different causing a little confusion and awkward programming. OPeNDAP Technical Working Conference

  20. Simplified Class Structure

  21. Recommendations • Develop OPeNDAP conventions framework. • Formalize existing DODS convention for netCDF files. • Develop in situ data access convention. • Develop a Java-DODS in situ API. OPeNDAP Technical Working Conference

  22. Develop Conventions Framework • Create a repository of OPeNDAP conventions • Define a standard attribute to indicate convention(s) being used. • Convention framework should allow: • Multiple conventions in effect at the same time. • Hierarchy of conventions. For example, XXX/time_series and XXX/station. • The netCDF convention facility is a good model for formal conventions. OPeNDAP Technical Working Conference

  23. netCDF Convention • netCDF files have a “standard” method of being expressed in the DAS and DDS. • Unfortunately • There is no “standard” way of determining that you should interpret Grid and Array objects and map DAS attributes to variables using this convention. • Proposal • That current practice be formally documented and a “OPeNDAP/netCDF” convention be created. OPeNDAP Technical Working Conference

  24. In Situ Convention and API • Develop convention that supports accessing in situ data. The convention should provide support for: • Retrieving dataset information. For example, geographic and temporal extent, available variables, etc. • Efficiently accessing station information. (Latitude, longitude, depth, time, etc…) This information can be used for the refinement of a coarse selection. • Standard method of determining a unique station identifier. • Develop Java API that supports and enforces the in situ conventions. DODS-URL would be constructed based on convention, and returned objects would be independent on transport specifics (whether DDX or DDS is used). OPeNDAP Technical Working Conference

More Related