1 / 14

Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center

Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center. Ellyn Montgomery USGS NERACOOS/NECOSP Data Management Workshop, Sept. 26, 2012. USGS Coastal and Marine Geology Program.

sagira
Télécharger la présentation

Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center Ellyn Montgomery USGS NERACOOS/NECOSP Data Management Workshop, Sept. 26, 2012

  2. USGS Coastal and Marine Geology Program • This presentation applies only to Oceanographic measurements acquired by Woods Hole researchers and colleagues to address scientific questions • The data is mostly used for validation of sediment transport and coastal process models • Management of the data is part of the larger Data Life-cycle that includes everything from planning to archival and publication

  3. About our data • Our holdings describe conditions at specific locations for limited durations. Except for a 25+ year program in Massachusetts Bay, we don’t do long term monitoring programs. • Once the data files collected during an experiment are processed and QC’d that data is published (Federal requirement) • Updates are made only if a problem is found in the data to correct that problem

  4. About our data II • The data are stored in netCDFformat files using EPIC conventions • NcML is used to convert to CF conventions • Files are delivered on-line via • file download • OPeNDAP using THREDDS • FGDC metadata for each experiment is being created and harvested to ocean.data.gov • Each file has ISO 19115 metadata (ncISO)

  5. How we manage our data - 1 • Our raw data is included in the WHSC field activity tracking system • We store 3 levels of data on local disks that are backed up nightly, with periodic backups made and stored off-site. These levels are : • Raw data as off-loaded from the instruments • Processed data (and stages leading to it) • Published data • As EPIC • As CF

  6. Publication and access • Our data may be published directly to the website after approval or with an Open File Report that documents the context of the measurements. • We periodically check the website for broken links or non-responsiveness • We add new interfaces as available (for example making KML files to use with GoogleEarth)

  7. Web access

  8. Web access

  9. THREDDS

  10. Metadata • NetCDF allows metadata to be stored as attributes with the data in the file • We use standards and conventions to enable interoperability • We have GCMD DIF, FGDC and ISO19115 metadata in as many catalogs and registries as possible • We enhance the metadata as new standards emerge (added UnidataDataset Discovery terms in 2011)

  11. Data Management Successes • Community Standards development has evolved to enable LOT of great functionality • We are happy with how THREDDS allows remote clients to access data for analysis in real-time, without having to down-load files • Clients like Matlab, IDV, ncBrowse and ncView allow access and visualization • Registering our THREDDS catalogs in larger repositories of ocean measurements enhances discoverability

  12. Data Management Challenges • Getting links to our holdings into the catalogs of the regional observatories • Since our data is historical, it hasn’t really fit the observatory mission, but including links to historical measurements in the area would be useful for enriching potential analyses • Identifying which of the emerging technologies will persist into the future • Finding ways to let potential users know the data is available

  13. Take home ideas • Use standards to enhance discoverability and enable use of community developed tools • Enable web-based access so the users can access what they need without assistance • Tailor the process to what your clients need to accomplish their missions • Adapt your data and services to take advantage of new techniques as they evolve

  14. Web access • The human-friendly interface to our experimental data, (EPIC conventions) is at: http://stellwagen.er.usgs.gov • The computer-friendly (CF conventions) version of our data holdings, served by THREDDS at: http://geoport.whoi.edu/thredds/allts_reg_catalog.html • Contact me at: emontgomery@usgs.gov • (508)457-2356

More Related