1 / 22

The NERC DataGrid

The NERC DataGrid. Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC etc) Representing the NDG and a dose of the “Portals Project”. +. +. =. +. +. NOCS, PML, STFC, BODC, BADC. Simulations. Assimilation.

arne
Télécharger la présentation

The NERC DataGrid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The NERC DataGrid Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC etc) Representing the NDG and a dose of the “Portals Project” + + = + + NOCS, PML, STFC, BODC, BADC

  2. Simulations Assimilation Complexity + Volume + Remote Access = Grid Challenge British Atmospheric Data Centre British Oceanographic Data Centre http://ndg.nerc.ac.uk

  3. Ground based observation networks Met Office surface stations Model output NWP, ECMWF reanalyses & Climate models Satellite data TOMS, Envisat & MSG NERC programmes data UTLS, CWVC & URGENT Data Sets “A collection of files with a common theme and administration” Datasets statistics • 147 datasets • ~130TB (unique) • ~80 Million files

  4. Find data Find out about data Subset and difference data Visualise data All using SECURE interdisciplinary technology. Sound easy doesn’t it? Want interdisciplinary semantic access to information, not abstract data getData(potential temperature from ERA-40 dataset in North Atlantic from 1990 to 2000) not: getData(“era40.nc”, ‘PTMP’, 20:50, 300:340, 190:200) or even worse: for j=1990:2000 getData(“era40_”+j+“.nc”, ‘PTMP’, 20:50, 300:340) The NDG Use Cases

  5. The big: Climate in 2010 – A graphic Illustration Figures from Gary Strand, NCAR, ESG website

  6. … and the little … land use!

  7. ISO19109 in a nutshell

  8. Dataset Metadata

  9. MOVIE(s)

  10. NDG Deployment Vocab Server: 120,000+ concepts 100 lists 75,000 Relationships (SKOS)

  11. Dataset Metadata

  12. MOLES: Metadata Objects for Linking Environmental Science A data production tool deployed at an observation station on behalf of an activity produces a dataset. Datasets have provenance

  13. A-Metdata We choose to deploy an “Application Schema” of the “Geographic Markup Language”. This allows us to define our data to support RE-USE by other communities – interdisciplinarity, and to be INSPIRE compliant! Getting down to the data itself

  14. CSML Dataset • Dictionaries: • coordinate reference systems, • phenomena (parameters) • units of measure • Feature Collections • CSML Features are SPECIALISATIONS of GML Features! • Storage Descriptor • Re-usable description of storage (NB: CSML Software Stack, WMS, WCS etc)

  15. CSML Feature Types Abstract Specific: point data, time-series of point data, trajectory data, point collections, profiles, profile series, Ragged profile series, section, ragged section, scanning radar, grid, gridseries and swath

  16. Beyond measuring things at points CF concepts: Cell-Methods (statistical e.g means etc) Cell-Bounds (intensive quantities, Extensive quantities, Relationship to coverage domain) New work: Exploiting Sensor Web Enablement

  17. Storage Descriptor

  18. April 2007, at The Cosenors House, Abingdon Brought NDG, Unidata, OGC, Italian NRC together to get a shared vision for scientific data types. Roadmap for bringing our activities together over next few years. Harmonising: NDG CSML OGC Observations and Measurements (with CSML and MOLES) UNIDATA Community Data Model INRC NCML-G Data Modelling Workshop

  19. Data Model Relationships Route to harmonising GeoSciML and CSML

  20. The Original Vision

  21. What we’ve done (are doing) Schema: MOLES CSML DIF + Lots of Population Issues Pylons Interface & portals Protocols: OGC +OAI Pylons based portal Python WCS/WMS Clients CSML Toolbox Vocab Server

  22. Now and the Future More protocol improvements (= wider community of providers and users) More deployment. More population. Evolution WITH the web.

More Related