1 / 15

Metadata Requirements for EOSDIS Data Providers

Metadata Requirements for EOSDIS Data Providers. Siri Jodha Singh Khalsa khalsa@colorado.edu. HDF-EOS Workshop II. Topics. Why metadata is important Types of metadata in HDF-EOS files Required metadata How metadata is encoded and delivered. HDF-EOS Workshop II. What is Metadata?.

keena
Télécharger la présentation

Metadata Requirements for EOSDIS Data Providers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Metadata Requirements for EOSDIS Data Providers Siri Jodha Singh Khalsa khalsa@colorado.edu HDF-EOS Workshop II

  2. Topics • Why metadata is important • Types of metadata in HDF-EOS files • Required metadata • How metadata is encoded and delivered HDF-EOS Workshop II

  3. What is Metadata? • Metadata is information that identifies and characterizes an information product. • Sometimes called “data about data” HDF-EOS Workshop II

  4. Users Need Metadata • Metadata is needed to answer questions such as: • What time and location does this data apply to? • Why type of instrument and processing produced the data? • What other inputs were used to generate the data? • What QA has been performed on this data? • Who do I contact if I have questions about this data? HDF-EOS Workshop II

  5. Metadata is Essential • Large data archive systems cannot function without metadata. • Metadata is used to keep track of such things as: • where the data is • what type of operations are possible on the data • whether there are any access restrictions on the data • how individual data files are logically grouped into “collections.” HDF-EOS Workshop II

  6. Key Concepts • A granule is the smallest aggregation of data that is independently described and inventoried by the ECS. A granule consists of 1 or more physical files. • A collection is a logical grouping of granules. • The ECS Data Model allows for: • “Core” attributes • “Product-Specific” Attributes (PSAs)

  7. Types of Metadata • Metadata in HDF files • stored as global text attributes • Types of Metadata used in HDF-EOS files: • Structural Metadata • Core Metadata (inventory, can include PSAs) • Archive Metadata (non-searchable, product-specific) • Collection level metadata • core and product-specific HDF-EOS Workshop II

  8. Required Metadata • Origins of metadata requirements: • what is required to archive and retrieve files • what is required to provide search and other services on data • what is federally mandated (FGDC) • There are 287 attributes in the ECS data model • only a subset are used for any given product • 101 are applicable at the granule level HDF-EOS Workshop II

  9. Metadata Coverage • Science Data that are delivered for archiving in ECS must meet what is called the Intermediate level of metadata coverage. This involves as few as: • 31 collection level attributes • 4 granule level attributes • Compliance at this level is not enforced by the system. HDF-EOS Workshop II

  10. Collection-Level Metadata for Intermediate Coverage • SpatialCoverageType • PointLatitude • PointLongitude • TimeType • DateType • TemporalRangeType • PrecisionofSeconds • EndsatPresentFlag • CalendarDate • TimeofDay • GuideName • DataCenter • DocumentVersion • DocumentUpdated • Title • DocumentCreated • ShortName • LongName • CollectionDescription • VersionID • ArchiveCenter • RevisionDate • VersionDescription • CollectionState • MaintenanceandUpdateFrequency • ECSDisciplineKeyword • ECSTopicKeyword • ECSTermKeyword • ECSVariableKeyword • ContactOrganizationName • Role HDF-EOS Workshop II

  11. Granule-Level Metadata for Intermediate Coverage • There are only four granule-level metadata attributes required: • ShortName • VersionID • SizeMBECSDataGranule • ProductionDateTime • ShortName and VersionID are identical to the collection-level attributes with these names. • For granules coming into ECS, SizeMBECSDataGranule and ProductionDateTime are supplied by the system upon insertion. HDF-EOS Workshop II

  12. How is Metadata Supplied? • Collection-level metadata is carried in an Earth Science Data Type (ESDT) Descriptor file. • Granule-level metadata is defined in the descriptor file and populated using a Metadata Configuration File (MCF). • Granule-level metadata is delivered in the HDF-EOS granule *or* in a populated MCF accompanying a non-HDF granule. • The DAAC where a collection will reside is responsible for descriptors and ingest routines. HDF-EOS Workshop II

  13. Metadata Work Flow for External Data Providers HDF-EOS Workshop II

  14. Metadata Resources on the Web • ECS Metadata Homepage http://ecsinfo.hitc.com/metadata/metadata.html • Metadata Works (ESDT Descriptor Tool) http://et3ws1.HITC.COM/metadata_works/ • EOSDIS Information Architecture http://spsosun.gsfc.nasa.gov/InfoArch.html • Federal Geographic Data Committee http://www.fgdc.gov/

  15. Q&A w/ Experts Panel • Q: “If you are a new data provider, how do you get your data into an HDF-EOS granule, given the bewildering array of utilities and tools available? What is the simplest solution for this?” • A: The recommended solution is to obtain the HCR package, which includes the HDF-EOS and HDF libraries. For populating the required metadata in the granule, obtain the Metadata/Time Toolkit_MDT. The steps would be: 1. Write an HCR and use the tools to turn this into a skeletal HDF-EOS granule. (This step is optional). 2. Use the HDF-EOS library to create a granule. (If starting with a skeletal HDF-EOS file generated from an HCR then plain HDF calls can be used to insert data into the granule ). 3. Use Toolkit_MDT calls to insert metadata into the granule. This requires generation of an MCF in ODL. Metadata_Works is available for doing this. As an alternative, a simple HDF call can be used to attach minimum metadata (in ODL) to an HDF file. Note: if the data are going to reside in a DAAC, or in an archive that must be interoperable with ECS, you will need to generate collection-level metadata. Metadata_Works is the recommended tool for this.

More Related