USGS/EROS WCS Experience and Ideas for Landsat Data Access

1. USGS/EROS WCS Experience and Ideas for Landsat Data Access WGISS #27 May, 2009 Toulouse, France Lyndon R. Oleson U.S. Geological Survey Earth Resources Observation and Science (EROS) Center Sioux Falls, SD

2. EROS WCS Experience and Ideas • Outline • Two Prototypes of OGC WCS • LPDAAC MOD14 - MODIS thermal anomalies and fire global composites • Landsat Prototype • Extrapolation to larger satellite archive collections (e.g. online Landsat archive)? • Ideas for the future

3. EROS WCS Experience and Ideas • Both prototypes used: • The mapserver (http://mapserver.org/) software together with the Geospatial Data Abstraction Library (GDAL) (http://www.gdal.org/). • Version 1.0.0 of the WCS protocol since that version was the most widely supported at the time. Substantial revisions were made to the protocol in version 1.1 and the current version is 1.1.2.

4. EROS WCS Experience and Ideas • LPDAAC MOD14 Prototype • Sponsored by the Land Processes Distributed Active Archive Center (LPDAAC) in conjunction with the NASA Geospatial Interoperability Office (GIO). • MOD14 product • MODIS thermal anomalies and fire • Global composites • Eight day (MOD14A2) and one day (MOD14A1) • The input data is in the form of tiles in HDF-EOS format

5. EROS WCS Experience and Ideas • LPDAAC MOD14 Prototype • WCS Constructs • The mapserver/GDAL software reads HDF-EOS natively and also allows the generation of a tile index • This allows a specific request to be quickly mapped to a set of tiles without the need to read each tile to determine the geographic coverage. • Thus the collection of tiles for each compositing period (eight days for the MOD14A2 and one day for the MOD14A1 collections) was assembled, with a tile index, and treated as a single coverage. • Since this collection of coverages dates to March 2000 it could become rather large, and • since the WCS allows temporal searching, they were aggregated into two coverages, one each for the eight-day (MOD14A2) and the one-day (MOD14A1) collection.

6. EROS WCS Experience and Ideas • LPDAAC MOD14 Prototype • WCS Constructs (cont.) • The WCS TIME parameter is used to find the corresponding collection of tiles for a specific composite period. • For WCS requests without the TIME parameter, the most recent composite period is returned. • The WCS specification is vague about how to handle the case where the TIME parameter does not have an exact match. • For this prototype, the closest available match is returned. • This could be misleading. • For example, a request for data from 1995 would return the first March 2000 data, since that is the earliest available. • A more appropriate response would probably be to throw an exception in that case.

7. EROS WCS Experience and Ideas • LPDAAC MOD14 Prototype • this prototype was successfully demonstrated, using a client developed by DataFed in December 2007 • Although initially a prototype, it continues to function in a production mode and is updated daily as new data becomes available. • In addition to the MODIS Terra (MOD14) data, the corresponding MODIS Aqua data (MYD14) is also available.

8. EROS WCS Experience and Ideas • Landsat Prototype • Sponsored by the Landsat Data Continuity Mission (LDCM) and focused on the delivery of Landsat image data using the WCS. • The data used for the prototype came originally from the global land survey (GLS) collection and later from the Landsat level-1 standard product. • The number of scenes was restricted, both spatially and temporally, to a few hundred scenes. • Each scene was represented as a WCS coverage with the scene ID as the coverage name. • This makes it easy to generate a WCS call from the metadata to retrieve the data (a potential connection to CS-W usage). • It is also straightforward from an implementation standpoint as the data for each scene is contained in a single file.

9. EROS WCS Experience and Ideas • Landsat Prototype (cont.) • The main problem encountered? • The response to a GetCapabilities request can get large, quickly. • Many WCS clients have an interactive mode that issues a GetCapabilities request and presents a list of coverages to the end user. • The user can then select one or more coverages and retrieve the data. • If this approach were to be applied to the entire Landsat archive, for example, the GetCapabilities request would return several hundred thousand coverages, which could overwhelm the client (the list is typically displayed as a scrolling list or drop-down menu) and would be unwieldy for the end user.

10. EROS WCS Experience and Ideas • More about the GetCapabilities Dilemma • GetCapabilities for the MOD14 prototype only returned two coverages: one for the daily composites and one for the eight-day composites. • This was a packaging choice. • Internally, we made each composite a separate layer, • Since they were all global coverage and at regular time intervals, it just seemed to make sense to combine them and allow the search by time. • This same approach doesn’t work for large collections of individual satellite observations like the Landsat archive.

11. EROS WCS Experience and Ideas • Landsat Prototype (cont.) • Another consideration is to possibly employ the Landsat World Reference System (WRS) grid structure of Paths and Rows. • One problem is that each scene has a slightly different footprint, although we could probably go with the "nominal" footprint. • It really doesn't work if you have a pointable sensor (like ASTER, EO-1 or LDCM), at least for off-nadir acquisitions. • Another problem is that the temporal coverage is sparse. • If someone does a search with a time parameter halfway through the 16-day repeat cycle, what do we return? • If we return the closest acquisition, there is no mechanism to tell them exactly what date was really returned. • The only other alternative is to throw an exception unless the date matches an acquisition (although really the time parameter can include date and time so is "within a day" close enough or does it have to match the exact acquisition window?).

12. EROS WCS Experience and Ideas • Ideas for the future • The LDCM prototype was necessarily limited in scope due to constraints in storage, processing and budgets, but the intent of the prototype was to gain insight into what would be necessary to make something like the existing Landsat collection available. • The first prerequisite is that the input data be available in a geo-referenced format. • While it is possible to offer level-0 data as a WCS coverage (using the ImageCRS coordinate reference system), it is not readily usable by most GIS packages. • This is less of a problem for newly collected data since most is routinely processed to a geo-referenced format. • It is a much larger problem for the existing 30 year archive, most of which is still in level-0 format and stored on tape.

13. EROS WCS Experience and Ideas • Ideas for the future (cont.) • Representing each scene as a separate WCS coverage with the scene ID as the coverage name makes it trivial to generate a GetCoverage or DescribeCoverage request from the scene metadata. • Unfortunately, as noted above, this does not work well with the GetCapabilities request. • The WRS Path/Row type of coverage approach might work well for the existing Landsat archive (and merits some prototyping), where scenes have a consistent footprint, • But it would be a problem for sensors that can be pointed off-nadir (e.g. Aster and EO-1).

14. EROS WCS Experience and Ideas • Discussion questions: • What are the core functional elements of our large satellite query and selection client systems (e.g. EarthExplorer, GloVis for USGS)? • What are the key types of query refinement mechanisms that we use to help users converge on their satellite acquisition granule of interest? • What are the key metadata elements that we employ to support this refinement? • What kind of Web service or set of services would mimic this kind of interaction between such clients and metadata/data servers? • If we conceptually define such services, would that help us to better describe and explain to the OGC what we need and where the current services fall short?