1 / 17

Land PEATE support for CERES processing

Land PEATE support for CERES processing. Ed Masuoka Gang Ye August 26, 2008 CERES Delta Design Review. Element Overview. Facilitate VIIRS Science Team assessment of Land EDRs for use as Climate Data Records Facilitate evaluation of improved Land EDR algorithms provided by Science Team

tamarr
Télécharger la présentation

Land PEATE support for CERES processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Land PEATE support for CERES processing Ed Masuoka Gang Ye August 26, 2008 CERES Delta Design Review

  2. Element Overview • Facilitate VIIRS Science Team assessment of Land EDRs for use as Climate Data Records • Facilitate evaluation of improved Land EDR algorithms provided by Science Team • Support NICSE Cal/Val activities through processing SDRs as requested • Provide access to data products and results of product quality assessment to VIIRS Science Team • Support CERES Instrument Team by providing CERES RDRs and VIIRS SDR subsets and VIIRS Aerosol EDRs in MODIS format (5 minutes granules in HDF-EOS)

  3. Overview: SDR/EDR evaluation • Land PEATE is sized to produce VIIRS SDR and EDRs at 3x post-launch for two versions of code (operational algorithms and improved algorithms) and to produce global gridded products from VIIRS EDRs for comparison with MODIS. • Products run with specific versions of software are maintained in separate archive sets • Product retention is governed by online storage • Improvements to algorithms running in IDPS will be provided by the VIIRS science team. • Land Q/A team will post results of their analyses of product quality at:http://landweb.nascom.nasa.gov/cgi-bin/NPP/NPPlogin.cgi

  4. Science and Calibration Team

  5. Land PEATE Test Result • Results from evaluation of each test by LDOPE are provided • Links to following are included: • Global browse • Issues uncovered in the data products • Time series plots

  6. Land PEATE – VIIRS systems Shared resource with MODIS

  7. CERES resources in Land PEATE • 12 Dual quad-core compute servers for product generation • 1 Dual core server and 19 TB of disk for product storage • Supports 1x forward processing plus reprocessing for SDR generation and subsetting and Aerosol EDR generation

  8. Overview: CERES support • Land PEATE provides CERES RDRs and VIIRS inputs to CERES processing in MODIS format on PDR server at GSFC using the SIPS Interface • Land PEATE runs VIIRS SDR subsetting software developed by CERES to produce subsets with desired bands • Land PEATE runs Aerosol PGE (IDPS or Science Team improved) to produce VIIRS Aerosol in MODIS format • Land PEATE will reprocess the VIIRS input products as requested by CERES to support of the generation of a long-term climate record using hardware acquired for CERES support.

  9. Operations Concept for CERES support • IDPS creates CERES Raw Data Records (RDRs) and provides RDRs to the SDS SD3E and to ADS/CLASS • Land PEATE pulls CERES RDRs and VIIRS xDRs from SD3E • Land PEATE places CERES RDRs on GSFC PDR server • Land PEATE runs sub-sampler on SDR and places output on GSFC PDR server • Land PEATE places Aerosol EDR in MODIS format (5 minute granules HDF-EDOS) on GSFC PDR server • Land PEATE handles clean-up of files in CERES directory on PDR server based on PANs provided by ASDC as defined in the SIPS Interface • Land PEATE will reorder products for CERES from CLASS or remake products upon request

  10. Requirements for CERES support • Land PEATE shall ingest CERES RDRs from the SD3E • Land PEATE shall integrate, test and run in production VIIRS SDR subsetting software developed by CERES IT • Land PEATE shall produce VIIRS Aerosol product in MODIS format (HDF-EOS, 5 minute granules) • Land PEATE shall transfer CERES RDRs, VIIRS SDR subsets and VIIRS Aerosol EDR to ASDC via PDR server • Land PEATE shall meet latency requirements in CERES Science Data Products Working Agreement for product delivery • Land PEATE shall reorder or remake CERES RDRs, VIIRS SDR subsets or Aerosol EDR upon request by CERES • Land PEATE shall reprocess VIIRS inputs for CERES Climate Data Record upon request by CERES IT with versions of VIIRS software selected by CERES IT.

  11. NPOESS Systems CERES Instrument Ops Team Commands, Loads and requests NPP Systems Mission Notices and Data requests NOAA Systems CERESSystems C3S CERES PEATE Data and Science Operations CERESData Data Users Mission Notices and Data requests SDS Ancillary data Data Providers CERES RDRs, VIIRS SDR sub-sampled, VIIRS Aerosol EDR in MODIS format RDRs for LEO&A Cmd & Tlm All RDRs SD3E L-PEATE RDRs SDRs EDRs IPs Svaalbard IDPS Mission Data All xDRs ADS CERES Operational Data Flow MODISSystem GSFC PDR server

  12. MODAPS/LPEATE-ASDC Interface • SIPS Interface • ASDC will implement EOSDIS SIPS PDR-PAN interface in their new system (ANGe). • Products will be placed in a disk partition reserved for NPP CERES on the MODIS PDR server located at GSFC • ASDC will transfer products from this directory using SIPS protocol.

  13. Operations Concept for CERES: Reprocessing for Climate-quality • During 2 years of VIIRS post-launch product evaluation, algorithm improvements will be developed by the VIIRS Science Team members and evaluated by NVCSE (VIIRS Calibration Support Element), LDOPE (Land Q/A team) and VIIRS Science Team. • Results of these evaluations will be available at LDOPE, NVCSE and ST web sites. • CERES Science Team will determine versions of SDR and Aerosol software and calibration tables* to use in reprocessing on Land PEATE resources dedicated for CERES support. • Reprocessing will proceed from earliest well calibrated scene to present using best available calibration. * NVCSE is tasked only with evaluation of SDR, so complete calibration tables to ensure a climate-quality SDR may not be available.

  14. Status of PEATE development • Proxy data generator (extended SDRgen) running in Land PEATE produces VIIRS SDR (MODIS bands with VIIRS geometry).* • 16-days of proxy SDRs has been produced to test code integrated from IDPS Build 1.5.0.15.1 • Cloud Mask, Active Fires, Aerosol Optical Thickness, Surface Reflectance and Vegetation Indices • Geolocation problem in Build 1.5.0.15.1 fixed in current delivery 1.5.0.25.2 . • Aerosol calculations use 6x6 elements not 8x8 • Additional days of proxy SDRs will be generated for CERES in October * CERES requested simulated day/night band be added and that modification will complete testing by 10/1/08

  15. CERES FM-5 Delta Design Review Schedule for CERES support 8/20/08: CERES compute servers received - COMPLETED 9/30/08: CERES disk storage received 10/1/08: SDRGEN modified to add simulated day/night band 10/1/08: VIIRS sub-sampling software complete and tested at LaRC 10/3/08: VIIRS sub-sampling software delivered to Land PEATE for integration by CERES 11/5/08: VIIRS sub-sampling software completes PEATE integration tests 11/27/08: FMD connects new power for SDS computing facility 12/1/08: VIIRS sub-sampling software running at SDS Land PEATE, expected sub-sampled output has been verified by CERES DMT 1/2/09: CERES processors and disk storage installed at Land PEATE . Dependency on new power in November. 1/15/09: Large set of VIIRS sub-sampled output produced at SDS Land PEATE and verified at LaRC by CERES DMT

  16. Open Items • Sample of CERES RDRs are needed to work ingest of these files into Land PEATE for delivery to CERES. When? • Aerosol using 8x8 calculation (in Build 1.5.?). • SIPS-interface/PDR server tests with ASDC does not involve Land PEATE delivery to ASDC system but rather ASDC pulling files from GSFC PDR server per working agreement. • Define how product re-order requests will come from CERES (email message with or w/o an attached file, or a file pushed to an ftp directory). Define content of message or file. • NVCSE is not responsible for ensuring the calibration tables are adjusted to maintain a climate quality NPP VIIRS record

  17. CERES FM-5 Delta Design Review SDS/Land PEATE Assumptions • Subscribe to SD3E for all CERES RDRs (Science, Diagnostic, and Housekeeping/Telemetry) • Acquire CERES RDRs and forward to ASDC via existing Land PEATE interface • Support CERES reordering of missing data • Produce aggregated VIIRS SDRs (MODIS HDF-EOS format) in 5 minute granules (VIAE, VMAE, VDAE). Sub-sample and ship 5 minute granules (VIMD) to ASDC • Produce VIIRS aerosol in 5 minute granules (VAOT) and ship to ASDC • Post-launch: When VIIRS calibration and/or algorithm changes, reprocess VIIRS from covers-open forward and ship to ASDC to support CDRs

More Related