Near-Real-Time HIRDLS Processing for START08

1. Near-Real-Time HIRDLS Processingfor START08 Vince Dean HIRDLS Science Team Meeting January 30, 2008 Boulder, CO

2. START08 Timeline Near Real Time Retrievals to support flight planning for START08. • Forecast exercises • Feb 29, 2008 • Mar 21, 2008 • Flight Windows – spring 2008 • April 21 .. May 16 (maybe earlier) • June 16 .. 28

3. Observation Times • START08 based in Jeffco. Covers Western US. • Approximate times of passes of interest • Morning • 06:30 .. 09:00 GMT • 23:30 .. 02:00 MST • Evening • 20:45 .. 23:00 GMT • 13:45 .. 16:00 MST • Flight Planning Meetings • 14:00 MST

4. Descending Passes22:00 ... 02:00 MST

5. Ascending Passes:13:00 ... 16:30 MST

6. START08 – timing requirements Descending HIRDLS passes over western US START08 Flight planning meeting 12 hours for data processing, delivery to START08 Field Catalog, and analysis 23:30 MST 02:00 MST 14:00 MST

8. Plots

9. Processing Flow Level 0 Science L1PP L1X Attitude L1C Ephemeris L2PP GEOS-5 Product: Temperature L2CLD HIRDLS2 L2

10. Plotting and Delivery HIRDLS2 Plotting Code Plot Files (.png) Rename and deliver EOL Field Catalog http://catalog.eol.ucar.edu/trex/index.html

11. HIRDLS Science Data at Goddard DISC Rate-Buffered Science Data File Conversion Standard Level 0 Science Data L1PP

12. Aura Attitude/Ephemeris at Goddard DISC Rate-Buffered GBAD Attitude “Direct Readout” Software L1X Ephemeris

13. Geolocation Test • L1X 7.4.1 • Standard vs. sample attitude/ephemeris • One 2-hour contact session • Differences:

14. L2CLD Today: 20 minute near-real-time job. Yesterday: 24-hour production job. HIRRADNC HIRRADNC L2CLD L2CLD Daily Average Radiance HIRRAD HIRRAD

15. GEOS-5 Temperature forLine of sight gradient GEOS-5 analysis product D5HIRDL1 L2 HIRDLS2 GEOS-5 Forecast Forecast Runs: Once or twice daily, up to five days in advance.

16. Strategy • Outsource software development to DISC where possible: • Rate-buffered to PDS conversion • Attitude/ephemeris file conversion • Relatively low risk: • Goddard’s plans seem attainable. • They seem confident. • But “best effort”

17. Tasks • Enhancements to science code • L2CLD • L2 • Software development at Goddard • Attitude/ephemeris • Rate-buffered to PDS conversion • Data management • SIPS software enhancements • Validation of approximations • Test runs needed

18. Extra Slides

19. Sources of HIRDLS science data • Standard PDS files from Goddard • Typically arrives 3 hours after last date (quite variable!) • Expedited PDS files from Goddard • Somewhat faster • Rate buffered data from EDOS • Typically arrives ½ hour after last data (quite variable!) • New shorter PDS files from Goddard

20. Level 0 Science Data Sources Aura Goddard DISC Ground Station EDOS PDS data RBD? PDS data Rate-buffered data. Expedited PDS data HIRDLS SIPS

21. PDS data • HIRDLS Science Packets • In order • No duplicates • Normally delivered in regular 2-hour files. • Additional latency to fill out complete 2-hour chunk—maybe wait for next pass.

22. Rate-Buffered Data • HIRDLS science packets with additional headers • Typically covers one contact session • About one orbit (100 min) • Size and timing are irregular • Packets may be out of order or duplicated • We understand file format, and we receive the data, but no production experience.

23. RBD processing at HIRDLS Standard HIRDLS Science file (HIR0SCI_FIXED) Rate-buffered data L1PP (modified) We already receive RBD data routinely. L1PP already sorts packets, part of fixing clock problem. Includes library code to read RBD files. Relatively small change to extend for RBD processing. Unknown operational issues—we have not used RBD routinely.

24. Goddard DISC to process rate-buffered data? • Goddard DISC is planning to process rate-buffered data for a quick-delivery PDS product • For MLS • For HIRDLS? • 15 minutes granules instead of 2 hours • Same collecting and sorting as done by EDOS. Goddard DISC 15-minute PDS science granules EDOS RBD

25. Attitude/Ephemeris • We require files in Toolkit format • Normal delivery takes about 1 day • Other sources of forecasts/approximations: • Use files from 16 days ago, advancing timestamps • Get a forecast from Flight Dynamics and convert file format • Use GBAD files and convert file format • Use orbsim program to generate forecasts

26. Attitude/Ephemeris16-day cycle 16-day old attitude & ephemeris Rewrite times in files “Forecast” attitude and ephemeris Small custom program. Not complex. File formats are well documented.

27. Attitude/EphemerisFlight Dynamics Forecast Forecasts of attitude/ephemeris in Toolkit Format Forecasts of attitude/ephemeris from Flight Dynamics ? Custom code to convert file formats. Unknown complexity.

28. Attitude/EphemerisGBAD to Toolkit DirectReadout code Attitude & Ephemeris in Toolkit format Aura GBAD file(s) MLS has provided leads to code which performs this conversion from NASA “DirectReadout” project. This code has been used for AIRS and Goddard DISC is installing it.

29. What is GBAD data?(Ground-Based Attitude Determination) Aura Forecasts of attitude/ephemeris (Flight Dynamics?) Converted to GBAD format and delivered to HIRDLS SIPS, via EDOS and DISC. GBAD files contain forecasts of attitude and ephemeris, delivered to the spacecraft and then back to earth. There is said to be software which will convert those files to toolkit format.

30. Attitude/EphemerisOrbsim Orbital Parameters Orbsim Attitude/Ephemeris in Toolkit Format Testing tool provided with Toolkit. Intended only for software testing, not for production forecasts.

31. GEOS-5D5HIRDL1 subset GMAO Assimilation Goddard DISC HIRDLS SIPS • Files created 4 times daily. • Delivered on regular 6-hour intervals? • Can they create forecasts as a special product?

32. Plots(Bill Hall’s IDL) HIRDLS2 IDL Plot code Rename and deliver to Field Catalog Plots EOL Field Catalog (Web site)

33. End to end test(Prototype) • Early prototype of processing steps and data products. • Explore: • Client requirements • System and configuration issues • Test bed for refinement of processing steps.