Planetary Science Archive (PSA) and PDS Introduction VEX / MEX Data Workshop

1. Planetary Science Archive (PSA) and PDS Introduction VEX / MEX Data Workshop 27th June to 1st July 2011 ESAC Dave Heather (dheather@rssd.esa.int)

2. Planetary Science Archive • Available since March 2004: • http://www.rssd.esa.int/PSA • Active development, PSA 3.0 to come in December • PSA Data Handling Team @ ESA-ESAC (Spain) • David Heather lead • Archive Scientist, setting up the requirements • Interface with Instrument Teams • Validating Data Sets • PSA Development Team @ ESA-ESAC (Spain) • Christophe Arviset lead • PSA systems software development

3. PSA Definition and Purpose The Planetary Science Archive (PSA) is the initiative, the setup, the process and the implementation to preserve data from ESA’s spacecraft to planetary bodies, as well as supplementary information acquired in laboratories or ground-based observatories. • The main objectives of the PSA are: • to support the experimenter teams in the preparation for the spacecraft and ground-based long-term archives • to enable and ensure the long-term preservation of these archives • distribution of scientific useful data to the world wide scientific community • provision of supplementary data services aiming to maximize the usage of planetary mission data and ease the scientific data analysis.

4. Giotto Huygens Rosetta Smart-1 Venus Express Mars Express PSA : one archive, several missions ALL IN PDS FORMAT

5. Current Status, Available Datasets • GIOTTO data from comet Halley and Gripp-Skellerup • Comet Halley ground-based observations (Halley-Watch) • Comet Wirtanen ground-based observations • Mars Express instrument and auxiliary data (ongoing) • Venus Express instrument and auxiliary data (ongoing) • Smart-1 instrument and auxiliary data (pending) • Rosetta instrument and auxiliary data (ongoing) • Huygens (complete) • BepiColombo data handling and archive support – in preparation • ESA supported instruments on Chandrayaan 1 – pipeline development and support from PSA

6. The PDS Standards • Within the PDS Standards, data are organized into volumes and data sets, collecting together observations of similar type, processing level, and/or from a specific mission phase or observation campaign. • All PSA data are compliant with NASA’s Planetary Data System (PDS) Standards • This is the first step in ensuring that the data stored in the PSA are as widely usable as possible and will remain so for many years to come • Widely accepted and understood within the planetary science community • Standards available online and downloadable as a ‘Standards Reference’ pdf file.

7. The PDS Standards • All PDS compatible data formats conform to a unified standard format incorporating documentation, calibration, and raw and processed data files. • The format distinguishes: • Meta-data Files • At the intermediate level are files which describe the circumstances and parameters of the data collection (for example: the instrument used, the observatory site, the type of data collected, etc.). These files are called "catalog objects" or just "catalog" files. • Volume/Dataset Description Files • These files are included on any distribution medium intended to be an entire, self-contained archive - for example, a set of DVDs or a ZIP file. The volume description files detail the organization of the data in the archive (in terms of directories, e.g.) and any additional documentation and software which might be included.

8. The PDS Standards • The lowest level is that of the data themselves. • Each file will have a PDS label either appended to the beginning of the file or, more often, in a separate but proximate file. "One file, one label" is the general rule. • The LBL file must contain a full description of the data object(s) • Allows for all PDS products to be read by existing standard software • Allows for long-term data usage with clear description of data to allow users to read files with their own software

9. PDS Resources • http://pds.jpl.nasa.gov • More information on the PDS and complete versions of the Standards Reference and Data Dictionary • The Data Dictionary can be found here • Supporting Software Tools can be downloaded here

10. A Few PDS Tools • http://pds.jpl.nasa.gov • Go to the Tools menu • NASA View • A useful tool for visualising labels and simple small data products • PDS Software Tools • Some general tools reading tables and manipulating PDS files.

11. NASA View Example • NASAView can open simple PDS compliant data • Click on File -> Open Object and select the product you want to see • You can also click on Label -> Full Label to view the complete PDS label with full information on the data product being viewed

12. ReadPDS • To retrieve the latest fully tested release or previous releases, check out the page: • http://pdssbn.astro.umd.edu/nodehtml/software.shtml • Within your IDLDE, select the File->Preferences and check that your path contains the ReadPDS. • ReadPDS is an IDL package that is maintained and distributed by PDS's Small Bodies Node (SBN). It allows most PDS compliant products to be read in to the IDL software.

13. ReadPDS 2 • To run the ReadPDS you also need to have the astrolib library installed. If not already done, you can retrieve the astrolib library at http://idlastro.gsfc.nasa.gov/ . • Read the aareadme.txt file distributed with the ReadPDS package. There is no further way to test your installation than to call a procedure within IDL. So, on your IDLDE command input, enter e.g.: • If the routine READPDS is compiled successfully, your installation is ok.

14. PSA Standard User Interface • User friendly web access : Java Applet • http://www.rssd.esa.int/PSA • Powerful queries, organized by panel • General query panels • Mission / instrument specific panels • Hierarchical data presentation • 1 Dataset -> several data products • Public data accessible to all, proprietary data accessible only to privileged users • Images preview (icons, full image) • Quick download (1 click) at various processing levels, shopping basket retrieval

15. PSA Mars Map Based Interface • Interface for Mars Express image data • http://www.rssd.esa.int/PSA • Java Applet • Currently HRSC and OMEGA • No need to be an expert • Easy for general public • Area selection by mouse • Display images footprints • Image download by 1 mouse click • Some are big (be patient! ) • Go back and forth to the standard interface to refine search parameters

16. Search Button Select Base Map Type (MOLA / VIKING…) Select Instrument / Detector 1. Click and drag to select your area of interest 2. Press ‘Search’ and wait for footprints Zoom function Access to ‘standard’ UI 3. Click on desired footprint Long / Lat information PSA UI – The Map Interface 5. Transfer at any time to the standard interface for advanced searches. Your search parameters will be remembered. 4. Download directly or view label / docs etc. 6. Return to the map browser, you can finish your search as required.

17. PSA Data Set Browser Interface • Opening Screen Click on the instrument that interests you Select the data set you wish to look at Browse through the directories to locate the files you want Right click the product you want to save etc. to bring up the menu Left click to view directly (where possible)

18. Automatic Download Scripts / Mirror Scripts Be careful when using automatic downloads, as the size of the datasets can be huge (several 100GBytes or larger)! Please contact the PSA Helpdesk beforehand. • You can use any automatic download script (mirror, etc) to recursively download the data on the PSA. • Before installing a regular download scheme on your side (mirror, etc), please inform us by sending a short note to psahelp@rssd.esa.int. • Be aware that the files that you see in your www-browser or your ftp-client are not the physical files, but a link into the PSA database system. When downloading data, the PSA server resolves the link on the fly and delivers the data back to you. • When building mirror scripts, you can compare your data items against size or time with the ones on the PSA server.

19. Notification Management:Registration I Goto Login Page Login with your user id / password Login and stay here Select ‘Change Notification Setting'

20. Registration II View Return Select Add

21. Notification via Email Once you have registered, you will receive e-mails with announcements of the new data in the archive that you are interested in.

22. Interoperability aspects in Planetary • IPDA contains representatives from all national agencies active in planetary data collection • Standards being defined in IPDA • Closer link required between IPDA and EuroPlanet • IDIS and IPDA ? • Closer link required between Planetary and Astronomy • Registry • Access protocols • But very different Data Model and data generation process

23. From Mars Map Browser, Select region of interest • Contact PSA and PDS using the PDAP (Planetary Data Access Protocol) • Display NASA PDS and ESA PSA images Interoperability ESA PSA – NASA PDS • Prototype in development In the IPDA context

24. VEX I/O via PDS Atmos

25. VEX I/O via PDS Atmos

26. Comments and Suggestions • Please send us your remarks: psahelp@rssd.esa.int • In case of problems, please visit the FAQ page before contacting us • In case you can not find the data you are looking for, please read the Mission Specific pages on the PSA home page (http://www.rssd.esa.int/psa)