The Perspective of Virtual Observatories Supporting Space Weather Research and Operations

1. The Perspective of Virtual Observatories Supporting Space Weather Researchand Operations Peter Fox Tetherless World Constellation Rensselaer Polytechnic Institute USA

2. Background Scientists should be able to access a global, distributed knowledge base of scientific data that: • appears to be integrated • appears to be locally available But… data is obtained by multiple means (models and instruments), using various protocols, in differing vocabularies, using (sometimes unstated) assumptions, with inconsistent (or non-existent) meta-data. It may be inconsistent, incomplete, evolving, and distributed And… there exist(ed) significant levels of semantic heterogeneity, large-scale data, complex data types, legacy systems, inflexible and unsustainable implementation technology

3. Diversity, Integration, Size, … • Not just large (well organized, long-lived, well-funded) projects/ programs want to make their data available, individuals as well • Data policies: highly variable or non-existent; affects users • How can data be managed to solve challenging scientific, application or societal problems without the continued need for a scientist or other end user to know every detail of complex data management systems?

4. Diversity, Integration, Size, … • Scientific data repositories: • Most data still created in a manner to simplify generation, not access or use • Very diverse organization of data; files, directories, metadata, emails, etc. • Source/origin management is driven by meta-mechanisms for integration and interoperability (but still need performance) • Virtual Observatories • Data Grids • Increasing realization: need management for all forms of ‘data’, I.e. virtual data products are becoming the norm • Size matters; personal data management is as big, or a bigger problem as source data management

5. Virtual Observatories • Conceptual examples: • In-situ: Virtual measurements • Related measurements • Remote sensing: Virtual, integrative measurements • Data integration • Managing virtual data products/ sets

6. Virtual Observatories Make data and tools quickly and easily accessible to a wide audience. Operationally, virtual observatories need to find the right balance of data/model holdings, portals and client software that researchers can use without effort or interference as if all the materials were available on his/her local computer using the user’s preferred language: i.e. appear to be local and integrated

7. But .. Data has Lots of Audiences Information Information products More Strategic Less Strategic Scientists too! From “Why EPO?”, a NASA internal report on science education, 2005

8. Data is Important in non-expert situations (e.g. Space Weather Operations)!! • Data is a critical component for understanding how science works and broader applications. With it, we can: • Design and conduct scientific investigations • Make decisions • Understand the quality of data and the role of uncertainty in results • Focus on quantitative analysis and reasoning • Explore tools for visual representation • Virtual Observatories provide new mechanisms for collecting, manipulating, and aggregating data, information and information products. They also provide the opportunity for new kinds of student and non-expert experiences.

9. What is a Non-Specialist Use Case? Someone should be able to query a virtual observatory without having specialist knowledge E.g. Teacher accesses internet goes to An Educational Virtual Observatory and enters a search for “Aurora”.

10. What should the User Receive? Teacher receives four groupings of search results: 1) Educational materials: http://www.meted.ucar.edu/topics_spacewx.php and http://www.meted.ucar.edu/hao/aurora/ 2) Research, data and tools mediated for them via VOs, knows to search for brightness, or green/red line emission 3) Did you know?: Aurora is a phenomena of the upper terrestrial atmosphere (ionosphere) also known as Northern Lights 4) Did you mean?: Aurora Borealis or Aurora Australis, etc.

11. Shifting the Burden from the Userto the Provider (with the help of VxOs)

13. ? Early days of VxOs - alas there shall be more than one! VO2 VO3 VO1 DBn DB2 DB3 … … … … DB1

14. Lightweight semantics Limited meaning, hard coded Limited extensibility Under review The Astronomy approach; data-types as a service Limited interoperability • VOTable • Simple Image Access Protocol • Simple Spectrum Access Protocol • Simple Time Access Protocol VO App2 VO App3 VO App1 Open Geospatial Consortium: Web {Feature, Coverage, Mapping} Service Sensor Web Enablement: Sensor {Observation, Planning, Analysis} Service use the same approach VO layer DBn DB2 DB3 … … … … DB1

15. Added value Education, clearinghouses, other services, disciplines, etc. Semantic interoperability Added value Added value Semantic query, hypothesis and inference Semantic mediation layer - mid-upper-level Added value VO API Web Serv. VO Portal Query, access and use of data Mediation Layer • Ontology - capturing concepts of Parameters, Instruments, Date/Time, Data Product (and associated classes, properties) and Service Classes • Maps queries to underlying data • Generates access requests for metadata, data • Allows queries, reasoning, analysis, new hypothesis generation, testing, explanation, etc. Semantic mediation layer - VSTO - low level Metadata, schema, data DBn DB2 DB3 … … … … DB1

16. Application: Synthesizing the solar spectrum

17. Inferred plot type and return required axes data Ability to quickly plot data to assess suitability, quality, and produce a quick copy with some customization for a preliminary study. Graphics also require data management. Numerous VOs in this community

18. Developments for Virtual Observatories • Scaling to large numbers of data providers • Building and delivering data products that are in demand (pre-computed and on-the-fly), covering the spectrum of time latencies • Crossing disciplines and beyond science use • Data quality (propagating and explaining) • Branding and attribution (where did this data come from and who gets the credit, is it the correct version, is this an authoritative source?) • Provenance/derivation (propagating key information as it passes through a variety of services, processing algorithms, …) • Security, access to resources, policy enforcement • Interoperability at a variety of levels (~3)

19. Summary/ Discussion • The VO paradigm in is wide-spread use in Earth and Space Sciences and is increasingly able to respond to end use communities and add value to data repositories • There is an active community; meeting, publishing, developing, implementing, i.e. they are organized and many are collaborating • Standards and practices are being developed, and leveraged from other sources (IVoA, SPASE, …) • Successful implementations are in production and use (some even have evaluations) • New science and applications are being enabled and performed

20. VOiG 2009, Spring, Locations TBD http://www.voig.net/ (voig@voig.net)

21. Use cases • Who (person or program) added the comments to the science data file for the best vignetted, rectangular polarization brightness image from January, 26, 2005 1849:09UT taken by the ACOS Mark IV polarimeter? • What was the cloud cover and atmospheric seeing conditions during the local morning of January 26, 2005 at MLSO? • Find all good images on March 21, 2008. • Why are the quick look images from March 21, 2008, 1900UT missing? • Why does this image look bad?