Astronomy in a nutshell e-Astronomy: the Virtual Observatory

1. e-Astronomy: the Virtual Observatory and its implementation Paolo PadovaniHead, Virtual Observatory Systems Department, ESO, Garching bei München, Germany&EURO-VO Facility Centre Scientist • Astronomy in a nutshell • e-Astronomy: the Virtual Observatory • The Virtual Observatory world-wide, in Europe, and at ESO

2. The Ultimate Goal

3. The way Astronomy works • Telescopes (ground- and space-based, covering the full electromagnetic spectrum)  Observatories • Instruments (telescope/band dependent)  Observatories/Consortia • Data analysis software (instrument dependent)  Observatories/Consortia/Researchers • Active Archives  Observatories/Agencies • Publications  Journals • Data curation (metadata + tables & catalogues)  Data curators

4. … and Public Outreach

5. Astronomy from an e-Science Perspective: the Good News • Observational data normally stored in digital repositories (“astronomical archives”), freely available on-line after < 1 year • Results published in academic journals, all available on-line • Table of contents and abstracts freely available • Full content in general freely available after 3 years • Some tables immediately available through data curators • One single entry point for journals: ADS • Two-way links between most archives and publications • Data curators (object metadata + catalogues) on-line; some links asawith archives and publications • Analysis software available from Observatories/Archives on-line • Press release and outreach material (pictures, movies) on-line

6. Astronomy from an e-Science Perspective: the “less Good” News • Different astronomical archives have widely different access/search interfaces and standards/conventions; serving mainly raw data • Widely specialized, complex analysis software for various sub-branches; steep learning curve, but multi-wavelength now the norm • Publication - Archive links point to raw, unprocessed data • Object metadata not homogeneously defined; links with archives and publications not complete • Press release and outreach material disconnected • Huge data collections: download and data analysis on desktop problematic/impossible (TB dataset: ~ 1 week at 10 Mbps) • Information avalanche!

7. Telescope Collective Area Increase Mountain & Gillett 1998

8. ESO Archive Growth

9. Astronomical Publication Growth White 2007

10. Outreach Products Calibrations Processed Data Raw Data Observatories Data Headers Software Data Products Logs Instruments Analysis VO Telescope control Catalogues Publications Computing Tables Figures Papers courtesy of P. Quinn

11. The Virtual Observatory • The Virtual Observatory (VO) is an innovative, still evolving, system to: • take advantage of astronomical data explosion • allow astronomers to interrogate multiple data centres in a seamless and transparent way and to utilize at best astronomical data • permit remote computing and data analysis • foster new science • Web: all documents inside PC; VO: all astronomical databases inside PC • VO  democratization of astronomy! • All of the above requires the various players to speak the same language VO standards and protocols are defined and adopted within the International Virtual Observatory Alliance (IVOA), which includes 16 VO projects world-wide

12. 16 China Australia Europe India Canada UK Russia Spain USA Italy Korea Armenia Hungary Germany France Japan

13. The IVOA: http://ivoa.net • Mission: To facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory • Needs: standardization of data and metadata, data exchange methods, and registry (list of available services and what can be done with them) • Organized in working groups which tackle various aspects (e.g., data access, data modelling, query language, registries, etc.) • IVOA Executive Board includes representatives from all VO projects • Works by telecons, “TWiki” pages, and bi-annual meetings (last one in Beijing [May 2007]) • VO projects starting to move from development to operational

14. VO in EUROPE: the EURO-VO Project http://www.euro-vo.org • Successor to the Astrophysical Virtual Observatory (AVO), which was a 5 M€, Phase A study (2001 - 2004/5) on the scientific requirements and technology for building the VO in Europe, 50% funded by European Community (Fifth Framework Programme [FP5]) • Includes 8 partners: ESO, European Space Agency (ESA), plus six national nodes: INAF (Italy), INSU (France), INTA (Spain), NOVA (Netherlands), PPARC (UK), and RDS (Germany) • Has three components: Data Centre Alliance, Technology Centre, Facility Centre

15. An alliance of European data centres who will populate the EURO-VO with data, provide the physical storage and computational fabric and who will publish data, metadata and services to the EURO-VO using VO technologies An operational organization, that provides the EURO-VO with a persistent, centralized registry for resources, standards and certification mechanisms as well as community support for VO technology take-up and scientific programs. EURO-VO’s “public face” A distributed organization that coordinates a set of research and development projects on the advancement of VO technology, systems and tools in response to scientific and community requirements

16. The EURO-VO Project: Components and funding situation • Data Centre Alliance (DCA) co-funded by the European Community at the 1.5 M€ level (FP6) for 2 years since Sept. 2006; 8.5 FTE/yr. Lead by CDS, Strasbourg. Main activities: • Community Workshop (March 2007 at ESAC) • Data Centre Workshop (June 2007 at ESAC) • VO-TECH co-funded by the European Community at the 3.3 M€ level (FP6) for 3(4) years since Jan. 2005; 21 FTE/yr. Lead by AstroGrid, UK. Main activities: • “Design Studies” meetings every 6 months • Facility Centre (FC), located at ESO, co-managed by ESO & ESA; support at “best-effort” level [~ 2 FTEs] (FP7 proposal May 2 2007). Main activities: • 1st EURO-VO technical Workshop (June 2005 at ESO) • Selection of EURO-VO Science Advisory Committee; held first two meetings • EURO-VO Web pages • EURO-VO Research Initiative

17. ESO • The European Organisation for Astronomical Research in the Southern Hemisphere • Intergovernmental organisation (founded 1962) to 'establish and operate an astronomical observatory in the southern hemisphere, equipped with powerful instruments, with the aim of furthering and organizing collaboration in astronomy…'. • Three observational sites, all in Chile: medium-sized telescopes (La Silla, north of La Serena), Very Large Telescopes [VLT] (Paranal, Antofagasta), APEX (in operation) and ALMA (giant array of 12-m submillimetre antennas under construction) (Llano de Chajnantor, San Pedro de Atacama). • 13 member states + agreement with Chile: 10% of telescope time and an annual fund for the development of astronomy

18. ESO and the VO • ESO operates one of the largest astronomical archives in the world • ESO is committed to become a key provider of data and resources in the Virtual Observatory: • Founding member of IVOA • Lead partner of Astrophysical Virtual Observatory • Lead partner of EURO-VO • To this aim, ESO has created (Nov. 2004) the Virtual Observatory Systems Dept. (17 people) to manage ESO’s VO activities but especially with the goal of turning its archive into a powerful scientific resource for the ESO and world-wide community

19. High-level VO Work on the ESO Archive • Build a “VO layer”: • “translate” any locally defined parameter to the standard (IVOA compliant) ones • hide any observatory/telescope/instrument specific detail and work in astronomical units  provide the right metadata • Improve archive interface • Creation of highly processed (“science-ready”) data: very important for VO users • Collection, validation, and publishing into archive (according to detailed guidelines) of highly processed community data [paradigm change] • Same from Large Programs and upcoming Public Surveys (VST & Vista) [now required]

20. Summary • Astronomy has done the groundwork to become e-Astronomy • Some more work is required to integrate and make interoperable the various components  the Virtual Observatory • Goal: all astronomical databases “one click away”  democratization of Astronomy! • The Virtual Observatory is an international effort. ESO plays a major role in Europe (EURO-VO) and world-wide (IVOA)

21. Organisation for Economic Co-operation and Development (OECD) comments on the VO • Findings • The Virtual Observatory concept is a bold community-led response to the challenges the astronomical community faces in data management and storage. Impressive progress has been made and the momentum of the International Virtual Observatory Alliance will ensure sustained progress, provided the agency level support and funding is available. • Recommendations • New projects and facilities must take the data management, storage, maintenance, and dissemination into account at the earliest planning stages, consulting potential users in the process. Agencies should recognise that this is an important long term issue and should co-ordinate plans, provide adequate funding on a long-term basis, and support development and maintenance of the needed infrastructure. Agencies should encourage the broadening of the existing VO collaboration into a fully representative global activity.