1. Australian Virtual Observatory Pacific Rim Applications and Grid Middleware Assembly The 4th Workshop 5th-6th June 2003 Monash University David Barnes School of Physics, The University of Melbourne

2. What is a Virtual Observatory? • A Virtual Observatory (VO) is a distributed, uniform interface to the data archives of the world’s major astronomical observatories. • A VO is explored with advanced data mining and visualisation tools which exploit the unified interface to enable cross-correlation and combined processing of distributed and diverse datasets. • VOs will rely on, and provide motivation for, the development of national and international computational and data grids.

3. Scientific motivation • Understanding of astrophysical processes depends on multi-wavelength observations and input from theoretical models. • As telescopes and instruments grow in complexity, surveys generate massive databases which require increasing expertise to comprehend. • Theoretical modeling codes are growing in sophistication to consume available compute time. • Major advances in astrophysics will be enabled by transparently cross-matching, cross-correlating and inter-processing otherwise disparate data.

4. Sample multi-wavelength data for the galaxy IC5332 (Ryan-Weber) Visible blue light - young hot stars Infrared light - old cooler stars H-alpha spectral line - star forming sites HI spectral line - gas to form stars HI velocity field - kinematics HI velocity dispersion - gas stability, parameters Integrated HI spectrum - total neutral gas mass, distance from redshift And this is just the data on one object from three Australian telescopes!

5. Fundamental VO Challenges • Data description: multi-wavelength, multi-resolution, multi-dimensional, multi-domain (optical, radio, X-ray, …), world coordinate systems, limited period ownership, … • Data provision: distributed mass storage, high-bandwidth networks, registries, … • Data processing: high performance clusters as grid nodes, data to code versus code to data, mountains of legacy software!, … • Interface: portals, visual data flow control, analysis tools, display tools, …

6. Aus-VO structure 2003 • Phase A funded AUD 260K by a 2003 ARC grant: • The University of Melbourne • The University of Sydney • CSIRO Australia Telescope National Facility • Anglo-Australian Observatory • Additional institutes participating w/o direct funding from the ARC grant: • ANU, Mount Stromlo Observatory & APAC • CSIRO Mathematical and Information Sciences • University of Queensland • VPAC & GridBus (Melb) • Lead investigator Rachel Webster (Melb) • Project scientist David Barnes (Melb)

7. Aus-VO projects 2003 • Common format on-line archive projects: • HIPASS catalog: HI Parkes All Sky Survey: neutral Hydrogen spectral line survey, ~4,300 sources with 138 parameters and 1024-channel spectra • SUMSS catalog: Sydney University Molonglo Sky Survey: radio continuum survey at 843 MHz, >100,000 sources • 2dFGRS QSO catalog: 2-degree Field Galaxy Redshift Survey: optical spectra of >20,000 southern quasi-stellar objects • ATCA archive: Australia Telescope Compact Array archive: all observations since 1988, circa 1.5 TB of more than 1,000 separate observing projects! Massive exercise in describing data with metadata. • MACHO archive: Massive Compact Halo Objects archive: 8yr lightcurves for >18M stars

8. Aus-VO projects 2003 • Server-based visualisation tools: • client Java canvas for legacy software package AIPS++ to draw on from a remote server (ATNF) • grid-service implementation of distributed volume rendering - remote data transferred to remote cluster, with display canvas applet supplied by coordinating portal • Pipelines to enable on-line reprocessing of archived raw or pre-processed telescope data: • Molonglo Observatory Synthesis Telescope • Interfaces: beta testers for the AstroGrid consortium software

9. VO Interface & Portal • Agreement with AstroGrid (UK e-Science project) to be testers for their data publication and portal creation code. • Collecting the necessary resources and intend to have an AstroGrid-based portal serving HIPASS catalog data for demonstration at IAU General Assembly in July 2003. • Separately testing IBM Lotus Notes and Domino Server for publication of astronomical catalogs.

10. Grid-based Visualisation • ATNF will build a Java PixelCanvas so that AIPS++ visualisation applications can be deployed as Web-Service and Grid- Service Java Applets • AIPS++ is modern, OpenSource software for reducing (radio) astronomy data, 1.6M lines of code.

11. Grid-based Volume Rendering • Agreement between Melbourne and AstroGrid to develop our existing distributed-data volume rendering code into a fully-fledged Grid-Service. [see my talk at GridBus this Saturday] • Challenge is to interactively render a multi-GB cube at the IAU GA 2003, using GridFTP to transfer the data volume from a remote data warehouse to a remote rendering cluster and display and control the rendering from an applet.

12. The near future: data grids for Aus-VO • Australian archives range from ~10 GB to ~10 TB in processed (reduced) size. • providing just the processed images and spectra on-line requires a distributed, high-bandwidth network of data servers – that is, a data grid. • users may want some simple operations such as smoothing or filtering, applied at the data server. This is a virtual data grid.

13. The near future: compute grids for Aus-VO • More complex operations may be applied requiring significant processing: • source detection and parameterisation • reprocessing of raw or intermediate data products with new calibration algorithms • combined processing of raw, intermediate or "final product" data from different archives • These operations require a distributed, high-bandwidth network of computational nodes – that is, a compute grid.

14. 2004 ARC LIEF grant • 10 partners! • more data archives on-line • more tools developed with special focus on server-based visualisation • construction of the Australian Astronomy Grid…

15. The Australian Astronomy Grid 2004

16. http://www.aus-vo.org