Gravitational Wave Open Data Workshop ligoltech/gwodw2011/ 2011 October 27/28

1. Gravitational WaveOpen Data Workshop http://www.ligo.caltech.edu/gwodw2011/2011 October 27/28 Organized by Steve Groom, Christian Ott, Roy Williams and the LOC at LIGO Livingston Observatory, LouisianaOperated by LIGO Lab (Caltech, MIT)On behalf of the LIGO Scientific Consortium Supported by National Science Foundation

2. Why are we here?

3. LIGO Observatory Livingston, Louisiana

4. GWODW workshop • Advanced LIGO • First science run in 2015 (we hope!) • Multiple GW detections in 2015 (we hope!) • Global observatory with VIRGO, Indigo (we hope!) • We are here to find out: • “What do astronomers want from LIGO Open Data?”

5. GWODW workshop = Science Program+ “Breakout Groups” Community input  Report  Implementation  Science

6. We want to know what is in YOUR heads … So here is some CANDY … howstuffworks.com

7. Breakout groups • A: Information Technology • B: Smoking Gun: finding the flash or afterglow • C: Full Data: science from the strain channel • D: New searches: new science, fresh ideas

8. LIGO Open Data Plans

9. “The top 10” benefits of data sharing in astronomy 1) Early data releases greatly improve the final product 2) Early data releases enable coeval science 3) More science is extracted from the same dataset 4) Enables reproducibility of science results 5) Synergy between different datasets 6) Cross-disciplinary science 7) Sometimes the only way to secure scarce resources 8) More citations and prestige to the team 9) Education and public outreach 10) Ethics and broader impact EM Smoking Gun Coincidence and multi-messenger Eg Astronomy and data-mining Need broad support Jobs for LIGO Postdocs

10. Data sharing is not free: comes with costs and risks: 1) requires higher standards than for internal use, including publications describing data formats, provenance and metadata (insiders’ "know-how") 2) the cost of curation (servers, help desk, etc.) 3) risk of being “scooped” (larger for very focused/specialized data streams, more likely for an experiment than for a survey) But: “Does releasing data weaken collaboration?” Not really, indeed Steve Ritz: “The Fermi collaboration became even stronger after the public data release.” and a lot of additional evidence from other surveys.

11. Timeline 2011 2012 2013 2014 2015 2016 2017 2018 2019 L1 H2 H1 S7 Community Engagement 2.5 years Phase 2B: Full Data (early date) Phase 2A: Immediate Triggers (early date) Now Phase 1: First Detections (early date) For greater details: See LIGO Data Management Plan, Jan 2011, LIGO-M1000066 https://dcc.ligo.org/cgi-bin/DocDB/RetrieveFile?docid=9967

12. Open Data Phase 1 • 1: LIGO will release detection data • Optimizing instruments, few detections • All data meaningless without expert knowledge • with published detections and important non-detections • real-time alerts distributed privately via MOU • Whole LSC writes 800-author papers

13. Open Data Phase 2 • 2A: Immediate triggers • Interoperable with other event streams • Follow-up of external triggers • Observer decides using LIGO tools • Trigger repository for cross-correlation study • Uniform among multi-mission data • Can publish with or without LSC review • 2B: Release all h[t] with catalog of past events • 24 month after Release 2A, proprietary period, 6 month releases

14. TransitionWhen Detections Become Ordinary • Transition to Phase 2 triggered by first of: • Volume trigger • Based on rate estimates (LIGO-T1000414) • 3x10^7 Mpc^3-yr space-time-volume searched for coalescing double neutron-star systems at a specified false alarm level. • Translates to 0.9 years of running at aLIGO design sensitivity. • Detection trigger • Plentiful detections: LIGO decides it’s time • Calendar trigger • Transition would occur a fixed amount of time after official aLIGO acceptance of all 3 interferometers. • Current proposal results in first bulk release ≥ 2018.

15. Sketches for LIGO open data

16. Machine status

17. When were the machines operating?

18. Mock data for one eventhttp://www.ligo.org/science/GW100916/

19. Mock data for one eventhttp://www.ligo.org/science/GW100916/ • Likely galaxies shown in Green • Dashed line 95% confidence • Dog shape is Canis Major • This plot was ready in 30 min

20. Mock data for one eventhttp://www.ligo.org/science/GW100916/ LSC Confidential. Please do not forward or copy.

21. Aladin Skymap + WMAP + Optical + Simbad + Veron QSO catalog

22. Microsoft WorldWide TelescopeSkymap over Fermi sky and constellations Canis Major Vela SNR

23. Event timeline click event for drilldown

24. Outreach

25. Monitoring, Triggers, Follow-upsAtom and RSS feed technology

26. Phone Apps LSST-funded Transient Events App for iPhone Runs from Skyalert reports of CRTS optical transients

27. Science Gateways: Ordering data and computing over the web

28. Citizen Science Galaxy Zoo Lintott, Szalay, lots more

29. Open Data Survey Please fill in the survey if you haven’t already

30. Charge to the Breakouts

31. Breakout Group Constitution • Leader controls the floor • Scribe takes notes of group dicussion • Make your scribe public? • Leader presents results in plenary Friday am • If you must float, keep quiet

32. Charge to each Breakout Group • Summary of what is there now • More GW science results • Technical objectives and difficulties • How LIGO open/interoperable data can help • Relevant present/future collaborations/groups • “Broader Community” • Relevant cites/URLs

33. A. Information Technology • For: rapid alerts, source catalog, full data • For data discovery, interoperability, accessibility, preservation

34. A. Information Technology • What infrastructure for real-time astronomy with GW triggers? • What protocols to push selected triggers to the right places? • What are the right places? • Socket, Jabber, email, VOEvent, Skyalert, GCN, ATEL, CBAT ….? • What data formats, software, metadata for GW waveform (strain data)? • How do we share uncertainties in calibration and features in data? • Triggers, catalog, full waveform • How can LIGO provide mock data in advance so the broader community can prepare? • What software should LIGO provide with the data release? • Find. Read. Metadata. Visualize. Analyze. • How can we achieve data curation and long-term data preservation?

35. B. Smoking Gunfinding the flash or afterglow • Multi-messenger before, at, after the event • Gamma, X, Optical, Radio, neutrino …

36. B. Smoking Gunfinding the flash or afterglow • What do observers want from LIGO low-latency triggers besides timing, significance, and sky location? • Detailed data analysis and parameter estimation? • Auxiliary channels & their meaning? • How do requirements differ between radio, optical, Xray follow-up observers? • Coordinated follow up? • Should there be a coordinated network? If so, should LIGO organize it? • How can LIGO encourage the broader community to share their results rapidly? • How will GW data be used in concert with other transient surveys? • Figures of Merit for follow up? • What criteria will observers use to select the triggers to observe? • Multiple trigger quality categories? • Should LIGO provide mock data in advance? If so, what? • What software should LIGO provide with the data release? • Where to point the telescope? Submitting follow-ups? • What protocols to push selected triggers to the right places? • What are the right places? • Socket, Jabber, email, VOEvent, Skyalert, GCN, ATEL, CBAT ….? • Broader community • What role for medium/small telescopes? • How can we utilize an army of citizen scientists? • How can the first detections be used to excite and educate the public/K-12 community?

37. C. Full Datascience from the strain channel • Access to all h(t) and data quality (whole enchilada) • The detected signals (source catalog) • Compare to other signal catalogs: numerical relativity • The “might be” signals (trigger catalog)

38. C. Full Datascience from the strain channel • What would a LIGO source catalog look like? • Astronomy • Numerical Relativity • Detailed data analysis and parameter estimation? • How much continuous calibrated strain h(t) around an event is wanted? • Auxiliary channels and their meaning? • What if the first detection is from a burst, continuous-wave, or stochastic? • What science from mining the catalog of triggers? • Full data release • Who wants to do intensive searches of 2-year old data to avoid joining the LSC? • How can LIGO communicate known glitches and data quality? • Should LIGO provide mock data in advance? • Is http://www.ligo.org/science/GW100916/ enough? • What software should LIGO provide with the data release? • Find. Read. Metadata. Visualize. Analyze. • Broader Community • How can the first detections be used to excite and educate the public/K-12 community? • How can we utilize an army of citizen scientists?

39. D. New searches, New sciencefresh ideas • New ideas in physics astronomy etc • New combinations of data, computing, citizens • Good web interfaces for GW science • Avoiding false “discoveries”

40. D. New searches, New sciencefresh ideas • What new searches can be done? • Post-Einstein gravity, dark matter, topological universe, high freq from SMBH, NS glitches, ‘special’ SN, ??? • Combined LISA, IceCube, Optical searches • Web-based portal for casual searches • Just want to check something … • Broader • Who wants to do full searches of 2-year old data to avoid joining the LSC? • How can we utilize an army of citizen scientists? • How to avoid finding glitches and claiming GW?

41. Finally • Take an insert A,B,C,D • Have lunch with your group