Public Resource Computing at CERN – LHC@home

1. Public Resource Computing at CERN – LHC@home Philippe Defert, Markku Degerholm, Francois Grey, Jukka Klem, Juan Antonio Lopez Perez, Eric Mcintosh, Jakob Pedersen, Ignacio Reguero, Frank Schmidt, Ben Segal, Christian Soettrup LHC@home Dr Jukka Klem CHEP06

2. Outline • Public Resource Computing and BOINC • LHC@home project and applications • Physics results with LHC@home • Precision of numerical results • Statistics • BOINC and grid computing LHC@home Dr Jukka Klem CHEP06

3. Public Resource Computing • Also called global computing, volunteer computing • Based on BOINC (Berkeley Open Infrastructure for Network Computing) • Platform for distributed computing using volunteered computing resources • Profit from unused CPU cycles for scientific computing LHC@home Dr Jukka Klem CHEP06

4. BOINC Infrastructure • Each project runs a server identified by a master URL (e.g. http://lhcathome.cern.ch). Project components: LHC@home Dr Jukka Klem CHEP06

5. Client Screensaver LHC@home Dr Jukka Klem CHEP06

6. Basic Principles • Communication initiated by the client • Security: BOINC uses code signing to prevent distribution of malicious executables. Each project has a key pair for code signing and the private key kept on network-isolated machine. LHC@home Dr Jukka Klem CHEP06

7. Redundant Computing • Results from different hosts are validated using application specific function • Credit: numeric measure of how much a user has contributed. Important in motivating users. LHC@home Dr Jukka Klem CHEP06

8. BOINC Applications • Public appeal • Independent parallelism • Easiest for low data/compute ratio • Usually available at least for Windows and Linux (also for MacOS, Solaris, ...) • Existing applications in e.g. C, C++, Fortran can run as BOINC applications with small modifications (BOINC API) LHC@home Dr Jukka Klem CHEP06

9. Some BOINC Projects • SETI@home: look for extraterrestrial life • Climateprediction.net: study climate change • Einstein@home: search for gravitational signals • Predictor@home: protein-related diseases • Rosetta@home: cures for human diseases • World Community Grid: IBM project • Africa@home: African humanitarian causes • LHC@home: improve LHC particle accelerator LHC@home Dr Jukka Klem CHEP06

10. LHC@home applications • Main application: SixTrack. Others prepared: ATLAS fast simulation, Garfield, Geant4 (poster presentation) • SixTrack application simulates protons as they travel around the LHC ring • Superconducting magnets generate unwanted multipole field errors, available phase space area (dynamic aperture) for stable particle motion limited • Each job typically tracks 60 particles 105 or 106 turns in the LHC (1-10 hours on a modern PC) • SixTrack program is part of SPEC CPU2000 benchmark suite LHC@home Dr Jukka Klem CHEP06

11. LHC@home Physics Results • Phase space images of stable particle motion (up) and unstable chaotic motion (down). • Chaotic motion predicts that the particle will be lost from LHC. • Map out conditions under which particle motion is stable. LHC@home Dr Jukka Klem CHEP06

12. LHC@home Physics Results • Long range and head-on beam-beam interactions reduce dynamic aperture • Different beam crossing schemes and effect of triplet errors studied • Average stable phase space area (Dynamic Aperture, DA) for different tune values. LHC@home Dr Jukka Klem CHEP06

13. LHC@home Physics Results • These studies would not have been possible without LHC@home resources • Large number of parameters can be carefully studied • Results used in LHC design to provide more interesting collisions for the experiments LHC@home Dr Jukka Klem CHEP06

14. Precision of Numerical Results (1/2) • LHC@home is a heterogeneous distributed system: different processors and operating systems • Redundant computing (each job sent to different computers) allows to find differences in results • IEEE-754 standard for floating-point arithmetic helps but does not specify everything needed (logarithm, trigonometric functions etc.) • Getting correct results depends on processor, operating system, programming language, compiler • Systems often optimized for performance LHC@home Dr Jukka Klem CHEP06

15. Precision of Numerical Results (2/2) • If particle motion chaotic, small errors can lead to large differences in final results • Some PCs give consistently wrong results (e.g. 10 year old desktop PC and one Linux batch PC) • Many small differences in results found (due to log and exp functions on different processors) • Solution: link executable statically to a portable library crlibm (https://lipforge.ens-lyon.fr/projects/crlibm/). Provides correct results on different platforms with performance cost less than 10%. • Need to be very careful if consistent results expected from heterogenous resources LHC@home Dr Jukka Klem CHEP06

16. Some Result Statistics • LHC@home has about 14000 active users in 108 countries • Users contribute about 25000 host computers • >800 CPU years processed for the LHC (assuming 1 CPU = 1 KSfp2K = 2.8 GHz Xeon) • BOINC projects combined: about 200.000 users and 350.000 hosts • Estimate: 1 billion PCs in operation (less than 0.05% participate) LHC@home Dr Jukka Klem CHEP06

17. BOINC and Grids • Bridges between BOINC and grid have been built for LCG and NorduGrid/ARC grid middleware • Sending jobs from BOINC to grid easier, grid to BOINC more difficult (security) • BOINC has lightweight infrastructure, some limitations with applications LHC@home Dr Jukka Klem CHEP06

18. Useful links • LHC@home server: http://lhcathome.cern.ch/Please join the project! • Some background information: http://athome.web.cern.ch/athome/LHCathome/whatis.html • BOINC web site: http://boinc.berkeley.edu/ • Unofficial BOINC Wiki: http://boinc-doc.net/boinc-wiki/ LHC@home Dr Jukka Klem CHEP06

19. Summary • Public resource computing projects built using BOINC platform • Can obtain large resources with low cost • Results very useful for the LHC • Numerical results checked with redundant computing. Have to be careful when using heterogeneous resources LHC@home Dr Jukka Klem CHEP06