EGEE: A Grid Infrastructure for e-Science

1. EGEE: A Grid Infrastructure for e-Science European Information Space: Infrastructures, Services and Applications Workshop 29-30 October 2007, Rome, Italy Frédéric Hemmer Deputy Head, IT Department, CERN Frederic.Hemmer@cern.ch

2. eScience • Science is becoming increasingly digital, needs to deal with increasing amounts of data and computational needs • Simulations get ever more detailed • Nanotechnology –design of new materials from the molecular scale • Modelling and predicting complex systems (weather forecasting, river floods, earthquake) • Decoding the human genome • Experimental Science uses ever moresophisticated sensors to make precisemeasurements • Need high statistics • Huge amounts of data • Serves user communities around the world Infrastructures, Services & Applications - October 2007

3. Mont Blanc (4810 m) Downtown Geneva Accelerating and colliding particles Large Hadron Collider • One of the most powerful instruments ever built to investigate matter • 4 Experiments: ALICE, ATLAS, CMS, LHCb • 27 km circumference tunnel • Due to start up in 2007 • 40 Million Particle collisions per second • Online filter reduces to a few 100’s of “good” events per second • recorded on disk and magnetic tape at 200-1,000 MegaBytes/sec and distributed worldwide • ~15 PetaBytes per year for all four experiments • Data analyzed by 100s of research groups world wide Infrastructures, Services & Applications - October 2007

4. Challenges for high throughput virtual docking 300,000 Chemical compounds: ZINC & Chemical combinatorial library Millions of chemical compounds available in laboratories High Throughput Screening 2$/compound, nearly impossible Molecular docking (Autodock) 100s CPU years, TBs data Data challenge on EGEE, Auvergrid, TWGrid ~6 weeks on ~2000 computers In vitro screening of 100 hits Hits sorting and refining Target (PDB) : Neuraminidase (8 structures) Infrastructures, Services & Applications - October 2007

5. Sumatra, March 28, 2005 Mw=8.5 Peru, June 23, 2001 Mw=8.4 Example: Determining earthquake mechanisms • Seismic software application determines epicentre, magnitude, mechanism • Analysis of Indonesian earthquake (28 March 2005) • Seismic data within 12 hours after the earthquake • Analysis performed within 30 hours after earthquake occurred • Results • Not an aftershock of December 2004 earthquake • Different location (different part of fault line further south) • Different mechanism Rapid analysis of earthquakes important for relief efforts Infrastructures, Services & Applications - October 2007

6. Main trend The size of data an organization owns, manages, and depends on is dramatically increasing: • Ownership cost of storage capacity goes down • Data generated and consumed goes up • Network capacity goes up • Distributed computing technology matures and is more widely adopted Infrastructures, Services & Applications - October 2007

7. EGEE Main Objectives Operate a large-scale, production quality grid infrastructure for e-Science Attract new resources and users from industry as wellas sciences Maintain and further improve“gLite” Grid middleware Flagship grid infrastructure project co-funded by the European Commission Now in 2nd phase with 91 partners in 32 countries Infrastructures, Services & Applications - October 2007

8. EGEE – What do we deliver? • Infrastructure operation • Currently includes ~240 sites across 45 countries • ~ 40000 CPUs and ~5 PBytes of storage • Continuous monitoring of grid services & automated site configuration/management • Support>200 Virtual Organisations from diverse research disciplines • Middleware • Production quality middleware distributed under business friendly open source licence • Implements a service-oriented architecture that virtualises resources • Adheres to recommendations on web service inter-operability and evolving towards emerging standards • User Support - Managed process from first contact through to production usage • Training • Expertise in grid-enabling applications • Online helpdesk • Networking events (User Forum, Conferences etc.) Infrastructures, Services & Applications - October 2007

9. 240 sites 45 countries 41,000 CPUs 5 PetaBytes >5000 users >100 VOs >100,000 jobs/day Archeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences … 32% Infrastructures, Services & Applications - October 2007

10. Active VOs • Number of “active” VOs growing steadily! • Turnover: Diff. VOs in last 6 / 12 / 24 months = 83 / 92 / 102 • Total VOs: 104 registered, 258 visible Infrastructures, Services & Applications - October 2007

11. gLite Grid Middleware Services Access CLI API Security Information & Monitoring Authorization Auditing Information &Monitoring Application Monitoring Authentication Data Management Workload Management MetadataCatalog File & ReplicaCatalog JobProvenance PackageManager Accounting StorageElement DataMovement ComputingElement WorkloadManagement Site Proxy Overview paper http://doc.cern.ch//archive/electronic/egee/tr/egee-tr-2006-001.pdf Infrastructures, Services & Applications - October 2007

12. Grid Middleware Applications • Applications access both Higher-level Grid Services and Foundation Grid Middleware • VOs complement gLite with other high-level services via the RESPECT programme • Rec. External Software Pkgs. for the EGEE Community • Identify useful, 3rd-party software that works with gLite • Make users aware of that software to avoid duplicated efforts Higher-Level Grid Services Workload Management Replica Management Visualization Workflow Grid Economies ... Foundation Grid Middleware Security model and infrastructure Compute (CE) and Storage Elements (SE) Accounting Information and Monitoring Infrastructures, Services & Applications - October 2007

13. A Software architecture for digital libraries Infrastructures, Services & Applications - October 2007

14. Infrastructures geographical or thematic coverage Support Actions key complementary functions Applications improved services for academia, industry and the public Registered Collaborating Projects 25 projects have registered as of September 2007:web page Infrastructures, Services & Applications - October 2007

15. GIN EGEE working with related infrastructure projects Infrastructures, Services & Applications - October 2007

16. GIN Worldwide Grids APAC DEISA EGEE Naregi NDGF NGSOSG Pragma Teragrid Infrastructures, Services & Applications - October 2007

17. Business Partners Signatories of the project contract with responsibilities in EGEE-II • CEA (France) • Working with the Fusion community • CGGVeritas (France) • Seismic Imaging and Reservoir simulation – BEinGRID (BE18) • CRSA (France) • Finance - Risk Management in Finance – BEinGRID (BE11) • CS-SI (France) • Responsible for Q.A. and Technology Transfer to SMEs • Support Large enterprises • DATAMAT (Italy) • gLite middleware development & testing • Metaware (Italy) • Dissemination and outreach to Businesses and Industry • Leads the Industry Forum and organises Industry Events • TID (Spain) • Telecoms Infrastructures, Services & Applications - October 2007

18. Collaborations with Businesses • EGEE Business Associates Program • GridWiseTech • Developing Grid tutorials to Industrial Mgt. • Outreach to Business • NICE • Developing EGEE compatible GENIUS Grid portal and EnginFrame Framework • Platform Computing • Making gLite middleware better in exploiting the LSF local resources management system • Avanade • IT Consultancy • Interface gLite with MS Windows based AGA software • Excelian • UK-based Grid Consulting Company for Finance and Energy • Aid in adapting EGEE infrastructure to meet needs of the Finance & energy sectors • Hitachi Sophia-Antipolis Laboratory • French based technical lab • Integrate gLite data management with INCA middleware Infrastructures, Services & Applications - October 2007

19. Routine Usage Testbeds Utility Service Evolution National European e-Infrastructure Global Infrastructures, Services & Applications - October 2007

20. European Grid Initiative • Need to prepare permanent, common Grid infrastructure • Ensure the long-term sustainability of the European e-Infrastructure independent of short project funding cycles • Coordinate the integration and interaction between National Grid Infrastructures (NGIs) • Operate the production Grid infrastructure on a European level for a wide range of scientific disciplines Must be no gap in the support of the production grid Infrastructures, Services & Applications - October 2007

21. Summary • EGEE provides a dependable production quality Grid infrastructure to a wide variety of scientific disciplines. • Collaboration and interoperability are key to providing the level of support required by user communities • Grids are increasingly becoming an essential part of the scientific computing infrastructure – sustainability needs to be ensured Infrastructures, Services & Applications - October 2007

22. High level services and middleware perspectives Scientific results using grid Application porting and deployment Demos Infrastructures, Services & Applications - October 2007