1 / 15

EGEE – A Large-Scale Production Grid Infrastructure

EGEE – A Large-Scale Production Grid Infrastructure. Erwin Laure EGEE Technical Director CERN, Switzerland. Contents. The EGEE Project Operational Aspects Application Usage Interoperability, Standardization Outlook. The EGEE project. Objectives

ivana
Télécharger la présentation

EGEE – A Large-Scale Production Grid Infrastructure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EGEE – A Large-Scale Production Grid Infrastructure Erwin Laure EGEE Technical Director CERN, Switzerland

  2. Contents • The EGEE Project • Operational Aspects • Application Usage • Interoperability, Standardization • Outlook GGF 16 - February 2006

  3. The EGEE project • Objectives • Large-scale, production-quality infrastructure for e-Science • leveraging national and regional grid activities worldwide • consistent, robust and secure • improving and maintaining the middleware • attracting new resources and users from industry as well as science • EGEE • 1st April 2004 – 31 March 2006 • 71 leading institutions in 27 countries, federated in regional Grids • EGEE-II • Proposed start 1 April 2006 (for 2 years) • Expanded consortium • > 90 partners in 32 countries (also non-European partners) • Related projects (such as BalticGrid, SEE-GRID, EUMedGrid, EUChinaGrid, EELA, …) GGF 16 - February 2006

  4. EGEE Infrastructure Scale > 170 sites in 39 countries > 17 000 CPUs > 5 PB storage > 10 000 concurrent jobs per day > 60 Virtual Organisations GGF 16 - February 2006

  5. Applications >20 supported applications from 7 domains • High Energy Physics • Biomedicine • Earth Sciences • Computational Chemistry • Astronomy • Geo-Physics • Financial Simulation Another 8 applications from 4 domains are in evaluation stage GGF 16 - February 2006

  6. From Accounting data: • ~3 million jobs in 2005 so far • Sustained daily rates (per month Jan – Nov 2005): [2185, 2796, 7617, 10312, 11151, 9247, 9218, 11445, 10079, 11124, 9491] • ~8.2 M kSI2K.cpu.hours  >1000 cpu years • Real usage is higher as accounting data was not published from all sites until recently 10,000 jobs /day GGF 16 - February 2006

  7. Some example uses LCG sustained data transfers using FTS; avg. 500 MB/s, peak 1000 MB/s Zeus collaboration at DESY WISDOM data challenge GGF 16 - February 2006

  8. EGEE Operations Structure • Operations Management Centre (OMC): • At CERN – coordination etc • Core Infrastructure Centres (CIC) • Manage daily grid operations – oversight, troubleshooting • “Operator on Duty” • Run infrastructure services • Provide 2nd level support to ROCs • UK/I, Fr, It, CERN, Russia, Taipei • Regional Operations Centres (ROC) • Front-line support for user and operations issues • Provide local knowledge and adaptations • One in each region – many distributed • User Support Centre (GGUS) • In FZK: provide single point of contact (service desk), portal GGF 16 - February 2006

  9. Operations Process • CIC – on – duty (grid operator on duty) • 6 teams working in weekly rotation • CERN, IN2P3, INFN, UK/I, Ru,Taipei • Crucial in improving site stability and management • Operations coordination • Weekly operations meetings • Regular ROC, CIC managers meetings • Series of EGEE Operations Workshops • Last one was a joint workshop with Open Science Grid • Bring in related infrastructure projects – coordination point • Geographically distributed responsibility for operations: • There is no “central” operation • Tools are developed/hosted at different sites: • GOC DB (RAL), SFT (CERN), GStat (Taipei), CIC Portal (Lyon) • Procedures described in Operations Manual • Infrastructure planning guide (cookbook): • Contains information on operational procedures, middleware, certification, user support, etc. Improvement in site stability and reliability is due to: • CIC on duty oversight and strong follow-up • Site Functional Tests, Information System monitor GGF 16 - February 2006

  10. Applications Example: LHC • Fundamental activity in preparation of LHC start up • Physics • Computing systems • Examples: • LHCb: ~700 CPU/years in 2005 on the EGEE infrastructure • ATLAS: over 10,000 jobs per day Note the usage of 3 Grid infrastructures LHCb ATLAS GGF 16 - February 2006

  11. Applications Example: WISDOM • Grid-enabled drug discovery process for neglected diseases • In silico docking • compute probability that potential drugs dock with target protein • To speed up and reduce cost required to develop new drugs • WISDOM (World-wide In Silico Docking On Malaria) • First biomedical data challenge • 46 million ligands docked in 6 weeks • Target proteins from malaria parasite • Molecular docking applications: Autodock and FlexX • ~1 million virtual ligands selected • 1TB of data produced • 1000 computers in 15 countries • Equivalent to 80 CPU years • Never done for a neglected disease • Never done on a large scale production infrastructure GGF 16 - February 2006

  12. Grid Interoperability • We currently see different flavors of Grids deployed worldwide • Because of application needs, legacy constraints, funding, etc. • Diversity is positive! – Competition to find the best solutions • Many applications need to operate on more than one Grid infrastructure • Pragmatic approach to interoperability is key • Applications need interoperable Grid infrastructures now • A production infrastructure cannot be an early adopter of quickly changing standards • EGEE is active contributor to interoperability and standardization efforts • Works with OSG, NAREGI, ARC, and the multi-grids interoperability effort • Provides valuable input on practical experiences to standardization process • Contributes to over 15 GGF WG/RG • Currently supplies two GGF area directors GGF 16 - February 2006

  13. Leaders from TeraGrid, OSG, EGEE, APAC, NAREGI, DEISA, Pragma, UK NGS, KISTI will lead an interoperation initiative in 2006. Six international teams will meet for the first time at GGF-16 in February 2006 Application Use Cases (Bair/TeraGrid, Alessandrini/DEISA) Authentication/Identity Mgmt (Skow/TeraGrid) Job Description Language Newhouse/UK-NGS Data Location/Movement Pordes/OSG Information Schemas Matsuoka/NAREGI Testbeds Arzberger/Pragma Grid Interoperation Interoperability workshop Monday and Tuesday afternoon “AuthZ interoperability here and now” Thursday morning Leaders from nine Grid initiatives met at SC05 to plan an application-driven “Interop Challenge” in 2006. GGF 16 - February 2006

  14. Future • EGEE-II supposed to start on April 1st • Smooth transition from current phase • Additional support for more application domains • Increased number of partners, also from US and AP • Unified EGEE middleware distribution gLite 3.0 • Continue and reinforce interoperability work • Standardization in GGF, other bodies, and industry important for long term sustainability of Grid Need for commonly accepted standards • Towards a long-term sustainable Grid infrastructure GGF 16 - February 2006

  15. Upcoming EGEE Events • Series of regional long term sustainability workshops in Europe during March • For more information get in contact with project-eu-egee-po@cern.ch • 1st EGEE User Forum, CERN, 1-3 March, 2006 • Bring together the EGEE community • Exchange experience and needs on EGEE • http://cern.ch/egee-intranet/User-Forum • EGEE-II conference, Geneva, 25-29 September, 2006 • Include sessions with related projects • Theme will be long term sustainability GGF 16 - February 2006

More Related