1 / 11

WLCG Middleware Support II

CERN-IT-GT. May 2011. WLCG Middleware Support II. Markus Schulz. Overview. Status Problems? What do we need?. Situation on the Factory Floor. Fundamentally not much changed since the last discussion The 3.5 Empires are alive and working OSG

hanae-william
Télécharger la présentation

WLCG Middleware Support II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CERN-IT-GT May 2011 WLCG Middleware Support II Markus Schulz

  2. Overview Markus Schulz Status Problems? What do we need?

  3. Situation on the Factory Floor • Fundamentally not much changed since the last discussion • The 3.5 Empires are alive and working • OSG • Manages their releases independently of EMI • NDGF • ARC still rules • dCache • gLitereleases include dCache • Very limited usage (none) • dCache(DESY) produce their own releases • These are used and are popular • Discussed at the WLCG-T1-Service Coordination M. (WTSCM) Markus Schulz

  4. Situation on the Factory Floor • gLite: gLite-3.2/1 • gLite-3.1 (SL4) • On the way out • gLite-3.2 (SL5) • De facto coordination by Maria Alandes (CERN) • Patch prioritisation in WLCG T1 Service Coordination Meeting • EGI • Coordinates gLite-3.2/3.1 Staged Rollout • EMI • Prepares the first EMI-1 release ( based on EPEL ) • Many structural changes Markus Schulz

  5. What we (WLCG) assumed • Many in WLCG assumed (naively) that EGI and EMI are some form of EGEE-x • Assumed that long standing request would stay on the work plan • Assumed that the privileged partnership between project, sites, experiments and WLCG would just happen (TMB/TCG) • Assumed that established informal exchange of change requests between experiments and developers would continue and drive the project Markus Schulz

  6. Problems? • Requirements: • EMI and EGI did what they stated in their work plans • Defined and documented processes to gather requirements and change requests • Deliverable and milestone documents • Circulated and discussed within the projects • Implemented these processes • Captured some old requirements (difficult) • Gathered requirements (NGIs, etc.) • Prioritised them • Exchanged them between the projects • Wrote work plans for the next year(s) • As a result EGI/EMI priority lists and WLCG expectations are not in good agreement • Resulting in discussions… • Example: • CREAM-CE HA • passing of arguments to the batch system Markus Schulz

  7. What happened II • For several components the direct interaction between sites, experiments and developers continued ( in a twilight zone) • ATLAS catalogue work • Infosystem • Monitoring • Condor/Cream etc. • FTS • Often not explicitly clear whether this is WLCG or EMI related Markus Schulz

  8. The Problems • EMI and EGI are strategic projects • Planning over long periods • Strong processes • Long term investment • EPEL, source RPMs, Debian Support • Less rooted in the past • LCG will benefit from the strategic goals • LCG frequently has tactical needs • Often discovered in production • Rarely whiteboard solutions • Iterative solutions • Sometimes treating the symptoms • Needs rapid reaction • LCG didn’t go through a re-birth and has memory Markus Schulz

  9. WLCG Needs • Understanding what information has been lost • Complete the official requirement list • Effort from both sides needed • A more continuous interaction between projects and WLCG • The experiments use >80% of all resources • Privileged partnership? • Influence on the priorities of the development effort? • WLCG needs to communicate needed middleware changes (also for relative fast changes) • For core components • Has to contain rollout plan ( including T2s) • Currently done for glexec and Cream-CE via WLCG-MB • Documentation and dissemination needed Markus Schulz

  10. WLCG questions • How to manage the middleware effort outside EMI? • Example: Support for extra batch systems • Several teams have resources that are not part of EMI • Do we need to formalize the direct communication between sites/users/developers? • How do we deliver the resulting changes? • Is there a risk of conflict between EGI/NGI and WLCG? • T1s, T2s, who decides what versions have to be run? • WLCG roadmap for SL6 migration. • A bit early……. • Do we have to take the LHC machine schedule into account? • Not very stable and computing doesn’t stop with the beam Markus Schulz

  11. Summary Markus Schulz How do we get the lost requirements back? How do we establish a fast feedback loop? How do we manage WLCG middleware work?

More Related