1 / 30

Pre-production in EGEEIII: Mandate, status and mid-term planning

Pre-production in EGEEIII: Mandate, status and mid-term planning. Antonio Retico SA1 Coordination Meeting CERN, Geneva 10 Jun 2008. Contents. Good Morning. The PPS Mandate in EGEE III What changed and why PP Service in EGEE III The way we want to be Resources What we need Planning

jana
Télécharger la présentation

Pre-production in EGEEIII: Mandate, status and mid-term planning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pre-production in EGEEIII: Mandate, status and mid-term planning Antonio ReticoSA1 Coordination Meeting CERN, Geneva 10 Jun 2008

  2. Contents Good Morning The PPS Mandate in EGEE III • What changed and why PP Service in EGEE III • The way we want to be Resources • What we need Planning • What do we do this summer EGEE-SA1 Coordination Meeting – 10th June 2008

  3. The PPS Mandate in EGEE III What changed and why EGEE-SA1 Coordination Meeting – 10th June 2008

  4. From the EGEE public web Preproduction Service: This service consists of a limited number of sites running a preview of the next release of the gLite software. This should be used to test existing applications against the new release, and to understand new gLite services. (http://technical.eu-egee.com/index.php?id=134) EGEE-SA1 Coordination Meeting – 10th June 2008

  5. Mandate in EGEEII The EGEE Pre-Production Service (PPS) is a distributed service whose goal is to give early access to new services to WLCG/EGEE users in order to evaluate new features and changes in the release. As a contribution to the overall quality of the gLite middleware the PPS provides • Evaluation of the deployment procedure and software in real operational conditions • Evaluation and debugging of new functionality done by the applications/VOs before they move into the production service • Feedback for early bug fix to the release before moving to production EGEE-SA1 Coordination Meeting – 10th June 2008

  6. Mandate in EGEEIII The EGEE Pre-Production provides access to grid services in preview to WLCG/EGEE interested users, in order to test, evaluate and give feedback to changes and new features of the middleware. In addition to that, the pre-production extends the middleware certification activity, helping to evaluate deployment procedures, [inter]operability and basic functionality of the software against operational scenarios reflecting real production conditions EGEE-SA1 Coordination Meeting – 10th June 2008

  7. Drivers For Change Promote Usage Optimise Work Reduce Costs EGEE-SA1 Coordination Meeting – 10th June 2008

  8. One Key Change Key change: not EVERY user at EVERY time. Setup and usage conditions of previews to be agreed with users in advance The EGEE Pre-Production Service (PPS) is a distributed service whose goal is to give early access to new services to WLCG/EGEE users in order to evaluate new features and changes in the release. The EGEE Pre-Production provides access to grid services in preview to WLCG/EGEE interested users in order to test, evaluate and give feedback to changes and new features of the middleware. EGEE-SA1 Coordination Meeting – 10th June 2008

  9. One Key Change • Promote Usage  A service is set-up only when a real interest in testing is expressed by the VOs • Optimise Work  The preview (or pilot) services are successful and they would be set-up anyway out of PPS • Reduce Costs  No more permanent platform with all services The EGEE Pre-Production Service (PPS) is a distributed service whose goal is to give early access to new services to WLCG/EGEE users in order to evaluate new features and changes in the release. The EGEE Pre-Production provides access to grid services in preview to WLCG/EGEE interested users in order to test, evaluate and give feedback to changes and new features of the middleware. EGEE-SA1 Coordination Meeting – 10th June 2008

  10. PP Service in EGEE III The way we want to be EGEE-SA1 Coordination Meeting – 10th June 2008

  11. Two Service Classes + Support Corresponding to the two statements in the mandate Middleware Quality Services – MQS Middleware Preview Services – MPS PPS Support - SUP EGEE-II/EGEE-III Transition Meeting - 8 May 2008

  12. Service breakdown PPS on Sept2008 PPS today MQS (PPS grid) pre-deployment tests (~12 sites from all ROCs) test coordinators per service SAM service (1 site) Release testing (?) Reference Site (grid-on-a-site) • General coordination (CERN) • pre-dep test coordinator (LIP-SWE) • pre-deployment tests (6 sites from CERN(1), CE(1), RU(1), SWE(3) • 14 PPS sites running basic service (all ROCs except NE) • SAM service (2 sites from UKI and CE) • PPS release managers (2 sites from SWE, IT) • EMT: release to production (CERN) • AMGA pilot service (CERN) • Cream pilot service, starting (SWE, IT, DECH, FR) • Pilot coordination (CERN) MPS (prod grid) • Client distribution tool (1 site) • List of production sites available to support pilots (‘PPS partners’) • Pilot Services in production … (on demand) • Pilot coordinators to be appointed PPS support PPS release management EMT: release to production (CERN) Activity management Metrics and quality Communication EGEE-SA1 Coordination Meeting – 10th June 2008

  13. Middleware Quality Services Mission: “To test the middleware deployment tools (packaging, documentation) against scenarios relevant for production” Workload distributed among 'PPS' sites (not registered as production) Services published in pre-production IS Testing interaction with different platforms, batch and storage systems Contributing to interoperability testing Providing additional info and advice for deployment in production Dedicated monitoring infrastructure for validation "service-oriented" testing: several deployment test managers sharing the tools NOT production-like service --> pre-deployment runs on-demand with releases EGEE-SA1 Coordination Meeting – 10th June 2008

  14. Middleware Preview Services Mission: “To provision previews of new middleware functionality” • Distribution of clients tools in preview mostly automated • Instances of pilot services created on-demand • Close interactions with the VOs for the definition of scope and goals of the pilots • Workload distributed among partner sites in production • Services published in production IS • Production-like service of limited duration • Pre-registration of sites to sets of tasks (PPS Partner Sites) • Silver Partners (or Silver Sites): Sites supporting the installation of non-backward-compatible client updates • Gold Partners (or Gold Sites): Sites providing support for pilots services in case of backward-compatible server updates or new services • Platinum Partners (or Platinum Sites): Sites providing support for pilots services in case of non-backward-compatible server updates • Details in https://twiki.cern.ch/twiki/bin/view/LCG/PreProductionServiceDescription EGEE-SA1 Coordination Meeting – 10th June 2008

  15. PPS Support Mission: “To support the work of the MPS and MQS and be the interface of PPS with the external world” MQS: • Maintain the release procedure • Set requirements for deployment scenarios coming from middleware development/certification and "global" VOs • Interface to EMT for release to production MPS • Interface with VOs, regions, sites and middleware providers to start-up pilots • Access point for PPS users Technical coordination • Activity tracking and statistics • Reporting Communication and outreach EGEE-SA1 Coordination Meeting – 10th June 2008

  16. Resources from the Regions What we need EGEE-SA1 Coordination Meeting – 10th June 2008

  17. Effort estimates One of the overall drivers of the re-organisation was to minimise the permanent allocation of resources dedicated to PPS Resources at sites are mainly called to: • Support pre-deployment test • Support pilot services The overall cost of the future PPS varies a lot with its usage • number of releases of glite services • number and complexity of the pilot services EGEE-SA1 Coordination Meeting – 10th June 2008

  18. Fixed costs SAM client • Currently done by RAL-PPS and CYFRONET • In future reduced to one client (but higher customisation needed) • 8 PH (Person Hours) per week (= 0.2 FTE) Client distribution tool • Single instance • Same as SAM client (roughly) Coordination • Regular Meetings and Reporting: 3PH per week (<0.1 FTE) EGEE-SA1 Coordination Meeting – 10th June 2008

  19. Variable costs Release management • PPS release coordination 3PH per update (to decrease) • PPS repository management 3PH per update • EMT and release to production (SA1 share): 8PH per update Pre-deployment testing • 1 service upgrade, initial testing and reporting: 3PH per update Pilot Services (estimate for a 3-weeks pilot) • PPS Coordination: 1.25PD (Person Day) • Gold Site: 5.5PD • Service expert support: 6PD (to be counted if provided by the Gold Site or SA1 personnel) Numbers available and explained in https://twiki.cern.ch/twiki/bin/view/LCG/PreProductionServiceDescription Difficult (and perhaps pointless) estimate EGEE-SA1 Coordination Meeting – 10th June 2008

  20. Measuring Activity As EGEE does not distinguish between PPS and PROD resources regions are not forced to allocate permanently part of their resources to PPS It would be good to level the contribution to PPS by the regions to the same fraction of their SA1 resources • E.g. all regions could be using 10% of their SA1 resources to support PPS • This quota could increase or decrease in time according to the PPS usage We propose to track all PPS activity via Savannah tasks of different weight and use them as accounting records. The objective of the activity management is: • To make sure that regions are accounted for work done in PPS • To try and keep the contribution of the regions balanced EGEE-SA1 Coordination Meeting – 10th June 2008

  21. Planning What do we do this summer EGEE-SA1 Coordination Meeting – 10th June 2008

  22. 6 major things to do MQS • Extend pre-deployment testing • Decommission permanent service MPS • Review Release Procedure • Set-up Client Preview Mechanism SUP • Set-Up Activity Management and Reporting System • Documentation (service description, website, EGEE08) This is in addition to the standard activity(pilots, pre-depl. etc.) EGEE-SA1 Coordination Meeting – 10th June 2008

  23. MQS: Extend pre-deployment testing EGEE-SA1 Coordination Meeting – 10th June 2008 Find gaps in the current testing Re-convert existing PPS sites to cover gaps Appoint service test managers Adapt tools for test reports Start Operations Start date: today Due date: 21 Jul Task Coordinator (proposed): Mario David Team: Antonio (CERN); Esteban, Alvaro (CESGA) Regions/sites concerned: ALL

  24. MQS: Decommission Permanent Service EGEE-SA1 Coordination Meeting – 10th June 2008 Disconnect sites not used in pre-deployment Notify concerned Users Start date: 29th Jun Due date: 13th Aug Task Coordinator (proposed): Antonio Team: - Regions/sites concerned: ALL

  25. MPS: Review Release Procedure EGEE-SA1 Coordination Meeting – 10th June 2008 Define new use of CNAF repository • To be used for pilot services Cut 2nd part of the procedure (Upgrade of PPS Sites) Start date: today Due date: 3rd Jul Task Coordinator (proposed): Danilo Dongiovanni (CNAF) Team: Antonio, Esteban, Alvaro, Mario Regions/sites concerned: CERN, SWE, IT

  26. MPS: Set-up clients preview EGEE-SA1 Coordination Meeting – 10th June 2008 Set-up pilot instance of tool for client distribution Indentify sites in production for BC and NBC client updates Identify site to run the tool once it’s set up Start operating client release at CERN_PPS Export changes to concerned production sites Review PPS Release Procedure accordingly Integrate changes with the overall release process Follow-up decommissioning of existing PPS UIs and WNs Start date: 24th Jun Due date: 21st Aug Task Coordinator (proposed): Antonio Team: Andreas (from SA3), Mario Regions/sites concerned: initially CERN and LIP, then others

  27. SUP: Set-up activity mgt system EGEE-SA1 Coordination Meeting – 10th June 2008 Create and customise Savannah project Define tasks and task weights Populate project users database Develop tools for automation of task creation and assignment Start operations and fine-tuning Develop tools for reporting and accounting Start date: 19th Jun Due date: 5th Sep Task Coordinator (proposed): Antonio Team: ? Regions/sites concerned: Initially CERN, ?

  28. SUP: Documentation EGEE-SA1 Coordination Meeting – 10th June 2008 Service Description • Finish implementation of use cases and effort estimate (URGENT!) Website • Changes in the mandate (urgent) • New usage rules (moderately urgent) • New service layout (later) EGEE08 • Contribution “The New PPS” (later) Start date: today Due date: 16th Sep Task Coordinator (proposed): Antonio Team: ? Volunteers welcome Regions/sites concerned: CERN

  29. For Further Reading • WLCG/EGEE Pre Production: Use Cases (EGEEIII) https://twiki.cern.ch/twiki/bin/view/LCG/PreProductionUseCases • WLCG/EGEE Pre Production: Service Description (EGEEIII) https://twiki.cern.ch/twiki/bin/view/LCG/PreProductionServiceDescription • Available in the PPS web site www.cern.ch/pps/index.php?dir=./PPSrocs/ EGEE-II/EGEE-III Transition Meeting - 6 May 2008

  30. Questions? ? EGEE-II/EGEE-III Transition Meeting - 6 May 2008

More Related