1 / 28

Project Report

Project Report. Alberto Di Meglio, Project Director, CERN ETICS 2 nd EU Review CERN, Geneva - 15 February 2008. Introduction. Overall project report, providing an overview of: Review Agenda Project Goals Achievements First Review Recommendations Workpackages Status Summary

williami
Télécharger la présentation

Project Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project Report Alberto Di Meglio, Project Director, CERN ETICS 2nd EU Review CERN, Geneva - 15 February 2008

  2. Introduction • Overall project report, providing an overview of: • Review Agenda • Project Goals • Achievements • First Review Recommendations • Workpackages Status Summary • Deliverables and milestones • Financial status • Manpower levels 2nd ETICS Review – Project Report - CERN, 15 February 2008

  3. Review Agenda Morning 09:00 - Welcome Frédéric Hemmer (CERN) 09:15 – Project ReportAlberto Di Meglio (CERN) 10:00 – WP1 (Management) Alberto Di Meglio (CERN) 10:15 – WP2 (Infrastructure)Miron Livny (Univ . of Wisconsin) 11:15 – WP3 (Build System) Elisabetta Ronchieri (INFN) 11:45 – WP4 (Test System)Eva Takacs (4D SOFT) 12:15 – WP5 (Dissemination, certification and web tools) Marc-Elian Begin (CERN) Afternoon 14:00 - GridQCMAdriano Rippa (Engineering) 13:30 - Demos Lorenzo Dini (CERN), Tomasz Kokoszka (CERN), Paolo Fabriani (Engineering) 16:00 - Reviewers Closed Session followed by Feedback 2nd ETICS Review – Project Report - CERN, 15 February 2008

  4. Project Goals To provide within a two year programme: • A professionally managed, integrated system of services and resources for running automated builds and test suites for distributed software projects • A repository of packages, test reports and interoperability information that development projects can use to develop, validate and certify their products • A forum where to discuss and promote software quality initiatives and collect common software engineering tools and processes, software configuration information and documentation. 2nd ETICS Review – Project Report - CERN, 15 February 2008

  5. Build system, software configuration, service infrastructure, dissemination, EGEE, gLite, project coord. Software configuration, service infrastructure, dissemination Web portals and tools, quality process, dissemination, DILIGENT The Condor batch system, distributed testing tools, service infrastructure, NMI Test methods and metrics, unit testing tools, EBIT The ETICS Partners 2nd ETICS Review – Project Report - CERN, 15 February 2008

  6. Project Structure 2nd ETICS Review – Project Report - CERN, 15 February 2008

  7. Project Structure Project Manager (PM) Project Management Board Project Technical Committee WP3 WP2 WP5 WP4 2nd ETICS Review – Project Report - CERN, 15 February 2008

  8. 1st Review Recommendations 1 To establish a stronger collaboration with other projects (not only EGEE and DILIGENT) and to promote the use of the ETICS services by as many projects as possible from the e-Infrastructure Unit and else (to defend the use of the SSA instrument). Especially it is proposed to cooperate closely with OMII-Europe towards the establishment of a “European scientific repository of software tools and reusable components”. As it concerns the collaboration with EGEE and DILIGENT, ETICS has to achieve that its services are used for all build, testing, configuration and distribution of their software 2nd ETICS Review – Project Report - CERN, 15 February 2008

  9. 1st Review Recommendations In the second year of activity ETICS has expanded its interaction with many additional projects, such as Health-e-Child, GridSphere, Beingrid, EUChinaGrid, Globus, ETSI, IceAge. In some cases the collaboration resulted in practical engineering activities (EUChinaGrid, Globus, GridSphere), in other cases in dissemination or standardization or training activities (ETSI, IceAge) The collaboration with OMII-Europe has been as strong as possible. The OMII-Europe repository is linked to the ETICS Build and Test Repository providing a single point of access to OMII-Europe users. EGEE and DILIGENT are now using ETICS for all build activities and most testing activities and EGEE in particular is distributing its software creating on-line deployment servers directly from the ETICS Repository 2nd ETICS Review – Project Report - CERN, 15 February 2008

  10. 1st Review Recommendations 2 To promote a Software “Quality Certification Process” that projects may want to adopt in order to assess and promote their products • One of the major achievements ETICS is the definition of the so-called Grid Quality Assurance Model (GridQCM) • More details on GridQCM and the active dissemination activities around it are provided later today in a dedicated presentation 2nd ETICS Review – Project Report - CERN, 15 February 2008

  11. 1st Review Recommendations 3 To intensify scientific production and contribution to standards • The scientific production of ETICS in the second year has been high, several papers have been produced and presented at international conferences or published on peer-reviewed journals • The contribution to standards is more in the QA process area. Again the GridQCM plays an important role. • More details on this subject in the dedicated presentations about dissemination and the GridQCM model 2nd ETICS Review – Project Report - CERN, 15 February 2008

  12. 1st Review Recommendations 4 To structure and integrate the users’ feedbacks and significantly improve the user interfaces • During the second year of activity ETICS has conducted several training and dissemination events, collecting user feedback • An on-line questionnaire on usage of the system has been prepared and has been filled by more than 40 users (about 1/3 or the current user base) • All replies and suggestions have been categorized and used to develop new requirements • The Web Application has been redesigned following user recommendations • The client performance and usability has been improved and more improvements are under development • Quotes: • “I've checked briefly the web, the improvement there compared to the end of last week is huge. Altogether, my feedback is quite positive this time ;-).” • “Congratulations for the new version of the etics client and the implementation of the cache in etics-build. Now it’s usable out-of-the-box” • “This new Etics portal is much better than the previous one, it seems faster and better arranged ! “ 2nd ETICS Review – Project Report - CERN, 15 February 2008

  13. 1st Review Recommendations 5 To pay emphasis to elements which are crucial for future commercial viability, such as charging, registration, service level agreements, extensibility, scalability and use of third party resources to run the ETICS services • The elements mentioned in the recommendation are of great importance to ETICS. The following activities have taken place: • User registration is one of the services provided by ETICS. A user registration application is provided to control access to the service and its features • Project registration and SLAs: ETICS policies in the first phase has been ‘open’ due to user requirements. Users have so far asked to have less formal relationships. Now that the services are stable and used, the policy is going to change in ETICS 2. MoU are being agreed for example with EGEE and 4DScience and will be established with other infrastructures like DEISA • Extensibility is based on the ‘plugin framework’, which allows to plug in the system additional tools or functionality. It is based on a published interface that users can also adopt to add custom tools (more details in WP4 presentation) • Scalability is based on the cross-site submission feature, which allows to create distributed pools of resources and automatically dispatch jobs where resources are available. For the metadata, a move to an Oracle back-end is foreseen in the future • Third-party resources can be plugged in directly into a resource pool or using the cross-site submission. An example implemented with EUChinaGrid is described in the WP2 presentation 2nd ETICS Review – Project Report - CERN, 15 February 2008

  14. 1st Review Recommendations 6 To involve at least two projects to validate the testing components • The testing components have been validated by all major projects using ETICS. More details are given in the WP2 and WP4 presentations and in the demos. In summary: • EGEE performs routinely code tests, unit tests, deployment tests and some regression testing with ETICS, Usage of ETICS is limited by the availability of automated tests, not by ETICS functionality • DILIGENT performs routinely code tests, unit tests, deployment tests and regression tests with ETICS • OMII-Europe has based its entire test strategy on ETICS and performs standard compliance and interoperability tests with ETICS • EGEE and EUChinaGrid have implemented with ETICS an IPv6 interoperability infrastructure distributed between CERN, IN2P3 and GARR to test IPv6 code compliance and regression testing in IPv6 environments 2nd ETICS Review – Project Report - CERN, 15 February 2008

  15. 1st Review Recommendations 7 To justify the cost acceleration, if the project plans to recover the current underspent • The budget carried over to year two has been used to recruit additional effort for the tasks that were affected by the initial delay • In particular INFN has recruited an additional person for 12 months to work on interoperability and development of unit tests and CERN has recruited a technical student for 3 months to work on deployment testing tools, which compensate the underspent • In the case of Engineering the underspent in year 1 was done as planned, since the Quality Assurance process feasibility study where Engineering has been mostly involved is a year 2 activity 2nd ETICS Review – Project Report - CERN, 15 February 2008

  16. 1st Review Recommendations 8 For some of the above recommendations to be well planned, it is proposed to prepare between month 16 and month 18 a draft version of the following deliverables (originally planned on or after month 20): D4.3, D5.7, D5.9. It is up to the project management to propose a time schedule for such drafts. The final delivery remains at the planned months • The drafts have been prepared and submitted to the EC as per recommended schedule 2nd ETICS Review – Project Report - CERN, 15 February 2008

  17. What has ETICS delivered? • Infrastructure • Three resource sites at CERN, INFN and UoW with more that 150 CPUsand more than 15 platforms. Additional third-party resources from OMII-Europe, GARR (EUChinaGrid), IN2P3 (EGEE/SA2) and DILIGENT • A Service Level Agreement and documented deployment procedures • Services • A set of web services and tools for configuring, building and testing software • Web and multiplatform CLI clients • A full repository of reports and packages with connectors for other types of repositories like APT and OMII-Europe • The ‘co-scheduling’ feature that allows automated deployment and execution of complex multi-node tests • Community Support • Build and test support for more than 25 projects and in particular EGEE (gLite, mpi, GridWay), DILIGENT and OMII-Europe • A structured user support mechanism organised in weekly shifts with dedicated personnel to handle user tickets • Dissemination and Training • Five major training events (EGEE’06, DILIGENT TCOM, INFN-CNAF in 2006, OGF19,JRA1 All-Hands in 2008) • A Build &Test Community Group in OGF and participation to several conference and events presenting original scientific and technical content 2nd ETICS Review – Project Report - CERN, 15 February 2008

  18. ETICS Portal (myETICS, Build/Test, Repository, Administration) Web Service Repository Service Architecture Via browser Project DB Build/Test Artefacts Report DB Metronome Execution Engine Clients Metronome Client Wrapper WNs ETICS Infrastructure 2nd ETICS Review – Project Report - CERN, 15 February 2008

  19. The ETICS Timeline (Year 1) OGF CCG Launched July 2006 ETICS Beta programme started (gLite 3.1 managed only with ETICS), DILIGENT using for testing, OMII-Europe project created February 2007 ETICS Service v1.0 Released April 2006 ETICS Alpha programme started (first gLite components registered) 11 Registered Projects • 1st Review All-hands meeting(Bologna) September 2006 ETICS Pre-release programme started, first major training event in Geneva All-hands meeting(Budapest) • Kick-off Jan 06 Jun 06 Dec 06 2nd ETICS Review – Project Report - CERN, 15 February 2008

  20. The ETICS Timeline (Year 2) November 2007 ETICS Service v2.0 Released Co-scheduling development started, first definition of GridQCM Repository Service Design New ETICS Portal Design 18 Registered Projects (+ several test and experimental projects) • 2nd Review OGF 19, Joint OMII-Europe Training All-hands meeting(CERN) October 2007 EGEE/JRA1 Training All-hands meeting(Wisconsin) Jan 07 Jun 07 Dec 07 2nd ETICS Review – Project Report - CERN, 15 February 2008

  21. Relations with Other Projects 2nd ETICS Review – Project Report - CERN, 15 February 2008 • In Year 2 ETICS has spent again considerable effort in establishing and maintaining relationships with other projects: • EGEE: SA3 (certification), JRA1 (development), NA4 (applications), SA2 (networks) • DILIGENT: integration and testing • OGF: co-chair of a Community Group on Build, Test and Certification of grid software • OMII-Europe: provider of the build and test infrastructure • EUChinaGrid: implementation of plugins for IPv6 compliance tests • ICEAGE: participation to the Grid Computing School 2007 • Health-e-Child: initial discussions about service testing • BeInGrid: on-going discussions on building and testing • ETSI: Standard compliance testing, IPv6 testing • ETICS has actively taken part to the EC/IST activities: • eIRG Workshops (Heidelberg) • IST Workshops and Concertation Meetings (Sophia Antipolis)

  22. Deliverables and milestones • All deliverables/milestones (PM13-24) done • 11 deliverables and 4 milestones in 12 months • Additional draft versions of three deliverables submitted to the Commission as per Reviewers’ recommendation n. 8 2nd ETICS Review – Project Report - CERN, 15 February 2008

  23. Deliverables List 2nd ETICS Review – Project Report - CERN, 15 February 2008

  24. Financial status (2nd Period) • The provisional Financial Report covering the 2nd reporting period shows a total cost of 944 877 € and requests a total EC contribution of 822 087 €. It represents 59% of the total maximum EC contribution as per the Annex I of the Contract. • Internal Cost Claims (Form C) have submitted by all partners to the Coordinator and from the Coordinator to the Commission. Audit certificates have been submitted. • The breakdown of expenses is as follows: • Direct Costs: 79% • Indirect Costs: 21% • Personnel: 89% • Travel, subsistence and other costs: 11% 2nd ETICS Review – Project Report - CERN, 15 February 2008

  25. Financial status (Full project) • The provisional Financial Report covering the full project duration shows a total cost of 1 551 769 € and requests a total EC contribution of 1 400 000 € as per the Annex I of the Contract. • The breakdown of expenses is as follows: • Direct Costs: 76% • Indirect Costs: 24% • Personnel: 87% • Travel, subsistence and other costs: 13% • TO BE UPDATED 2nd ETICS Review – Project Report - CERN, 15 February 2008

  26. Manpower levels • PPT tool to monitor project effort consumption (timesheets) • 28 people registered across Europe and USA • Total equivalent of ~ 10 FTE/year 2nd ETICS Review – Project Report - CERN, 15 February 2008

  27. Summary ETICS has closely followed the planned work program and it has delivered a fully functional distributed service for Building and Testing software used or considered by 18+ projects and more than 180 users It has exceeded the expected number of supported projects and users for the two years of its plan (originally 4, actual 18+) and has 3 major projects (EGEE (JRA1, SA3 and NA4), DILIGENT and OMII-Europe) using its services for production tasks ETICS has established strong relationships with other projects and initiatives in the field of software quality assurance and certification and has established itself as a reliable service provider and partner in this area The project is close to the expected levels of effort and budget and has completed all the planned tasks 2nd ETICS Review – Project Report - CERN, 15 February 2008

  28. Thanks http://www.eu-etics.org 2nd ETICS Review – Project Report - CERN, 15 February 2008

More Related