1 / 23

Report on GLUE activities

GriPhyN. PPDG. iVDGL. Report on GLUE activities. Flavia Donno, INFN-Pisa EDG Retreat 27-28 August 2002. Summary . Resource Discovery and GLUE schema Authorization services from VO LDAP to VOMS Common software deployment procedures IST2002 and SC2002 joint EU-US demos.

grayp
Télécharger la présentation

Report on GLUE activities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GriPhyN PPDG iVDGL Report on GLUE activities Flavia Donno, INFN-Pisa EDG Retreat 27-28 August 2002 Flavia Donno, INFN-Pisa

  2. Summary • Resource Discovery and GLUE schema • Authorization services from VO LDAP to VOMS • Common software deployment procedures • IST2002 and SC2002 joint EU-US demos Interoperability : between Grid domains for core grid services, coexistence of optimization/collective services Flavia Donno, INFN-Pisa

  3. Resource Discovery and GLUE schema Computing Resources Structure Description Entry point into queueing system Container grouping subcluster or nodes Homogeneous collection of nodes Physical computing nodes Flavia Donno, INFN-Pisa

  4. Logical structure seems to have reached agreement and is stable Recently added file system, versioning Interoperability: EDG needs one CE working on a cluster made of a single Subcluster Information about Hardware are in the subcluster Information about Queue Status are in the CE Information providers have to provide Subcluster and CE info LDAP implementation defined, agreed and released! Glue Schema will be included in Globus MDS 2.2 (September 15th) Globus Info Providers fill the host level info EDG Info providers filling Subcluster and CE are needed !! When ? Is the 31st of August confirmed ? It might be necessary to include the EDG info providers in VDT This implementation can be tested with MDS 2.1 Resource Discovery and GLUE schema CE Schema implementation status Flavia Donno, INFN-Pisa

  5. Resource Discovery and GLUE schema SE Schema implementation status • Logical structure almost agreed • Missing agreement on how to model connection between storage services and file system • LDAP prototype implementation set up • Questions to EDG for release 2: • Do we need to model multiple instances of Storage Service on an SE ? Multiple VO etc... • Could it be possible to remove the SE concept and introduce the 'storage services' insisting on the same hardware or 'Storage Device‘? Flavia Donno, INFN-Pisa

  6. The US side set up the top-level GIIS Resource Discovery and GLUE schema GIIS structure proposal for the GLUE testbed mds-vo-name=glue mds-vo-name=iVDGL mds-vo-name=DataTAG Flavia Donno, INFN-Pisa

  7. GIIS giis.ivdgl.org mds-vo-name=glue Gatekeeper: Padova-site DataTAG iVDGL GIIS giis.ivdgl.org mds-vo-name=ivdgl-glue GIIS edt004.cnaf.infn.it Mds-vo-name=‘Datatag’ LSF Resource Broker Gatekeeper: US-CMS Gatekeeper grid006f.cnaf.infn.it Condor Gatekeeper edt004.cnaf.infn.it Gatekeeper: US-ATLAS WN1 edt001.cnaf.infn.it WN2 edt002.cnaf.infn.it Computing Element-1 PBS Computing Element -2 Fork/pbs Gatekeeper LSF hamachi.cs.uchicago.edu dc-user.isi.edu rod.mcs.anl.gov Job manager: Fork Flavia Donno, INFN-Pisa

  8. GLUE Schema testbed on-going activity Resource Discovery and GLUE schema Cnaf/Bologna site status • GIIS vo=Datatag edt004.cnaf.infn.it • EDG 1.1.14, MDS 2.1 Glue Schema CE v1.0, Glue Schema SE v0.1upgrade to edg1.2 • GRIS grid006f.cnaf.infn.it • EDG 1.1.14, MDS 2.1 Glue Schema CE v1.0 • Upgrade to edg 1.2 • LDAP test (no EDG software, no Globus) gummo.cnaf.infn.it • Glue Schema CE v1.0, Glue Schema SE v0.1 • Configure RB EDG 1.2 in edt003.cnaf.infn.it • Modify the RB to query the Glue Schema • Install EDG/WP4 Information Provider • Install SE EDG 1.2 and Glue Schema SE v0.1 Flavia Donno, INFN-Pisa

  9. GLUE Schema testbed on-going activity Resource Discovery and GLUE schema • US side: • Top level GIIS with Glue schema CE v1.0 • GRIS with Glue Schema CE v1.0 and MDS 2.1 and static info provider to be upgraded to MDS 2.2 and Globus Info Provider • Node with Glue Schema for SE ready for testing. Flavia Donno, INFN-Pisa

  10. Resource Discovery and GLUE schema R-GMA implementation and interoperability with MDS • The Glue UML model is general and can have different implementations: LDAP, Relational DB, XML etc. • Schema implementation in R-GMA : • implement the Glue UML model in R-GMA and test in the Glue testbed, with EDG WP3 • adapt the info providers to fill the the relational table, this should be zero or very low impact • Does MDS need to interoperate with R-GMA ? Flavia Donno, INFN-Pisa

  11. iVDGL Contacts: • B. Tierney, J. Schopf, E. Deelman, C. Kesselman, A. Shoshani • EDG Contacts: • M. Sgaravatto, P. Kunstz, J. Gordon, O. Barring Resource Discovery and GLUE schemaPeople and HW resources • PEOPLE: • DataTAG/WP4.2 people : INFN (1.5 fte) • Coordinator: Cristina Vistoli (40%) - Cristina.Vistoli@cnaf.infn.it • INFN CNAF: Sergio Andreozzi (100%) - Sergio.Andreozzi@cnaf.infn.it • Vicenzo Ciaschini (20%) - Vicenzo.Ciaschini@cnaf.infn.it • Hardware Resources: • Local HW at CNAF/Padova • Local HW at ISI Flavia Donno, INFN-Pisa

  12. Authorization services from VO LDAP to VOMS • EDG Tools proposed for use to iVDGL collaborators • VO LDAP server • VO GroupMAN • edg-mkgridmap • VOMS under study by iVDGL collaborators. • FNAL – Working on KCA/X509 • Becoming aware of LCAS/LCMAPS GLUE Flavia Donno, INFN-Pisa

  13. iVDGL Contacts: • Doug Olson • EDG Contacts: • Roberto Cecchini Authorization services People and HW resources • PEOPLE: • DataTAG/WP4.2 people : INFN (2.5 fte) • Coordinator: Roberto Cecchini (30%) - Roberto.Cecchini@fi.infn.it • INFN CNAF: Luca Dell’Agnello (30%) - Luca.DellAgnello@cnaf.infn.it • Fabio Spataro (100%) - Fabio.Spataro@cnaf.infn.it • Vicenzo Ciaschini (80%) - Vicenzo.Ciaschini@cnaf.infn.it • Hardware Resources: • Local HW at CNAF Flavia Donno, INFN-Pisa

  14. Common software deployment procedures EDG/WP4 Goals: • Contribute to a distribution that: • Is EDG and VDT compliant (can be included in VDT/EDG) • Includes main EDG and VDT components • Allows for interoperation of testbeds • Enforces common flexible configuration rules Plan: ? Distribute EDG CA ? Distribute EDG Replica Catalogue with GSI ? Distribute UI, RB, JSS, L&B as “standalone” components (End of September, edg-release 1.2 + new schema) ? Distribute information providers if requested ? Propose a configuration strategy to both VDT and EDG ? Work on a common EDG-VDT versioning strategy ? Helping experiments with software distribution (CMS/ATLAS PACMAN distribution) Done and installed on GLUE TB Done In progress In progress GLUE In progress Flavia Donno, INFN-Pisa

  15. RB RB JSS JSS EDG Structure Many Ces and SEs Many light UIs Few heavy RBs UI CE SE CE UI SE SE CE UI CE SE One LB, IS and RC LB IS RC Flavia Donno, INFN-Pisa

  16. RB RB JSS JSS SE SE SE SE CE CE CE CE VDT Structure Many VDT Clients UI Many VDT Servers One LB, IS and RC UI LB IS RC Flavia Donno, INFN-Pisa

  17. iVDGL Contacts: • Alain Roy, Saul Youseff, Jerry Gieraltowski • EDG Contacts: • Enrico Ferro Common software deployment proceduresPeople and HW resources • PEOPLE: • DataTAG/WP4.3 people : INFN (1.5 fte) • Coordinator: Flavia Donno (20%) - Flavia.Donno@pi.infn.it • INFN Milano: David Rebatto (30%) - David.Rebatto@mi.infn.it • Luca Vaccarossa (50%) - Luca.Vaccarossa@mi.infn.it • INFN Padova: Sergio Fantinel (50%) - Sergio.Fantinel@lnl.infn.it • Hardware Resources: • Local HW in Milan • Local HW in Padova • HW in US Flavia Donno, INFN-Pisa

  18. DataTag / iVDGL People: • Nikolai Smirnov, David Rebatto, Mike Wilde • EDG Contacts: • P. Kunstz Data Movement and Management • It is part of GLUE phase II. • Setting up of working groups in progress with involvement with EDG/WP2. • The existing collaboration between the Globus Team and EDG/WP2 is a good starting point. • Experiment requirements drive the effort. GLUE Flavia Donno, INFN-Pisa

  19. Joint EU-US grid demos IST2002 4-6 November, Copenaghen SC2002 16-22 November, Baltimore Goals: • Basic collaboration between European and US grid projects • Interoperability between grid domains for applications submitted by users from different virtual organizations • Controlled use of shared resources subject to agreed policy • Integrated use of heterogeneous resources from iVDGL and EDG testbed domains Infrastructure: ? WEB site: http://www.ivdgl.org/demo Hypernews: http://atlassw1.phy.bnl.gov/HyperNews/get/intergrid.html Mailing list:igdemo@datagrid.cnr.it Archive:http://web.datagrid.cnr.it/hypermail/archivio/igdemo ? GLUE testbed with common schema ? VO (DataTAG and iVDGL) LDAP Servers in EU and US ? PACMAN cache with software distribution (DataTAG or iVDGL) ? Planning document outline GLUE Flavia Donno, INFN-Pisa

  20. Joint EU-US grid demos ? GLUE testbed with Common GLUE schema and authorization/authentication tools. • EDG 1.2 + extensions, VDT 1.1.3 + extensions • “Old” authentication/authorization tools : VO LDAP servers, mkgridmap, etc. • We rely on the availability of the new GLUE schema, RB and Information Providers ? Concentrate on visualization: CMS/GENIUS, ATLAS/GRAPPA, EDG/MAPCENTER (EDG/WP7), Farm Monitoring tools (EDG/WP4), iVDGL/GANGLIA, DataTAG/NAGIOS • Use WEB portals for job submission (CMS/Genius, ATLAS/Grappa) • Provide a World Map of the sites involved (EDG-WP7/MapCenter, DataTAG/Nagios) • Monitor job status and statistics (EDG-WP4, DataTAG/Nagios). In Nagios implemented “top users”, “top applications”, “total resources”, “averages over time”, “per VO”,… • Monitor farms (EDG/WP4, DataTAG/Nagios) • Developing plugins/sensors for WP1/L&B info (only on EDG part) • Services Monitoring (EDG/WP4, DataTAG/Nagios [using MDS info]) ? CMS and ATLAS demo • ATLAS simulation jobs, GRAPPA modified to use both RB or explicit resources • Pythia & CMSIM simulation jobs submitted to intercontinental resources with IMPALA/BOSS interfaced to VDT/MOP, EDG/JDL, Genius portal. • Definition of the application demos in progress. GLUE Flavia Donno, INFN-Pisa

  21. Joint EU-US grid demos GLUE Flavia Donno, INFN-Pisa

  22. Joint EU-US grid demos Hardware definition in progress. First simple sumission tests in progress. Work on infrastructure and application demos as much as possible before the middle of September. Joint Task force in Chicago 23-30 September, 2002 for first DataTAG/iVDGL and ATLAS rehearsal. GLUE Flavia Donno, INFN-Pisa

  23. Suggestions to EDG for release 2 • Work on installation/configuration issues • Relocatability • Configuration environemnt • Service configuration and startup • Make sure “vanilla functionality” of core services is kept. • Security system with VO support • Information System • Submission services such as the Globus Gatekeeper, • JobManager, Condor-G,... • Schedule with time introduction of GLUE components in EDG. • Active collaboration. • Make service interfaces clear so to allow for easy software • sharing and integration whenever possible. • Discuss architecture design for Job scheduling, Data Management • and Storage Services. • ... GLUE Flavia Donno, INFN-Pisa

More Related