EGEE information system Jozsef Patvarczki MTA SZTAKI

1. EGEE information system Jozsef PatvarczkiMTA SZTAKI EGEE is a project funded by the European Union under contract IST-2003-508833

2. Contents • The Information System of the LCG-2 • LCG-2 LDAP based IS • R-GMA • Architecture • GRIS • GIIS • BDII • LDAP protocol • GLUE Schema EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

3. The Information Sytem • Two main Information System technologies are used in LCG-2: one LDAP based from Globus and one developed by the European DataGrid Project, R-GMA • The Information System(IS) provides information about the Grid resources and their status • LCG-2 LDAP based IS GT Monitoring and Discovery Service (MDS) architecture together with Berkley Database Information Indexes (BDII) • MDS is Part of Globus Toolkit, compatible with other elements • Used most often for resource selection • aid user/agent to identify host(s) on which to run an application • Standard mechanism for publishing and discovery • Decentralized, hierarchical structure • Soft-state protocols • Caching • Grid Security Infrastructure credentials EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

4. The MDS-2 Architecture and BDII • Computing and storage resources at site report their static and dynamic status via the GRISes (Grid Resources Information Servers) to the GIIS(Grid Index Information server) • The role of the GIIS is to collect info from all the GRISes and other GIISes information sources, but it has shown his scalability limits, growing and growing the number of sites • Because of this, the BDII(Berkely DB Information Index) was introduced. • The GIIS has been kept at site level, to collect info from the site GRISes. EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

5. The Architecture of the MDS-2 EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

6. The responsible services • Lower level: GRIS • Scripts and configuration files generate ldif files containing theinformation (for example, general information of the nodes) • Other tools responsible of the dynamic information (for example, available and/or used space into a SE) – the so called information providers • Medium level: local GIIS • Same procedure taking the information from the registered GRISes • The GRIS/GIIS system can answer 1query/15min • Top level: BDII • Publish the information of the site GIISs making a refresh every 2 minutes EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

7. Berkely Database Information IndexBDII • This is the information server directly invoked by users and services • The sites listed in the BDII are really exist (it registers site GIISs) • It provides information to the RB (to find resources) • It is needed by the data management tools. The “lcg-utils” tools use it • Fundamental service to allow for stability • It can be configured by each VO following its needs • The Resource Broker uses a BDII for matchmaking purposes EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

8. The BDII • The BDII queries the GIISes and acts as a cache storinginformation about the Grid status in its database. • Each BDII contains information from the site GIISes definedby a configuration file, which it accesses through a webinterfaces. • Users and other Grid services (such as the RB) caninterrogate BDIIsto get information about the Grid status • Very up-to-date information can be found by directlyinterrogating the site GIISes or the local GRISesthat runon the specific resources. EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

9. The complete hierarchy • Local GRISes run on CEs and SEs at each site and report dynamic and static information regarding the status and availability of the services ldapsearch –x –h <hostname> -p 2135 –b “mds-vo-name=local,o=grid” • At each site, a site GIIS collects the information of all resources given by the GRISs ldapsearch –x –h <hostname> -p 2135 –b “mds-vo-name=<name>,o=grid” • Each site can run a BDII It collects the information coming from the GIISs and collects it in a data base ldapsearch –x –h <hostname> -p 2170 –b “o=grid” EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

10. The LDAP • The information system is built on the Light-weight Directory Access Protocol • It offers a hierarchical view of information • The entries are arranged in a Directory Information Tree (DIT) • Resources (computers, storage, …) each publish their part in this tree • Queries can be posed to the current Information and Monitoring Service using LDAP search commands • It establishes the transport and format of the messages used by a client to access a directory • LDAP can be used as access protocol for a large number of databases • It is the internal protocol used by the EGEE/LCG services to share information EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

11. The LDAP Protocol The LDAP information model is based on entries • These are attributecollections defined by a unique and global DN (Distinguished Name) • Information is organized in a tree-like structure. A special attribute, objectclass, can be defined for each entry. It defines the classes tree corresponding to this entry. This attribute can be used to filter entries containing that object class • The information is imported and exported from and to the LDAP server by LDIF files(LDAP Data Interchange Format) EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

12. The Glue Schema • A Schema describes the attributes and the types of the attributes associated with data objects • The offered data conforms to the GLUE Schema • Grid Laboratory for a Uniform Enviroment • The GLUE Schema activity aims to define a common conceptual data model to be used for Grid resources monitoring and discovery • There are three main components of the GLUE Schema, they describe the attributes and the value of Site information • The Computing Element • The Storage Element • Network Monitoring • Publication according to predefined GLUE schema • Key information for the RB: • GlueCEApplicationRuntimeEnvironment tags • TotalCPUs, FreeCPUs • EstimatedTraversalTime (ETT) • Network Cost • It describes the Grid resources information stored by the IS • It follows the DIT (Directory Information Tree) hierarchical structure for objectclasses and attributes EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

13. An LDAP Hierarchy EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

14. LDAP Browser • Command lineto query a BDII: • ldapsearch -h gridit-cert-rb.cnaf.infn.it -p 2170 -b "mds-vo-name=local, o=grid" -x • To query a GIIS/GRIS: • ldapsearch -h gridit-ce-001.cnaf.infn.it -p 2135 -b "mds-vo-name=local, o=grid" -x • ldapsearch -h grid007g.cnaf.infn.it -p 2135 -b "mds-vo-name=local, o=grid" -x • Windows tool • Softerra LDAP Browser 2.6(freeware), • http://www.ldapadministrator.com/download/index.php • Linux • LDAP Browser\Editor, http://www.iit.edu/~gawojar/ldap • GQ LDAP client, http://biot.com/gq/ EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

15. R-GMA: Monitoring Job Information • Relational Grid Monitoring Architecture • Dynamic job information • LDAP does not allow queries over different objects • I.e. you can only query based on attributes of an object • MDS is not designed for applications to publish their own data • It has relatively static descriptions of the data being published – the schema. • The relational model is very flexible and allows complex queries which make use of information in multiple objects • R-GMA provides a means for anyone to publish any information on the Grid – can also do the job of the current MDS • It is highly dynamic – with new Producers of information being noticed by existing Consumers EGEE Grid’05 Summer School – July 11-16, 2005 Budapest

16. THANK YOU FOR YOUR ATTENTION EGEE Grid’05 Summer School – July 11-16, 2005 Budapest