SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Integrating Databases into Grid Computing: G-DSE and Query Elements

river avatar
river
11 diapositives · 227 Views · Aug 16, 2014

This presentation discusses the challenges of integrating databases as resources within Grid computing environments. Launched in 2004 as part of the GRID.IT project, the G-DSE (Grid Data Source Engine) provides a solution for seamless database access, enabling the execution of complex workflows where data retrieval and analysis occur on the Grid. Key features include support for various database backends, compliance with evolving Grid middleware, and robust user access control. This advancement ensures that databases can be effectively utilized alongside other Grid resources.

Integrating Databases into Grid Computing: G-DSE and Query Elements - slide 1
01 / 11Integrating Databases into Grid Computing: G-DSE and Query Elements
1 / 17

Télécharger la présentation

Integrating Databases into Grid Computing: G-DSE and Query Elements
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Databases in the Grid Grid-Data Source Engine and QE A New Data Source Oriented CE for GRID Claudio Vuerli – INAF-OATs Giuliano Taffoni – INAF-OATs EGEE ’07 – Budapest – 3 October 2007

  2. The Database crisis • The Grid limit: it is able to execute binary code or shell scripts and to store files. • We didn’t find any suitable solution that allowed us to have databases as an integrated resource in Grid. • We started to tackle this issue in 2004 (GRID.IT project). EGEE ’07 – Budapest – 3 October 2007

  3. DB in Grid: requirements / I Databases includes metadata and/or data • I have a DB and I want to USE it on GRID. • I have a number of DBs and I need using all of them for my application. • I want to submit to the Grid complex jobs (workflows) where data retrieval/storage and data analysis are intermixed EGEE ’07 – Budapest – 3 October 2007

  4. DB in Grid: requirements / II • Sometimes I need to move the execution to the data and not data to the code. • The integration/interoperability between the Grid technology and for example the Virtual Observatory is of crucial importance for my everyday work • I need to stay fully compliant with the evolution of the Grid M/W EGEE ’07 – Budapest – 3 October 2007

  5. G-DSE / I • With G-DSE+QE, Databases become embedded resources in Grid as any other resource (CPU, Data Storage). • Database machines (the new Grid nodes where the new DB resources reside) can be used to both access/retrieve data and make computations on them. EGEE ’07 – Budapest – 3 October 2007

  6. G-DSE / II • The Grid Resource Framework Layer, Information System and Data Model is extended so that a software virtual machine as a Data Source Engine becomes a valid instance for a Grid computing model. • A new Grid component(G- DSE) that enables the access to a Data Source Engine and Data Source, totally integrated with the Grid Monitoring and Discovery System and Resource Broker is defined • A new Grid Element, the Query Element, can be built on top of the G-DSE component. EGEE ’07 – Budapest – 3 October 2007

  7. G-DSE / III GIS GRAM gatekeeper MDS GRIS ldif JobManger QueryManger Ldap query plug-in Scheduler p-in Grid Providers (snmp) Query DB specific driver Pbs/LFS JobProcess QueryProcess RDBMS RDBMS EGEE ’07 – Budapest – 3 October 2007

  8. G-DSE / IV CE - Runs on any linux/unix flavor: GT>=2.4.3 - Backends: any DB vendor + flat files code query QE QE - Two protocols: GRAM or WS - API: C, C++, python, Java, perl - If it works with Globus it works with G-DSE EGEE ’07 – Budapest – 3 October 2007

  9. G-DSE / V • Access control using GSI and VOMS • The certificate + roles identify the user permissions on DB • Super user: create, modify, admin, grant and revoke users…. ANYTHING!!! • Standard user: select+insert • Simple user: select VOMS roles and groups mapping with db user: Attribute:/vo/dbuser/ROLE=astrouser/CAPABILITY=select EGEE ’07 – Budapest – 3 October 2007

  10. Summing up • G-DSE supports Data Source (DS) and DSE indexing, monitoring, management and recovery through a rich set of Meta-Data bound to standard GIS. • DS have their core engine into G-DSE, that provides a framework for activity and task management. • A RSL/JDL Transaction/Query permits a number of tasks to be specified, together with their parameters, inputs, outputs and control flow. • The response to a request is generated by the G-DSE within a JobQueryManager Session. The G-DSE analyses incoming Task and conducts authentication and authorisation • The standard Grid WorkLoad Manager constructs an optimised execution graph. • GIS will monitor a DS’s and DSE’s status digest produced by its internal monitor. • The G-DSE has been designed to support dynamic configuration, sessions, transactions, recovery and concurrency. EGEE ’07 – Budapest – 3 October 2007

  11. End of Presentation Thank you for your attention EGEE ’07 – Budapest – 3 October 2007