1 / 17

R-GMA: First results after deployment

R-GMA: First results after deployment . Steve Fisher (EDG - WP3) s.m.fisher@rl.ac.uk https://edms.cern.ch/document/376535/. Who we are. Heriot-Watt, Edinburgh Andrew Cooke, Werner Nutt IBM-UK James Magowan, (Manfred Oevers), Paul Taylor INFN

azra
Télécharger la présentation

R-GMA: First results after deployment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. R-GMA: First results after deployment Steve Fisher (EDG - WP3) s.m.fisher@rl.ac.uk https://edms.cern.ch/document/376535/

  2. Who we are • Heriot-Watt, Edinburgh • Andrew Cooke, Werner Nutt • IBM-UK • James Magowan, (Manfred Oevers), Paul Taylor • INFN • Roberto Barbera, Giuseppe Save, Gennaro Tortone • Queen Mary, University of London • Roney Cordenonsi, (Ari Datta) • CCLRC • Linda Cornwall, Abdeslem Djaoui, Steve Fisher, Robin Middleton • PPARC • Rob Byrom, Laurence Field, Steve Hicks, Manish Soni, Antony Wilson, (Xiaomei Zhu), Jason Leake • SZTAKI, Hungary • Peter Kacsuk, Norbert Podhorszki • Trinity College Dublin • Brian Coghlan, Stuart Kenny, David O’Callaghan, (John Ryan)

  3. R-GMA Producer • Uses the Grid Monitoring Architecture from Global Grid Forum • R-GMA is a relational implementation • Applied to both information and monitoring • Creates impression that you have one RDBMS per Virtual Organisation Registry Information flow Meta-data flow Consumer

  4. Relational Approach • Not a general distributed RDBMS system, but a way to use the relational model in a distributed environment where global consistency is not important. • Producers announce: SQL “CREATE TABLE” publish: SQL “INSERT” • Consumers collect: SQL “SELECT” • Some producers, the Registry and Schema make use of RDBMS as appropriate – but what is central is the relational model.

  5. Producers • DataBaseProducer – Supports History Queries • Information not lost • Supports joins • Clean up strategy • StreamProducer – Supports Continuous Queries • In memory data structure • Can define minimum retention period • ResilientStreamProducer – Supports Continuous Queries • Like the StreamProducer but won’t lose data if system crashes • So slightly slower • LatestProducer – Supports Latest Queries • Just holds the latest information for any “primaryish” key • Supports joins • CanonicalProducer – Supports anything • Offers anything as relations

  6. Archiver (Re-publisher) • It is a combined Consumer-Producer • You just have to tell it what to collect and it does so on your behalf • Re-publishes to any kind of “Insertable” (i.e. not to the CanonicalProducer)

  7. Schema & Contributions

  8. The Mediator • Producers, associated with views on a virtual data base. • Queries posed against the virtual data base • The Mediator must: • find the right Producers • combine information from them • Can now merge information from several producers • The final mediator will take “any” SQL statement and do the right thing

  9. R-GMA Tools • R-GMA CLI • Command Line Interface (similar to MySQL) • Supports single query and interactive modes • R-GMA Browser • JSP application dynamically generating web pages • Supports pre-defined and user-defined queries • Pulse • R-GMA Java client-based GUI • Supports streaming and simple graphical displays

  10. A user application: CMS • BOSS for job tracking on local farm • It currently forks the executable and parses stdout to publish info directly to an SQL DB • They publish to one table per job type and one table which is common to all job types • They are now ready to publish via R-GMA instead • Providing a scaleable Grid solution

  11. GIN and GOUT (Gadget IN and Gadget OUT) Archiver LDAP InfoProvider Consumer (CE) ConsumerAPI DataBase Producer Consumer (SE) GIN RDBMS Consumer (SiteInfo) R-GMA CircularBuffer Producer GOUT CircularBuffer Producer LDAP Server R-GMA Consumers GIN LDAP InfoProvider

  12. CE and SE Tables “Select a ComputingElement with at least 1 free CPU that also has a CloseStorageElement with at least 1000 MB of free space” SELECT DISTINCT ComputingElement.CEId FROM ComputingElement, CloseStorageElement,StorageElementStatus WHERE ComputingElement.FreeCPUs > 0 AND (ComputingElement.CEId = CloseStorageElement.CEId AND CloseStorageElement.CloseSE = StorageElementStatus.SEId AND StorageElementStatus.SEfreespace > 1000)

  13. Registry OGSIfied R-GMA Consumer Factory Application • All Grid Services • OGSA Factories, GSH, GSR • Registry includes HandleMapper • SQL as Service Data Element Query Language Consumer API Consumer Instance Producer API Producer Instance Schema Sensor Producer Factory

  14. Other technicalities – no time today • Soft-state Registration and the Registry • Registry records existence of Producers and Consumers • Registry holds last contact time and ‘expiry’ time • Producers and Consumers periodically refresh their time stamps • Scheduled removal of entries that have timed-out • Registry & schema distribution • Will have one logical registry and schema per VO • Each logical registry will have multiple physical “copies” • Self healing algorithm • Security • etc …

  15. Performance • By design: • Very flexible - to avoid bottlenecks • Powerful queries allow a single query to be made • Performance and Optimisation • Use NetLogger and profiling tools to identify possible bottlenecks

  16. Results • It has only just been deployed in the EDG development testbed and we do not yet have the results which the title of this talk implied.

  17. Summary and the future • R-GMA is a combined Grid information and monitoring system • Just deployed in the EDG development testbed • Focusing on reliability, stability and performance for the rest of the project (9 months) Thanks to the EU and our national funding agencies for their support of this work

More Related