1 / 22

The Globus Toolkit: Description and Applications Review

The Globus Toolkit: Description and Applications Review. Steve Tuecke & Ian Foster Argonne National Laboratory The University of Chicago Globus Co-PI: Carl Kesselman, USC/ISI. Overview. The need for Grid services The Globus toolkit Globus application case studies

michel
Télécharger la présentation

The Globus Toolkit: Description and Applications Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Globus Toolkit:Description and Applications Review Steve Tuecke & Ian Foster Argonne National Laboratory The University of Chicago Globus Co-PI: Carl Kesselman, USC/ISI

  2. Overview • The need for Grid services • The Globus toolkit • Globus application case studies • Microtomography: on-line instrumentation • SF-Express and Overflow: distributed supercomputing • CAVERNsoft: collaborative engineering • Nimrod-G: high-throughput computing • ECCE’: problem solving environment • Summary

  3. Creating a Usable Grid :Grid Services (“Middleware”) • Standard grid services that • Provide uniform, high-level access to a wide range of resources (including networks) • Address interdomain issues of security, policy, etc. • Permit application-level management and monitoring of end-to-end performance • Middleware-level and higher-level APIs and tools targeted at application programmers • Map between application and Grid

  4. Distributed computing Collab. design Remote control Application Toolkit Layer Data- intensive Remote viz Information Resource mgmt . . . Grid Services Layer Security Data access Fault detection Grid Services Architecture High-energy physics data analysis Collaborative engineering On-line instrumentation Applications Regional climate studies Parameter studies Grid Fabric Layer Transport . . . Multicast Instrumentation Control interfaces QoS mechanisms

  5. The Globus Project: Argonne, USC/ISI, NCSA, Aerospace, NASA Ames, LBNL, others • Basic research in grid-related technologies • Resource management, security, adaptation, etc. • Development of Globus toolkit • Core services for grid-enabled tools & applns • Construction of large grid testbed: GUSTO • Largest grid testbed in terms of sites & apps • Application experiments • Tele-immersion, distributed computing, etc.

  6. GUSTO Testbed Map

  7. Globus Grid Services • The Globus toolkit provides a range of basic Grid services • Security, information, fault detection, communication, resource management, ... • These services are simple and orthogonal • Can be used independently, mix and match • Programming model independent • For each there are well-defined APIs • Standards are used extensively • E.g., LDAP, GSS-API, X.509, ...

  8. Grid Services Layer (1) • Grid Security Infrastructure • Single-sign on, run anywhere [if authorized] • PKI, X.509 certificates • Identity/credential mapping at each resource • Allows programs to act as user for limited period: delegation of rights

  9. Grid Services Layer (2) • Grid Information Service • Currently an LDAP-based directory service • Publish structure and state info, dynamic performance info, software info, etc., etc. • Resource discovery: “find me an X with property Y available at time T” • Auto-configuration: “tell me what I need to know to use A efficiently/securely/...” • Gateways to other data sources required • Example of integrating “middleware” service

  10. Grid Services Layer (3) • Access to remote data (GASS) • Uniform access to diverse storage management systems • Cache management • Integration with SRB, DPSS, HPSS • Communication (Nexus) • Application-level interfaces to comm services • Multiple methods: reliable/unreliable, IP/other, unicast/multicast • QoS interfaces

  11. Grid Services Layer (4) • Globus Resource Allocation Manager (GRAM) • Uniform interface to resource management • Globus Arch. for Reservation and Allocation • Co-allocation of compute resources • Immediate and advance reservation of network and computers in prototype form • Fault detection service • Network measurement tools • Code management and distribution infrastructure

  12. Application Toolkit Layer: e.g. • Message Passing Interface • Multi-method communication, specialized • CAVERNsoft • Shared state for collaborative environments • Condor, Nimrod-G • High-throughput computing • Parallel Application Workspace (PAWS) • High-speed parallel transfers for coupled apps

  13. Globus Progress • Selected “Grid Services” are being migrated into the infrastructure • Grid information service • Grid security infrastructure • Grid resource management services • Simultaneously these and other Globus services are being applied to develop • Grid-enabled tools • Grid-enabled applications • An ongoing iterative refinement process

  14. Case Study 1:Online Instrumentation Advanced Photon Source wide-area dissemination desktop & VR clients with shared controls real-time collection archival storage tomographic reconstruction DOE X-ray source grand challenge: ANL, USC/ISI, NIST, U.Chicago

  15. CMT Processing Now

  16. Additional Opportunities • End-to-end advance reservation of network, storage, computers • Dynamic discovery and allocation of supercomputers, networks, etc. • Adaptive determination of display resolution, reconstruction fidelity, etc., etc. • Reliable multicast for data, control, video • Access control and discovery for collaborative sessions • Integration with mass storage systems

  17. Case Study 2:Distributed Supercomputing • Starting point: SF-Express parallel simulation code • Globus mechanisms for • Resource allocation • Distributed startup • I/O and configuration • Fault detection • 100K vehicles (2002 goal) using 13 computers, 1386 nodes, 9 sites NCSA Origin Caltech Exemplar CEWES SP Maui SP SF-Express Distributed Interactive Simulation: Caltech, USC/ISI

  18. OVERFLOW with latency-tolerant algorithms MPICH-G “Grid-enabled” message passing Globus services Security Directory Scheduling Process mgmt Communication ARC SGI O2000 (California) Argonne SGI O2000 (Illinois) OVERFLOW simulation: NASA Ames

  19. Case Study 3:Collaborative Engineering • Manipulate shared virtual space, with • Simulation components • Multiple flows: Control, Text, Video, Audio, Database, Simulation, Tracking, Haptics, Rendering • Uses Globus comms: (un)reliable uni/multicast • Future: Security, QoS, allocation, reservation CAVERNsoft: UIC Electronic Visualization Laboratory

  20. Case Study 4:High-Throughput Computing • Schedule many independent tasks (e.g., parameter study) • Uses Globus security, discovery, data access, scheduling • Future: Reservation, accounting, code management, etc. Deadline Cost Available Machines Nimrod-G: Monash University

  21. Case Study 5:Problem Solving Environment • Problem solving environment for comp. chemistry • Globus services used for authentication, remote job submission, monitoring, and control • Future: distributed data archive, resource discovery, charging ECCE’: Pacific Northwest National Laboratory

  22. Summary • Grids require Grid services that make resources accessible and usable and Grid toolkits for application development • The Globus project is building essential services and partnering with tool developers • Significant success stories in a range of problem classes • We’re looking forward to working with applications throughout the community!

More Related