SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

SP3.1: High-Performance Distributed Computing

conway avatar
conway
14 diapositives · 304 Views · Aug 25, 2014

SP3.1: High-Performance Distributed Computing. the Ibis Java-centric grid middleware. The KOALA grid scheduler. and. Henri Bal, Thilo Kielmann, Jason Maassen, Rob van Nieuwpoort, et al. Dick Epema Catalin Dumitrescu, Alex Iosup, Hashim Mohamed, Ozan Sonmez. TUDelft: KOALA.

SP3.1: High-Performance Distributed Computing - slide 1
01 / 14SP3.1: High-Performance Distributed Computing
1 / 17

Télécharger la présentation

SP3.1: High-Performance Distributed Computing
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. SP3.1: High-Performance Distributed Computing the Ibis Java-centric grid middleware The KOALA grid scheduler and Henri Bal, Thilo Kielmann, Jason Maassen, Rob van Nieuwpoort, et al. Dick Epema Catalin Dumitrescu, Alex Iosup, Hashim Mohamed, Ozan Sonmez

  2. TUDelft: KOALA • KOALA is a multicluster/grid scheduler • Main goals of KOALA: • Load sharing of jobs across the sites in a grid: • Automatic resource selection • Co-allocation of jobs across the sites in a grid: • In order to use more resources • As dictated by the structure of applications (e.g., simulation/visualization) • KOALA has been released on the DAS in september 2005

  3. KOALA: Scheduling global queue KOALA local queues with local schedulers load sharing LS LS LS co-allocation clusters global job local jobs

  4. VU: Ibis • Ibis: Java-centric grid middleware for distributed supercomputing • Satin: divide-and-conquer parallelism in grids • GAT: Grid Application Toolkit • Implemented several Java applications from • SP 1.3 (Medical/Vumc) • SP 1.6 (Telescience/AMOLF) • SP 2.1 (iPSE/ UvA) • SP 2.2 (AID/UvA)

  5. Ibis: Grid’5000 experiments • Grid’5000: French computer scienceGrid with 2000 nodes at 9 sites • Used Grid’5000 for • Running Satin applications • Nqueens challenge (2nd Grid Plugtest) Ibis/Satin/GAT application running on 960 nodes at 6 sites, ~85% efficiency • Large-scale peer-to-peer experiments using Zorilla (Gnutella-like latency-based flooding of ads for joining a compution)

  6. KOALA feature 1: the Runners • There are many ugly application types out there • No way they can all be supported by a single scheduler • Solution: runners (=interface modules) • Currently supported: • Any type of single-component job • MPI/DUROC jobs • Ibis jobs • HOC applications runner

  7. KOALA feature 2: the policies • Originally supported co-allocation policies: • Worst-Fit: balance job components across sites • Close-to-Files: take into account the locations of input files to minimize transfer times • Different application types require different ways of component placement • So: • Modular structure with pluggable policies • Take into account internal communication structure of applications

  8. KOALA feature 3: support for HOCs • Higher-Order Components: • Pre-packaged software components with generic patterns of parallel behavior • Patterns: master-worker, pipelines, wavefront • Benefits: • Facilitates parallel programming in grids • Enables user-transparent scheduling in grids • Most important additional middleware: • Translation layer that builds a performance model from the HOC patterns and the user-supplied application parameters • Supported by KOALA (with Univ. of Münster) • Initial results: up to 50% reduction in runtimes

  9. TUDelft: GrenchMark • GrenchMark is a flexible grid workload generator, submitter, and results analyzer • Main goals of GrenchMark: • Generic workload definition for many types of workloads and application characteristics • Grid workload generation • Submitting and replaying workloads in different grid settings • GrenchMark released in november 2005 • GrenchMark used to test KOALA

  10. KOALA future (1) • Support for more applications types, e.g., • Workflows • Parameter sweep applications • Communication-aware and application-aware scheduling policies: • Take into account the communication pattern of applications when co-allocating • Also schedule bandwidth (in DAS3) • Better interface KOALA-local schedulers • KOALA is too nice

  11. KOALA future (2) • Peer-to-peer structure instead of hierarchical grid scheduler • Support heterogeneity • DAS3 • DAS2 + DAS3 • PoC • DAS3 + Grid’5000

  12. CPU’s R CPU’s R CPU’s R NOC CPU’s R CPU’s R KOALA and Ibis future DAS-3

  13. Conclusions • SP3.1 is well on track • SP3.1 has delivered reliable software tools for everybody to use: • KOALA/Grenchmark • Ibis/Satin • SP3.1 has a bright future • Still many research challenges • (Access to) great new heterogeneous testbeds

  14. More information • Web sites: • www.st.ewi.tudelft.nl/koala: • general description • KOALA tutorial • papers • grenchmark.st.ewi.tudelft.nl: • general description • download • papers • www.cs.vu.nl/ibis: • Ibis distribution • documentation • papers