1 / 10

Resource Management

This document outlines the principles of runtime-based scheduling, focusing on the selection and management of eligible jobs using priority queues. It describes the phases involved, including the initial scheduling of jobs based on system capabilities and fairness, followed by backfill scheduling strategies such as conservative and aggressive backfilling. The decision-making process for determining job priorities, reserving resources, and the management of pivot jobs is examined. We use examples to illustrate how various jobs interact and the effect of scheduling decisions on overall performance.

mora
Télécharger la présentation

Resource Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ewa Deelman Resource Management

  2. Runtime-based Scheduling • Priority Queue Phase • Eligible jobs are determined – based on fairness, system limitations, job state, etc... • The list of eligible jobs is then sorted by job priority and jobs are scheduled to run until a job cannot be scheduled due to resource limitations • Resources and start/end times are reserved for the first job that could not be scheduled • Backfill Scheduling Phase • Jobs following the reserved job which would not delay the reserved job are scheduled on the remaining idle rsources

  3. Single Queue Backfilling • A job is allowed to jump in the queue ahead of jobs that are delayed due to lack of resources • Non-preemptive • Conservative backfilling • A job is allowed to jump ahead provided it does not delay any previous job in the queue • Aggressive backfilling • A job is allowed to jump ahead if the first job is not affected • Better performing then conservative backfilling

  4. Aggressive backfilling • Job (arrival time, number of procs, expected runtime)‏ • Any job that executes longer than expected runtime is killed • Define pivot—first job in the queue • If enough resources for pivot, execute pivot and define a new pivot • Otherwise sort all currently executing jobs in order of their expected completion time • Determine pivot time—when sufficient resources for pivot are available

  5. Aggressive backfilling • At pivot time, any idle processors not required for pivot job are defined as extra processors • The scheduler searches for the first queued job that • Requires no more than the currently idle processors and will finish by the pivot time, or • Requires no more than the minimum currently idle processors and the extra processors • Once a job becomes a pivot, it cannot be delayed • Possible that a job will end sooner and pivot starts before pivot time

  6. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 processors 8 time Queue Job B (12 procs, 1 hour)‏ Job C (20 procs, 2 hours)‏ Job D (2 procs, 50 mins)‏ Job E (6 procs, 1 hour)‏ Job F (4 procs, 2 hours)‏ Running Job A(10 procs, 40 mins)‏

  7. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 PIVOT Job B processors 8 JOB D time queue Job B (12 procs, 1 hour)‏ Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Job F (4 procs, 2 hours)‏ Running Job A(10 procs, 40 mins)‏ Job D (2 procs, 50 mins)‏

  8. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 PIVOT Job B processors 8 Job F Job D time Queue Job B (12 procs, 1 hour)‏ Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Running Job A(10 procs, 40 mins)‏ Job D (2 procs, 50 mins)‏ Job F (4 procs, 2 hours)‏

  9. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 Job B processors 8 Job F Job D time Queue Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Running Job D (2 procs, 50 mins)‏ Job F (4 procs, 2 hours)‏ Job B (12 procs, 1 hour)‏

  10. Aggressive backfilling example Job A (10 procs,40mins)‏ Pivot Job C (20 procs, 2 hours)‏ processors 10 Job B 8 Job F Job D time Queue Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Running Job D (2 procs, 50 mins)‏ Job F (4 procs, 2 hours)‏ Job B (12 procs, 1 hour)‏

More Related