Scheduling

1. Scheduling Landon Cox February 26, 2018

2. Switching threads • Thread returns control to scheduler • For example, via timer interrupt • Scheduler chooses next thread to run • Scheduler saves state of current thread • To its thread control block • Scheduler loads context of next thread • From its thread control block • Jump to PC in next thread’s TCB

3. Scheduling goals • Minimize average response time • Elapsed time to do a job (what users care about) • Try to maximize idle time • Incoming jobs can then finish as fast as possible • Maximize throughput • Jobs per second (what admins care about) • Try to keep parts as busy as possible • Minimize wasted overhead (e.g., context switches)

4. Scheduling goals 3. Fairness • Share CPU among threads equitably • Key question: what does “fair” mean? Job 1 Job 2 Needs 100 seconds of CPU time Needs 100 seconds of CPU time

5. What does “fair” mean? • How can we schedule these jobs? • Job 1 then Job 2 • 1’s response time = 100, 2’s response time = 200 • Average response time = 150 • Alternate between 1 and 2 • 1’s response time = 2’s response time = 200 • Average response time = 200 Job 1 Job 2 Needs 100 seconds of CPU time Needs 100 seconds of CPU time

6. Fairness • Fairness may come at a cost • (in terms of average response time) • Finding a balance can be hard • What if you have 1 big job, 1 small job? • Response time proportional to size? • Trade-offs come into play

7. FCFS (first-come-first-served) • FIFO ordering between jobs • No pre-emption (run until done) • Run thread until it blocks or yields • No timer interrupts

8. FCFS example • Job A (100 seconds), Job B (1 second) • Average response time = 100.5 seconds A arrives and starts (t=0) A done (t=100) B arrives (t=0+) B done (t=101)

9. FCFS • Pros • Simplicity • Cons? • Short jobs stuck behind long ones • Non-interactive • (for long CPU job, user can’t type input)

10. Round-robin • Goal • Improve average response time for short jobs • Solution to FCFS’s problem • Add pre-emption! • Pre-empt CPU from long-running jobs (timer interrupts) • This is what most OSes do

11. Round-robin • In what way is round robin fairer than FCFS? • Makes response times job length • In what way is FCFS fairer than round robin? • First to start will have better response time • Like children playing with a toy • Should they take turns? • “She’s been playing with it for a long time.” • Should first get toy until she’s bored? • “I had it first.”

12. Round-robin example • Job A (100 seconds), Job B (1 second) • Average response time = 51.5 seconds A arrives and starts (t=0) A swapped out (t=1) A runs (t=2) A done (t=101) B arrives (t=0+) B starts (t=1) B done (t=2) Does round robin always provide lower response times than FCFS?

13. Round-robin example 2 • Job A (100 seconds), Job B (100 second) • Average response time = 199.5 seconds A arrives and starts (t=0) A swapped out (t=1) A runs (t=2) A done (t=199) B arrives (t=0+) B starts (t=1) B swapped out (t=2) B done (t=200) Any hidden costs that we aren’t counting? Context switches

14. Round-robin example 2 • Job A (100 seconds), Job B (100 second) • Average response time = 199.5 seconds A done (t=199) B done (t=200) What would FCFS’savg response time be? 150 seconds

15. Round-robin example 2 • Job A (100 seconds), Job B (100 second) • Average response time = 199.5 seconds A done (t=199) B done (t=200) Which one is fairer? It depends …

16. Round-robin • Pros • Good for interactive computing • Better than FCFS for mix of job lengths • Cons • Less responsive for uniform job lengths • Hidden cost: context switch overhead • How should we choose the time-slice? • Typically a compromise, e.g., tens of ms • Most threads give up CPU voluntarily much faster

17. STCF and STCF-P • Idea • Get the shortest jobs out of the way first • Improves short job times significantly • Little impact on big ones • Shortest-Time-to-Completion-First • Run whatever has the least work left before it finishes • Shortest-Time-to-Completion-First (Pre-emp) • If new job arrives with less work than current job has left • Pre-empt and run the new job

18. STCF is optimal • (among non-pre-emptive policies) • Intuition: anyone remember bubble sort? Job B Job A Start Finish What happened to total time to complete A and B?

19. STCF is optimal • (among non-pre-emptive policies) • Intuition: anyone remember bubble sort? Job A Job B Start Finish What happened to the time to complete A? What happened to the time to complete B?

20. STCF is optimal • (among non-pre-emptive policies) • Intuition: anyone remember bubble sort? Job A Job B Start Finish What happened to the average completion time?

21. STCF-P is also optimal • (among pre-emptive policies) • Job A (100 seconds), Job B (1 second) • Average response time = 51 seconds A arrives and starts (t=0) A runs (t=1) A done (t=101) B done (t=1) B arrives and pre-empts (t=0+)

22. I/O • What if a program does I/O too? • To scheduler, is this a long job or a short job? • Short • Thread scheduler only cares about CPU time while (1) { do 1ms of CPU do 10ms of I/O }

23. STCF-P • Pros • Optimal average response time • Cons? • Can be unfair • What happens to long jobs if short jobs keep coming? • Legend from the olden days … • IBM 7094 was turned off in 1973 • Found a “long-running” job from 1967 that hadn’t been scheduled • Requires knowledge of the future

24. Knowledge of the future • You will see this a lot in CS. • Examples? • Cache replacement (next reference) • Banker’s algorithm (max resources) • How do you know how much time jobs take? • Ask the user (what if they lie?) • Use the past as a predictor of the future • Would this work for banker’s algorithm? • No. Must know max resources for certain.

25. Grocery store scheduling • How do grocery stores schedule? • Kind of like FCFS • Express lanes • Make it kind of like STCF • Allow short jobs to get through quickly • STCF-P would probably be considered unfair

26. Final example • Job A (CPU-bound) • No blocking for I/O, runs for 1000 seconds • Job B (CPU-bound) • No blocking for I/O, runs for 1000 seconds • Job C (I/O-bound) while (1) { do 1ms of CPU do 10ms of I/O }

27. Each job on its own B: 100% CPU, 0% Disk A: 100% CPU, 0% Disk C: 1/11 CPU, 10/11 Disk CPU Disk Time

28. Mixing jobs (FCFS) B: 100% CPU, 0% Disk A: 100% CPU, 0% Disk C: 1/11 CPU, 10/11 Disk CPU Disk How well would FCFS work? Not well.

29. Mixing jobs (RR with 100ms) B: 100% CPU, 0% Disk A: 100% CPU, 0% Disk C: 1/11 CPU, 10/11 Disk CPU Disk How well would RR with 100 ms slice work?

30. Mixing jobs (RR with 1ms) B: 100% CPU, 0% Disk A: 100% CPU, 0% Disk C: 1/11 CPU, 10/11 Disk CPU Disk How well would RR with 1 ms slice work? Good Disk utilization (~90%) Good principle: start things that can be parallelized early A lot of context switches though …

31. Mixing jobs (STCF-P) B: 100% CPU, 0% Disk A: 100% CPU, 0% Disk C: 1/11 CPU, 10/11 Disk B? CPU Disk How well would STCF-P work? (run C as soon as its I/O is done) Good Disk utilization (~90%) Many fewer context switches Why not run B here? When will B run? When A finishes

32. Real-time scheduling • So far, we’ve focused on average-case • Alternative scheduling goal • Finish everything before its deadline • Calls for worst-case analysis • How do we meet all of our deadlines? • Earliest-deadline-first (optimal) • Used by students to complete all homework assignments • Used by professors to juggle teaching and research • Note: sometimes tasks get dropped (for both of us)

33. Earliest-deadline-first (EDF) • EDF • Run the job with the earliest deadline • If a new job comes in with earlier deadline • Pre-empt the current job • Start the next one • This is optimal • (assuming it is possible to meet all deadlines)

34. EDF example • Job A • Takes 15 seconds • Due 20 seconds after entry • Job B • Takes 10 seconds • Due 30 seconds after entry • Job C • Takes 5 seconds • Due 10 seconds after entry

35. EDF example B: takes 10, due in 30 A: takes 15, due in 20 C: takes 5, due in 10 A B C 0 5 20 30 35 40 45 50 55

36. Proportional-share scheduling • We might not know much about our tasks • Don’t know how long they’ll take to finish • Don’t know if they’ll block on I/O • Don’t know by when they need to finish • We probably know which jobs are important • Could express the relative importance with priorities • Assign each process a value • High priority (-20) to low priority (19) • High priority tasks should run more often than low

37. Priority scheduling Info bus manager (high priority) Communicate with ground (medium priority) Sense weather (low priority) (2) block on lock Mars Pathfinder bug collect data, acquire lock, copy to bus bus () { while (1) { lock.acquire () dispatch local messages lock.release () } } weather () { while (1) { collect sensor data lock.acquire() copy data to bus lock.release() } } (3) pre-empt weather, further stalling bus comm () { while (1) { send/rcv remote messages } } watchdog () { while (1) { sleep (1s) if (bus hasn’t run) { panic and reboot! } } } (4) bus is stalled, freak out!

38. Priority inversion • Exemplified by Pathfinder bug • High priority tasks stuck behind lower priorities • Issue was dependency between bus and weather • Bus was blocked waiting for weather • Weather should inherit bus’s priority • Lottery scheduling • Allows programmers to express tasks’ ideal share of resources • Goal is proportional sharing of resource • Optimize for fair share rather than response time • Allows easy inheritance of shares

39. Lottery scheduling • Tasks are assigned tickets • Percent of total tickets represents share of resource • Example: • A has 75 tickets (0 through 74) • B has 25 tickets (75 through 99) • Goal: A gets 75% of CPU, B gets 25% of CPU • To schedule A and B, hold a lottery for decision

40. Lottery scheduling • Choose random number between 0, 99 • Whoever holds the winning ticket runs 63 85 70 39 76 17 29 41 36 39 10 99 68 83 63 62 43 0 49 49 A B A A B A A A A A A B A B A A A A A A This isn’t exactly what we wanted, but over longer periods, A should get CPU 75% of the time. Lottery scheduling is most effective over many scheduling decisions. A runs 16 times during the 20 time slices B runs four times during the 20 time slice

41. Lottery scheduling • Lotteries can be fast, require little state Scheduler picks random number between 0 and 19 … 15! Task A (10 tix) Task B (2 tix) Task C (5 tix) Task D (1 tix) Task E (2 tix) Sum = 10 Sum < 15 Sum = 12 Sum < 15 Sum = 17 Sum >= 15

42. Lottery scheduling • Tickets can be dynamically assigned • Currencies • Allows tasks to express importance of sub-tasks • Lotteries always convert tickets to base currency • Compensation • Handles case when task doesn’t use entire slice • Idea is to inflate tickets proportional to unused resources • Transfers • Allows tasks to hand off tickets to other tasks • Very useful when dependencies arise

43. Currencies Whole system (200 tix) User A (100 tix) User B (100 tix) Curr A (1000 tix) Curr B (10 tix) Task 2 (500 tix) Task 1 (10 tix) Task 1 (500 tix) 50 in base 50 in base 100 in base

44. Compensation • Two tasks: A and B • Both have 400 tickets in base currency • Want A to run 50%, B to run 50% time • Time slice is 100ms • What happens if A completes after 100ms, B after 20ms? • B will only consume 20% of its allotment • A will consume the full 100% of its allotment • Shares over time will not be 50-50 • How to solve this? • Inflate B’s funds by 1/f where f is fraction used • When B stops after 20ms, give it extra 1600 tix for next lottery • B will have 2000 tix total for next scheduling decision

45. Transfers What should happen to bus tix when it blocks? Info bus manager (high priority) Communicate with ground (medium priority) Sense weather (low priority) Mars Pathfinder bug bus () { while (1) { lock.acquire () dispatch local messages lock.release () } } weather () { while (1) { collect sensor data lock.acquire() copy data to bus lock.release() } } comm () { while (1) { send/rcv remote messages } } watchdog () { while (1) { sleep (1s) if (bus hasn’t run) { panic and reboot! } } }