Fairness in Queues: Minimizing User Response Times

1. FAIRNESS IN QUEUESAdam WiermanCarnegie Mellon Universityacw@cs.cmu.eduJoint work with Mor Harchol-Balter

2. users CPUs Internet Web Servers Databases Routers Disks …also p2p, wireless, operating systems… Locks “SCHEDULING SUCCESS STORIES” ARE EVERYWHERE Biersack, Rai, Urvoy-Keller, Harchol-Balter, Schroeder, Agrawal, Ganger, Petrou, Misra, Feng, Hu, Zhang, Mangharam, Sadowsky, Rawat, Dinda, McWherter, Ailamaki, & others

3. bottleneck resource Processor Sharing (PS) Use a different scheduling policy THE ESSENCE OF A “SCHEDULING SUCCESS STORY” web server, router, etc. Goal Minimize user response times

4. ? What about fairness to large jobs? Sched- uling Sched- uling FCFS PS mean response time ? SRPT 0 0.25 0.5 0.75 load SRPT WINS BIG WHAT POLICY SHOULD WE USE? SRPT

5. ? What does “fair” mean? ARE POLICIES THAT PRIORITIZE SMALL JOBS FAIR TO LARGE JOBS? M/GI/1 Preempt-Resume Queue

6. WHAT IS FAIRNESS? Definition: A fair decision is one that makes the proper balance of conflicting interests • The “proper balance” is • determined by • consulting the majority, • employing logic, • referring to the divine.

7. ? Which is more fair when two jobs of the same size arrive one afer the other... B A 1 B A 2

8. ? Which is more fair when a small job and a large job arrive at the same time... B 1 A B 2 A

9. BUT, THE SETTING MATTERS OUR SETTING:Are the response times of large jobs “unfairly” long?

10. WHAT IS FAIRNESS? Definition: A fair decision is one that makes the proper balance of conflicting interests • The “proper balance” is • determined by • consulting the majority, • employing logic, • referring to the divine.

11. Response times should be proportional to job sizes. T(x)/x should be constant EMPLOYING LOGIC Aristotle’s notion of fairness Like cases should be treated alike, different cases should be treated differently, and different cases should be treated differently in proportion to their differences. slowdown

12. Share the server evenly among all jobs Processor Sharing (PS) is fair EMPLOYING LOGIC Rawls’ Theory of Social Justice All social goods should be distributed equally, unless unequal distribution is to the advantage of the least favored

13. If you beat PS for all sizes, you’re still fair EMPLOYING LOGIC Min-Max fairness (Pareto Efficiency) All jobs deserve an equal share of the resources ... but if some jobs can use more without hurting others, that’s okay

14. ? ? Is 1/(1-ρ) really a min-max criteria Is dividing by “x” the right scaling? MIN-MAX FAIRNESS Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if:

15. ? ? Is 1/(1-ρ) really a min-max criteria Is dividing by “x” the right scaling? Theorem: In an M/GI/1 with an unbounded, continuous service distribution having finite E[X2], under any non-idling policy we have and further [Wierman and Harchol-Balter 2003]

16. ? How do common policies perform? MIN-MAX FAIRNESS Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if:

17. PS – FCFS – SRPT Processor Sharing E[T(x)] / x PS x

18. PS – FCFS – SRPT First Come First Served The unfairness can be unbounded E[T(x)] / x PS FCFS x Under a Pareto with ρ=0.8, this is >80% of the jobs

19. E[T(x)] / x E[T(x)] / x x x PS – FCFS – SRPT Shortest Remaining Processing Time SRPT PS PS SRPT Theorem SRPT is fair for ρ<0.5, regardless of the job size distribution. Theorem For every service distribution with finite E[X2], SRPT is unfair for high enough load.

20. OTHER POLICIES Always Unfair SRPT PS Always Fair Sometimes Fair FCFS

21. PLCFS PS OTHER POLICIES Always Unfair SRPT LRPT Prioritize Small Jobs Prioritize Large Jobs Always Fair PLJF PSJF LCFS Sometimes Fair ROS FCFS SJF LJF FB [Wierman and Harchol-Balter 2003, 2005]

22. PLCFS PS ! There’s a tradeoff between fairness and efficiency OTHER POLICIES Remaining size based Always Unfair SRPT LRPT Preemptive size based Non-preemptive non-size based Always Fair PLJF PSJF LCFS Sometimes Fair ROS FCFS SJF LJF Age based Non-preemptive size based FB [Wierman and Harchol-Balter 2003, 2005]

23. PLCFS PS ? Is there a fair policy with near optimal performance? FAIRNESS VS. EFFICIENCY Always Unfair SRPT LRPT Always Fair PLJF PSJF LCFS Sometimes Fair ROS FCFS SJF LJF more circles better mean response time FB

24. Fair Sojourn Protocol (FSP) [Henderson and Friedman 2004] Guarantees that every job finishes earlier than it would under PS.

25. Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” • Compute the time when each job would finish in a virtual PS system. • Order the jobs according to their virtual completion times. • Devote the full processor to the job with the earliest PS completion time

26. ! FSP did the same thing as SRPT Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times”

27. ? How good are response times under FSP? Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times”

28. ? How good are response times under FSP? ...your answer here...

29. ! Worries about fairness are not limited to the mean MIN-MAX FAIRNESS Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if:

30. Raw moments E[T(x)i] Central moments Var[T(x)], etc Cumulant moments E[T(x)]  ? X BEYOND EXPECTATION: Higher Moments X [Wierman and Harchol-Balter 2005]

31. ? Do these look familiar? CUMULANTS Cumulants are a descriptive statistic, similar to the moments. They can be found as a function of the moments: or from the log of the moment generating function:

32. WHYCUMULANTS? Cumulants have many nice properties: additivity: homogeneity: 1st cumulant is shift-equivariant & the rest are shift-invariant

33. ? ! Why is this the right generalization? Lots of open questions here MIN-MAX FAIRNESS Wierman and Harchol-Balter 2005 Definition: Consider an M/GI/1 queue. A policy P is min-max fairif, for all i:

34. MIN-MAX FAIRNESS IS NOT ALWAYS APPROPRIATE FCFS is the most fair policy in some settings

35. TEMPORAL FAIRNESS It is unfair to violate the seniority of a job Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. [Wierman 2004]

36. FCFS 1 1-ρ 0 E[Pol(x)] x Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. FCFS –PLCFS – PS – SRPT First Come First Served

37. FCFS 1 1-ρ 0 E[Pol(x)] PLCFS x Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. FCFS– PLCFS – PS – SRPT Preemptive Last Come First Served

38. FCFS–PLCFS – PS – SRPT Processor Sharing FCFS FCFS 1 1-ρ 0 E[Pol(x)] PS PS PLCFS PLCFS x job size %-tile 0 1

39. ! Prioritizing small jobs helps overall politeness FCFS–PLCFS – PS – SRPT Shortest Remaining Processing Time FCFS FCFS 1 1-ρ 0 SRPT E[Pol(x)] SRPT PLCFS PLCFS x job size %-tile 0 1

40. MANY INTERESTING QUESTIONS... • Can you think of a policy (other than FCFS) that avoids being maximally impolite to some job size? • How polite are scheduling heuristics & techniques? • What is the tradeoff between temporal fairness and proportional fairness? and efficiency?

41. PROPORTIONAL FAIRNESS VSTEMPORAL FAIRNESSVS EFFICIENCY min-max fairness politeness mean response time

42. Min-max Fairness less fair more fair Politeness less polite more polite Theorem In an M/GI/1 any Always Fair policy has FCFS FCFS SRPT FSP PS PLCFS FSP SRPT PS LRPT LRPT PLCFS

43. Min-max Fairness less fair more fair Politeness less polite more polite FCFS SRPT FSP PS more circles better mean response time LRPT PLCFS

44. WRAPUP

45. a universal definition is unlikely to be useful (or possible) ! Prioritizing small jobs provides a balance between fairness & efficiency FAIRNESS IS AN AMORPHOUS CONCEPT PROPORTIONAL FAIRNESS min-max TEMPORAL FAIRNESS politeness ...THIS IS A VERY ACTIVE AREA OF RESEARCH,MANY OPEN QUESTIONS REMAIN

46. AN EXPLOSION OF WORK ON FAIRNESS since the introduction of min-max fairness in 2003 defining other fairness metrics analyzing policies [Wierman 2004] [Raz, Avi-Itzhak 2004] [Levy, Raz, Avi-Itzhak 04] [Sandmann 2005] Many papers by: Wierman, Harchol-Balter, Henderson, Friedman, Biersack, Rai, Ayesta, Aalto, Nunez-Queija, Misra, Feng, Vernon, Williamson, Brown, Bansal, and others extending definition to higher moments [Wierman & Harchol-Balter 2005a,b][Williamson, Gong 2003, 2004] [Brown 2006] and others.

47. DiscFreq = ni + c∙mi ni = number of jobs that arrived later and completed earlier than job i mi = number of larger jobs (at the arrival of job i) that complete earlier than job i fair service percentage percent of service given to job i [Sandmann 2005] MANY OTHER INTERESTING METRICS [Levy, Raz, Avi-Itzhak 04]

48. Adam WiermanCarnegie Mellon Universityacw@cs.cmu.eduPapers are available at: http://www.cs.cmu.edu/~acw