1 / 35

Storage: Isolation and Fairness

Storage: Isolation and Fairness. Lecture 24 Aditya Akella. Performance Isolation and Fairness in Multi-Tenant Cloud Storage , D. Shue , M. Freedman and A. Shaikh , OSDI 2012. Setting: Shared Storage in the Cloud. Y. Y. F. F. Z. Z. T. T. Shared Key-Value Storage. S3. EBS. SQS.

lis
Télécharger la présentation

Storage: Isolation and Fairness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Storage: Isolation and Fairness Lecture 24 Aditya Akella

  2. Performance Isolation and Fairness in Multi-Tenant Cloud Storage, D. Shue, M. Freedman and A. Shaikh, OSDI 2012.

  3. Setting: Shared Storage in the Cloud Y Y F F Z Z T T Shared Key-Value Storage S3 EBS SQS

  4. Predictable Performance is Hard Z Z Z Y Y Y Y T T F F F F F Shared Key-Value Storage DD DD DD DD Multiple co-located tenants ⇒ resource contention

  5. Predictable Performance is Hard Z Z Z Y Y Y Y T T F F F F F Fair queuing @ big iron DD Shared Key-Value Storage DD DD DD DD DD DD DD Multiple co-located tenants ⇒ resource contention

  6. Predictable Performance is Hard Z Z Z Y Y Y Y T T F F F F F SS SS SS SS Multiple co-located tenants ⇒ resource contention Distributed system ⇒ distributed resource allocation

  7. Y F T Z keyspace keyspace keyspace keyspace Predictable Performance is Hard data partition popularity Z Z Z Y Y Y Y T T F F F F F SS SS SS SS Multiple co-located tenants ⇒ resource contention Distributed system ⇒ distributed resource allocation

  8. Predictable Performance is Hard GET SET SET GET 1kB 10B 10B 1kB (large reads) (small reads) (large writes) (small writes) Z Z Z Z Z Z Y Y Y Y Y Y Y Y T T T T F F F F F F F F F F SS SS SS SS Multiple co-located tenants ⇒ resource contention Distributed system ⇒ distributed resource allocation Skewed object popularity ⇒ variable per-node demand Disparate workloads ⇒ different bottleneck resources

  9. Tenants Want System-wide Resource Guarantees demandz = 120 kreq/s ratez = 80 kreq/s 80 kreq/s 120 kreq/s 160 kreq/s 40 kreq/s Zynga Yelp TP Foursquare demandf = 120 kreq/s Z Z Z Y Y Y Y T T F F F F F ratef = 120 kreq/s Hard limits ⇒ lower utilization Shared Key-Value Storage SS SS SS SS Multiple co-located tenants ⇒ resource contention Distributed system ⇒ distributed resource allocation Skewed object popularity ⇒ variable per-node demand Disparate workloads ⇒ different bottleneck resources

  10. Pisces Provides Weighted Fair-shares wz = 20% wy = 30% wt = 10% wf = 40% Zynga Yelp TP Foursquare demandz = 30% Z Z Z Y Y Y Y T T F F F F F ratez = 30% demandf = 30% ratef = 30% Shared Key-Value Storage Max-min fair share ⇒ lower bound on system performance SS SS SS SS Multiple co-located tenants ⇒ resource contention Distributed system ⇒ distributed resource allocation Skewed object popularity ⇒ variable per-node demand Disparate workloads ⇒ different bottleneck resources

  11. Pisces: Predictable Shared Cloud Storage • Pisces • Per-tenant max-min fair shares of system-wide resources ~ min guarantees, high utilization • Arbitrary object popularity • Different resource bottlenecks • AmazonDynamoDB • Per-tenant provisioned rates ~ rate limited, non-work conserving • Uniform object popularity • Single resource (1kB requests)

  12. RS FQ PP Predictable Multi-Tenant Key-Value Storage Tenant A Tenant B VM VM VM VM VM VM GET 1101100 Controller RR

  13. WA PP FQ RS WA2WB2 Predictable Multi-Tenant Key-Value Storage WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM GET 1101100 Controller RR WA2 WB2 WA1 WB1

  14. PP RS FQ WA WA2WB2 Strawman: Place Partitions Randomly WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM Controller RR

  15. WA FQ RS PP WA2WB2 Strawman: Place Partitions Randomly WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM Controller Overloaded RR

  16. WA PP RS FQ WA2WB2 Pisces: Place Partitions By Fairness Constraints WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM Collect per-partition tenant demand Controller RR Bin-pack partitions

  17. WA FQ RS PP WA2WB2 Pisces: Place Partitions By Fairness Constraints WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM Controller RR Results in feasible partition placement

  18. RS WA FQ PP WA2WB2 Strawman: Allocate Local Weights Evenly WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM Controller excessive demand Overloaded RR WA1 =WB1 WA2 =WB2

  19. FQ WA PP RS WA2WB2 Pisces: Allocate Local Weights By Tenant Demand WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM Compute per-tenant +/- mismatch max mismatch Controller M/M/1 queuing delay RR WA1 =WB1 WA2 =WB2 WA1 > WB1 WA2 < WB2 Reciprocal weight swap A←B A→B

  20. FQ WA RS PP WA2WB2 Strawman: Select Replicas Evenly WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM GET 1101100 Controller 50% 50% RR WA1 > WB1 WA2 < WB2

  21. WA PP RS FQ WA2WB2 Pisces: Select Replicas By Local Weight WeightA WeightB Tenant A Tenant B VM VM VM VM VM VM GET 1101100 Controller detect weight mismatch by request latency 50% 40% 60% 50% RR WA1 > WB1 WA2 < WB2

  22. FQ WA PP RS WA2WB2 out req out req Strawman: Queue Tenants By Single Resource Tenant A Tenant B VM VM VM VM VM VM Bandwidth limited Request Limited Controller bottleneck resource (out bytes) fair share RR GET 0100111 GET 1101100 WA1 > WB1 WA2 < WB2

  23. FQ WA PP RS WA2WB2 out req out req Pisces: Queue Tenants By Dominant Resource Tenant A Tenant B VM VM VM VM VM VM Bandwidth limited Request Limited Controller dominant resource fair share RR bottlenecked by out bytes Track per-tenant resource vector WA2 < WB2

  24. FQ WA PP RS WA2WB2 Weight Allocations Replica Selection Policies Pisces Mechanisms Solve For Global Fairness Maximum bottleneck flow weight exchange feasible partition placement Controller global FAST-TCP basedreplica selection demand-driven weight allocation fairness and capacity constraints RR System Visibility weight-sensitive selection policy ... RR DRR token-basedDRFQ scheduler dominant resource fair shares SS local ... SS microseconds seconds minutes Timescale

  25. Issues • RS < -- > consistency • Turn off RS. Why does it not matter? • What happens to previous picture? • Translating SLAs to weights • How are SLAs specified? • Overly complex mechanisms: weight swap and FAST-based replica selection? • Why make it distributed? • Paper argues configuration is not scalable, but requires server configuration for FQ anyway • What happens in a congested network? • Assumes stable tenant workloads  reasonable? • Is DynamoDB bad?

  26. Partition Placement Weight Allocation Replica Selection Fair Queuing PP WA RS FQ Pisces Summary • PiscesContributions • Per-tenant weighted max-min fair shares of system-wide resources w/ high utilization • Arbitrary object distributions • Different resource bottlenecks • Novel decomposition into 4 complementary mechanisms

  27. Evaluation

  28. Evaluation • Does Pisces achieve (even) system-wide fairness? • Is each Pisces mechanism necessary for fairness? • What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? • Does Pisces provide local dominant resource fairness? • Does Pisces handle dynamic demand? • Does Pisces adapt to changes in object popularity?

  29. Evaluation • Does Pisces achieve (even) system-wide fairness? • Is each Pisces mechanism necessary for fairness? • What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? • Does Pisces provide local dominant resource fairness? • Does Pisces handle dynamic demand? • Does Pisces adapt to changes in object popularity?

  30. Pisces Achieves System-wide Per-tenant Fairness Ideal fair share: 110 kreq/s (1kB requests) Unmodified Membase Pisces 0.57 MMR 0.98 MMR 8 Tenants - 8 Client - 8 Storage Nodes Zipfian object popularity distribution Min-Max Ratio: min rate/max rate (0,1]

  31. WA PP RS RS FQ WA2WB2 Each Pisces Mechanism Contributes to System-wide Fairness and Isolation FQ RS FQ PP FQ PP WA WA Unmodified Membase 0.57 MMR 0.59 MMR 0.93 MMR 0.98 MMR 0.64 MMR 0.90 MMR 2xvs1x demand 0.58 MMR 0.74 MMR 0.96 MMR 0.89 MMR 0.97 MMR 0.36 MMR

  32. Pisces Imposes Low-overhead > 19% < 5%

  33. Pisces Achieves System-wide Weighted Fairness 0.98 MMR 0.89 MMR 0.91 MMR 4 heavy hitters 20 moderatedemand 40 lowdemand 0.91 MMR 0.56 MMR

  34. Pisces Achieves Dominant Resource Fairness 1kB workload bandwidth limited 10B workload request limited 76% of bandwidth 76% of request rate GET Requests (kreq/s) Bandwidth (Mb/s) 24% of request rate Time (s) Time (s)

  35. Pisces Adapts to Dynamic Demand Tenant Demand Constant Diurnal (2x wt) Bursty even ~2x

More Related