Distributed Virtual-Time Scheduling in Rings (DVSR)

1. Distributed Virtual-Time Scheduling in Rings (DVSR) Chun-Hung Chen 2004.04.30 National Taipei University of Technology

2. Outlines • RPR Recall • Problems in RPR • Ring Ingress Aggregated with Spatial Reuse Fairness (RIAS) • Distributed Virtual-Time Scheduling in Rings (DVSR) • Simulation Results • Conclusions

3. RPR Recall • RPR stands for Resilient Packet Ring, which is in IEEE 802.17 Draft State • Dual rings structure with Destination strip mechanism

4. Traffic is classified in three classes: • Class A (A0 or A1), Class B (CIR or EIR), Class C • When congested, the station will compute its approximation fair rate by • Dividing the available bandwidth between all upstream stations that are currently sending frames through this station • Using its own current add rate

5. Two operation mode • Conservative Mode • Congested station will wait a FRTT to send a new fair rate if it is still in congestion • Aggressive Mode • Congested station sends new fair rate in every 100μs if it is still in congestion

6. Problems in RPR • Single Rate Controller • Per-destination rate controller is optional • Permanent Oscillation With Unbalanced Constant-Rate Traffic Inputs • Unbalanced traffic will trigger severe and permanent oscillations • Computed add_rate or Capacity/Active_Stations do not reflect the true situation • Throughput Loss • Utilization degrades due to oscillation • AM & CM • Convergence • Slow convergence time

8. … … allow_rate my_rate congested After approximately 500 iterations, converges to the fair rates in this case Throttle traffic and propagate rate upstream Throttle traffic • Transit traffic has priority over ingress “station” traffic • Each node measures my_rate of ingress traffic • If a node is congested: • send my_rate upstream • upstream nodes throttle to my_rate

9. The Problem with Darwin • my_rate is NOT the ring-wide fair rate • Example of permanent oscillation and throughput degradation in Darwin:

10. Model accurately matches simulation Aggressive Mode Conservative Mode Modeling RPR Oscillations (Analytical and Simulation Results)

11. RIAS • Ring Ingress Aggregated with Spatial Reuse Fairness • Define the level of traffic granularity for fairness determination at a link as an ingress-aggregated (IA) flow • Ensure maximal spatial reuse subject to the first constraint • Steps of RIAS • Allocate bandwidth on each link locally fair according to an ingress aggregated granularity (IA traffic) • Refine bandwidth allocation for each IA flow according to its egress point and bottlenecks • Reclaim unused bandwidth fairly by iterating • Highly Similar to Max-Min Flow Control

12. Comparison • Proportional Fair Allocation • Penalizes flows farther away from the destination • Important for TCP in the Internet (rate decrease with RTT) • Fairness with Ingress-Egress flow granularity • Incorrectly rewards nodes for spreading out traffic to many destination versus all to hub node

13. Illustration of RIAS Fair (1/3) • Parking Lot • 4 flows each receive rate ¼ 1/4 1/4 1/4 1/4

14. Illustration of RIAS Fair (2/3) • Parallel Parking Lot • Each flow receives rate ¼ on downstream link • Left 1-hop flow fully reclaims excess bandwidth (RIAS) 3/4 1/4 1/4 1/4 1/4

15. Illustration of RIAS Fair (3/3) • Upstream Parallel Parking Lot • Key points: • Flow granularity for fairness • Spatial reuse 1/2 1/4 1/4 1/4 1/4 1/4 1/2 3/4

16. .12 .16 .24 .48 Proportional Fair • “Proportional fairness” • Penalizes flows farther away from the hub • Important for TCP in the Internet (rate decreases with RTT) • TCP/GigE approximates this in the parking lot • Variants of all of these have been discussed and proposed in the RPR standard meetings

17. Ingress-Egress Flow Granularity • Fairness with Ingress-Egress flow granularity • Incorrectly rewards nodes for spreading out traffic to many destinations vs. all to hub node • Wrong flow granularity counts 6 flows and gives rate 1/6 • (RIAS-fair: all green flows together get ¼ vs ½)

18. DVSR • Nodes construct a proxy of virtual time at the ingress-aggregated flow granularity • Using per-ingress byte counts • The proxy is a lower bound on virtual time temporally aggregated over time and spatially aggregated over traffic flows sharing the same ingress point (IA flows)

19. Distributed Fair Bandwidth Allocation • Remote Fair Queuing • Control of upstream rate controllers via use of ingress-aggregated virtual time as a congestion message received from downstream nodes • Conceptually an ideal GPS processor • Delayed and Temporally Aggregated Control Information • Proxy of Virtual Time • Multinode RIAS Fairness • Three Steps to approximate RIAS

20. Remote Fair Queuing: Single Resource Illustration • Control of upstream rate controllers via downstream virtual time progression • True fair queueing replaced with rate controllers + multiplexer • Note: no packets queued in mux when D = 0

21. Packet Size Packet Size 1 1 t t 0 0 1 2 3 4 5 1 2 3 4 (a) Traffic Arrival for Flow 1 (b) Traffic Arrival for Flow 2 v( t ) 6 5 4 3 2 1 0 t 1 2 3 4 5 6 7 8 9 10 ( c) Virtual Time Example • Link capacity = 1 pkt/sec • T = 10 pkt transmission times • b = 0.8 (fraction of time busy) •  > 0 • Controller set at t for rates in [t-T- , t- ] Limiter value = 0.8

22. Step I: Local Fairness • Label nodes 1, …, N and links 1, …, N-1 • rij is the traffic demand between nodes i and j at a particular time instant • rin is the Ingress Aggregated traffic from ingress node i at link n rin = ∑j>nrij • The locally fair allocation on link n is Rin = max_mini(C,r1n,r2n,…,rin,…, rnn)

23. Footnote on max_min • What is max_mini( )? • The “textbook” definition of (locally) fair • Would be achieved by fair queueing if fair queueing was performed on ingress aggregates • Can write down the exact computation [BerGal92,p527] • Maximizing the network use allocated to the sessions with the minimum allocation

24. Step II: Ingress Fairly Sub-allocates Per-link Bandwidths • Rijn = max_minj(Rin,ri,n+1,ri,n+2,…,ri,j,…,ri,N) • Ingress has bandwidth Rin on link n and divides it fairly among flows traversing n • End-to-End rate is the bottleneck rate • ri,j = minnRijn, n=i, i+1,…,j-1

25. Step III: Iterate • There may be further bandwidth available for spatial reuse • Due to multiple congestion points • Iterate process such that all excess capacity is fairly reclaimed • Set new capacity to all unallocated capacity • Cn=Cn-∑ijRijn • Go to Step I

26. DVSR Protocol • Scheduling of Station versus Transit Packets • FIFO queue • Class A is not taken in consideration • Feedback Signal Computation • Feedback Signal Transmission • Control message is N bytes while there exist N stations • Each station i writes its value at i bytes • Rate Limit Computation • Suballocate its per-link fair rates to the flows with different egress nodes

27. Scheduling FIFO (or SP) Computation of feedback signal Byte count for each ingress node - lower bound of virtual time Order such that l1≤ l2 ≤…≤ lk DVSR Protocol

28. Analysis of DVSR • Fairness Bound • Lemma 1 • A node-backlogged flow in DVSR can be under-throttled by at most (1-(1/N))CT • Lemma 2 • A node-backlogged flow in DVSR can be over-throttled by at most (1-(1/N))CT • Lemma 3 • The service difference during any interval for two flows i and j with infinite demand is bounded by 2(C-(1/N)C)T under DVSR

29. Simulations Results • Fairness and Spatial Reuse • Fairness in the Parking Lot • Performance Isolation for TCP Traffic • RIAS versus Proportional Fairness for TCP Traffic • Spatial Reuse in the Parallel Parking Lot • Convergence Time Comparison

30. Fairness in the Parking Lot • Four constant-rate UDP flows sending at 622 Mbps • DVSR provides RIAS fair shares • GigE does not

31. Spatial Reuse in the Parallel Parking Lot CBR UDP flows sending at the link capacity • DVSR is within 1% of RIAS fair rates • GigE favors downstream flows & cannot achieve spatial reuse • Darwin achieves only if using “multi-choke” option

32. Upstream Parallel Parking Lot(Results in Unbalanced Traffic Even with Balanced Inputs) • Darwin oscillation range is 0.25 to 0.75 and throughput loss is 14% • Many other scenarios can result in traffic imbalances and throughput losses • DVSR within 0.1% of RIAS Darwin Behavior

33. RIAS vs. Proportional Fairness for TCP Traffic • Each flow =1 TCP micro flow (ftp/TCP Reno) • Rate within 1% of RIAS fair rates for 1 TCP micro-flow • GigE tends to provide “proportional fair” rates

34. Convergence Time in the Parking Lot DVSR Gandalf • CBR UDP flows with rate 0.4 (248.8Mbps) • Flow(1,5), (2,5), (3,5), (4,5) begin transmission at times 0.0, 0.1, 0.2, and 0.3 seconds respectively • Convergence time 0.2 msec for DVSR, 50 msec for Darwin • Richer feedback signal allows faster convergence

35. Inter-Node Performance Isolation of TCP/UDP Traffic • Flow (1,5) TCP micro-flows • Others are CBR UDP flows with rate 0.3 • More TCP micro-flows – DVSR able to achieve RIAS fairness • Darwin performance unknown (MAC sim incompatible with TCP)

36. Conclusions • Link capacity does not be considered in RPR • Do my_rate and forward_rate in RPR fit the bandwidth allocation? • DVSR approximate RIAS quicker than RPR • RPR may have better performance if feedback mechanism is modified

37. Reference • V. Gambiroza, P. Yuan, B. Balzano, Y. Liu, S.Sheafor, “Design, Analysis, and Implementation of DVSR: A Fair High-Performance Protocol for Packet Rings”, IEEE/ACM Transactions on Networking, Feb. 2004 • F. Davik, M.Yilmaz, S. Gjessing, N. Uzun, “IEEE 802.17 Resilient Packet Ring Tutorial”, IEEE Communicaion Magazine, Mar. 2004 • http://www.ece.rice.edu/networks/RPR/