Sizing Router Buffers

1. Sizing Router Buffers Guido Appenzeller Thesis DefenseMay 24th, 2004

2. Routers need Packet Buffers • It’s well known that routers need packet buffers • It’s less clear why and how much • Goal of this work is to answer the question: How much buffering do routers need?

3. How much Buffer does a Router need? Source Destination Router C • Universally applied rule-of-thumb: • A router needs a buffer size: • 2T is the two-way propagation delay (or just 250ms) • C is capacity of bottleneck link • Context • Mandated in backbone and edge routers. • Appears in RFPs and IETF architectural guidelines.. • Usually referenced to Villamizar and Song: “High Performance TCP in ANSNET”, CCR, 1994. • Already known by inventors of TCP [Van Jacobson, 1988] • Has major consequences for router design 2T

4. Example • 10Gb/s linecard • Requires 300Mbytes of buffering. • Read and write 40 byte packet every 32ns. • Memory technologies • DRAM: require 4 devices, but too slow. • SRAM: require 80 devices, 1kW, $2000. • Problem gets harder at 40Gb/s • Hence RLDRAM, FCRAM, etc.

5. Outline of this Work • Main Results • The rule of thumb is wrong for a core routers today • Required buffer is instead of • Outline of this talk • Where the rule of thumb comes from • Why it is incorrect for a core router in the internet today • Correct buffer requirements for a congested router • Buffer requirements for short flows (slow-start) • Experimental Verification • Conclusion

6. Outline • The Rule of Thumb • Where does the rule of thumb comes from? (Answer: TCP) • Interaction of TCP flows and a router buffers • The buffer requirements for a congested router • Buffer requirements for short flows (slow-start) • Experimental Verification • Conclusion

7. Router TCP Only W=2packets may be outstanding Source Dest C’ > C C • TCP Congestion Window controls the sending rate • Sender sends packets, receiver sends ACKs • Sending rate is controlled by Window W, • At any time, only W unacknowledged packets may be outstanding • The sending rate of TCP is

8. Link not fully utilized Single TCP FlowRouter without buffers Only Wpackets may be outstanding Dest Source Router C’ > C C Congestion Window • Rule for adjusting W • If an ACK is received: W ← W+1/W • If a packet is lost: W ← W/2 W 8 4 t

9. For every W ACKs received, send W+1 packets RTT Window size t Single TCP FlowRouter with large enough buffers for full link utilization B Source Dest C’ > C C

10. Required buffer is height of sawtooth B 0 t

11. Buffer = rule of thumb

12. Microscopic TCP BehaviorWhen sender pauses, buffer drains one RTT Drop

13. Over-buffered Link

14. Under-buffered Link

15. Origin of rule-of-thumb • Before and after reducing window size, the sending rate of theTCP sender is the same • Inserting the rate equation we get • The RTT is part transmission delay T and part queuing delay B/C . We know that after reducing the window, the queuing delay is zero. 

16. Rule-of-thumb • Rule-of-thumb makes sense for one flow • Typical backbone link has > 20,000 flows • Does the rule-of-thumb still hold? • Answer: • If flows are perfectly synchronized, then Yes. • If flows are desynchronized then No.

17. Outline • The Rule of Thumb • The buffer requirements for a congested router • Synchronized flows • Desynchronized flows • The 2T×C/sqrt(n) rule • Buffer requirements for short flows (slow-start) • Experimental Verification • Conclusion

18. If flows are synchronized • Aggregate window has same dynamics • Therefore buffer occupancy has same dynamics • Rule-of-thumb still holds. t

19. When are Flows Synchronized? • Small numbers of flows tend to synchronize • In ns2 simulation they are synchronized • In at least some cases holds for real networks as well • Large aggregates of flows are not synchronized • For > 500 flows, synchronization disappears in ns2 • On a Cisco GSR, 100 flows were not synchronized • Measurements in the core give no indication of synchronization • C. Fraleigh, “Provisioning Internet Backbone Networks to support Latency Sensisitve Applications”, Ph.D. Thesis, Stanford • Hohn, Veitch, Papagiannaki and Diot – “Bridging Router Performance and Queuing Theory”

20. Probability Distribution Buffer Size If flows are not synchronized B 0

21. Quantitative Model model as where • For many de-synchronized flows • We assume congestion windows are independent • All congestion windows have the same probability distribution • Model congestion window of a flow as random variable • Now central limit theorem gives us the distribution of the sum of the window sizes

22. Buffer vs. Number of Flowsfor a given Bandwidth • For a given C, the window W scales with 1/n and thus • Standard deviation of sum of windows decreases with n • If for a single flow we have • Thus as n increases, buffer size should decrease

23. Required buffer size Simulation

24. Summary • Flows in the core are desynchronized • Substantial experimental evidence • Supported by ns2 simulations • For desynchronized flows, routers need only buffers of

25. Outline • The Rule of Thumb • The buffer requirements for a congested router • Buffer requirements for short flows (slow-start) • M/G/1 Model • Experimental Verification • Conclusion

26. Short Flows • So far we were assuming a congested router with long flows in congestion avoidance mode. • What about flows in slow start? • Do buffer requirements differ? • Answer: Yes, however: • Required buffer in such cases is independent of line speed and RTT (same for 1Mbit/s or 40 Gbit/s) • In mixes of flows, long flows drive buffer requirements • Short flow result relevant for uncongested routers

27. A single, short-lived TCP flowFlow length 62 packets, RTT ~140 ms 32 Flow Completion Time (FCT) 16 8 4 fin ackreceived syn 2 RTT

28. Modelling Short Flows Idea: Find buffer size by modelling queue behaviour • Problem: Arrival process is hard to model • Simplify by modelling bursts as independent • Buffer empties several times during one RTT Poisson arrivals offlows Service time is Lflow, the flow length in packets Poisson arrivals of bursts Service time is the lengthof the burst: 2,4,8,16…

29. M/G/1 Model for short flows • TCP flows generate independent bursts • Service times is burst length : • Poisson arrivals of rate • To verify if this approach works, let’s compare the average queue length in the model and in simulation

30. Average Queue length

31. Buffer B Q Packet Loss P(Q = x) Buffer Requirements for Short Flows • Buffers absorb fluctuations in queue, reduce packet loss • Reduce retransmits, Timeouts and thereby flow completion time • Utilization not a good measure of QoS as load << 1 • We can find a good upper bound for loss, if we have the queue length distribution for an infinite buffer • If a packet arrives and queue length is shorter than buffer, packet will not be dropped • Problem: For M/G/1 there is no closed form expression for the queue distribution

32. Queue Distribution We derived closed-form estimates of the queue distribution using Effective Bandwidth • Gives very good closed form approximation

33. Short Flow Summary • Buffer requirements for short flows • Can be modeled by M/G/1 model • Only depends on load and burst size distribution • Example - for bursts of up to size 16 at load 0.8 • For 1% loss probability B = 115 Packets • For 0.01% loss probability B = 230 packets etc. • Bursts of size 12 is maximum for Windows XP • Independent of line speed and RTT • In mixes of flows, long flows dominate buffer requirements

34. Outline • The Rule of Thumb • The buffer requirements for a congested router • Buffer requirements for short flows (slow-start) • Experimental Verification • Conclusion

35. Experimental Evaluation Overview • Simulation with ns2 • Over 10,000 simulations that cover range of settings • Simulation time 30s to 5 minutes • Bandwidth 10 Mb/s - 1 Gb/s • Latency 20ms -250 ms, • Physical router • Cisco GSR with OC3 line card • In collaboration with University of Wisconsin • Experimental results presented here • Long Flows - Utilization • Mixes of flows - Flow Completion Time (FCT) • Mixes of flows - Heavy Tailed Flow Distribution • Short Flows – Queue Distribution

36. Long Flows - Utilization (I)Small Buffers are sufficient - OC3 Line, ~100ms RTT 99.9% 2× 99.5% 98.0%

37. Long Flows – Utilization (II) Model vs. ns2 vs. Physical RouterGSR 12000, OC3 Line Card

38. Mixes of FlowsFlow Completion TimeFCT of 14 packet flows that share a link with long-lived flows.

39. Heavy-tailed flow length distribution • Experiment • Flow arrivals are a Poisson process • Flow lengths are Pareto distributed • Results • Buffers in the order of are still sufficient • Number of “long-lived” flows n is now defined as number of flows in congestion avoidance mode

40. Pareto Flow DistributionFinding the number of flows No. of flows in CA mode For buffer sizing, pick n = 100 time [seconds]

41. Pareto Flow Distribution Flow arrivalson link 1 Bottleneck utilization Buffer Occupancy

42. Short Flows – Queue DistributionM/G/1 Model vs. GSR 12000, OC3 Line Card

43. Outline • The Rule of Thumb • The buffer requirements for a congested router • Buffer requirements for short flows (slow-start) • Experimental Verification • Conclusion

44. Related Work • Related Publications • Buffer sizing • “High Speed TCP in ANS Net” - Villamizar and Song, ACCR 1994 • “TCP behaviour with many flows” – R. Morris, IEEE ICNP 1997 • “Scalable TCP congestion control” – R. Morris, INFOCOM 2000 • Queue Modelling • “Modelling, Simulation and Measurement of Queuing Delay” – Garetto and Towsley, SIGMETRICS 2003

45. Original Contributions • Main original contributions of this work • Routers only require buffers of instead of • Models on TCP buffer interaction for • Congestion avoidance mode • Slow start • Experimental Verification • Publication • “Sizing Router Buffers – Guido Appenzeller, Isaac Keslassy and Nick McKeown, to appear at SIGCOMM 2004

46. The Commercial Internet Today • Today’s internet differs from our assumptions • Core is overprovisioned, almost never congested • Access links are usually the bottleneck (DSL, Modem) • Flows are usually limited by Maximum Window size • Maximum Window is from 6-12 (Windows) to 42 (Unix) • Are the results still relevant? • Answer: Yes, we were intentionally pessimistic • Routers still needs to work in case of congestion • Even if this “worst case” scenario is rare • Slow access links and small TCP windows reduce buffer requirements further • Bursts are smoothed out • Converges towards constant rate source, poisson packet arrivals • We verified reduced buffer requirements experimentally

47. How much buffer does a router need? The old “Rule-of-Thumb” Our Contribution

48. Impact on Router Design • 10Gb/s linecard with 200,000 x 56kb/s flows • Rule-of-thumb: Buffer = 2.5Gbits • Requires external, slow DRAM • Becomes: Buffer = 6Mbits • Can use on-chip, fast SRAM • Completion time halved for short-flows • 40Gb/s linecard with 40,000 x 1Mb/s flows • Rule-of-thumb: Buffer = 10Gbits • Becomes: Buffer = 50Mbits

49. Questions?

50. Backup