Congestion Control on High-Speed Networks

1. Congestion Control on High-Speed Networks Injong Rhee, Lisong Xu Computer Science Department North Carolina State University

2. Outline • Motivation • Related Work • BIC-TCP • Future Work

3. NewYork ABILENE UK It SuperJANET4 GARR-B STARLIGHT ESNET GENEVA GEANT CALREN NL STAR-TAP SURFnet Fr Renater High-Speed Networks • Many high-speed networks are being developed. • BW: 10Gbps, and expected to be upgraded to higher speeds in the near future.

4. NewYork ABILENE UK SuperJANET4 NL STARLIGHT SURFnet ESNET GEANT GENEVA It GARR-B CALREN STAR-TAP Fr Renater ESNET: Energy Science Network ESNET • ESNet is funded by the Department of Energy to provide a high-speed network serving thousands of scientists worldwide

5. High-Speed Applications Weather Simulation Video Conference Telemedicine High-Speed Applications Transport Protocols be able to transfer a large amount of data over a long distance within a short amount of time High-Speed Networks

6. TCP Performance Utilization of a link with 5 TCP connections Cannot fully utilize the huge capacity of high-speed networks! NS-2 Simulation (100 sec) • Link Capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps, • Drop-Tail Routers, 0.1BDP Buffer • 5 TCP Connections, 100ms RTT, 1000-Byte Packet Size Ns-2: capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps 100 sources, 100 ms round trip propagation delay, 1500-Byte Packet Size

7. cwnd = cwnd + 1 cwnd = cwnd * (1-1/2) Packet loss Packet loss TCP Packet loss Packet loss cwnd Slow start Congestion avoidance Time (RTT) TCP Congestion Control • The instantaneous throughput of TCP is controlled by a variable cwnd, • TCP transmits approximately a cwnd number of packets per RTT (Round-Trip Time).

8. 1.4 hours 1.4 hours 1.4 hours 100,000 10Gbps 50,000 5Gbps TCP over High-Speed Networks • A TCP connection with 1250-Byte packet size and 100ms RTT is running over a 10Gbps link (assuming no other connections, and no buffers at routers) slow increase TCP Packet loss Packet loss Packet loss Packet loss big decrease cwnd Slow start Congestion avoidance Time (RTT)

9. Response Function of TCP is : • R : Average Throughput • MSS: Packet Size • RTT: Round-Trip Time • P : Packet Loss Probability J. Padhye, V. Firoio, D. Towsley, J. Kurose, "Modeling TCP Throughput: a Simple Model and its Empirical Validation", Proceedings of SIGCOMM 98 Response Function of TCP • Response function of TCP is the average throughput of a TCP connection in terms of the packet loss probability, the packet size, and the round-trip time.

10. Response Function of TCP 10Gbps requires a packet loss rate of 10-10, or correspondingly a link bit error rate of at most 10-14, which is an unrealistic (or at least hard) requirement for current networks Assuming 1250-Byte packet size, and 100ms RTT

11. Outline • Motivation • Related Work • BIC-TCP • Future Work

12. Proposed High-Speed Protocols • Window-Based Protocols • AIMD (Additive Increase Multiplicative Decrease) • Jumbo Frame, GridFTP, PFTP, PSockets • HSTCP (High-Speed TCP) by Sally Floyd at ICIR, Berkeley • STCP (Scalable TCP) by Tom Kelly at Cambridge University • FAST (Fast AQM Scalable TCP) by Steven Low at California Institute of Technology • Rate-Based Protocols • SABUL (Simple Available Bandwidth Utilization Library ) by Robert Grossman at University of Illinois at Chicago window-based protocols are known for safer incremental deployment. D. Bansal, H. Balakrishnan, S. Floyd, and S. Shenker, "Dynamic behavior of slowly responsive congestion controls", In Proceedings of SIGCOMM 2001, San Diego, California.

13. cwnd = cwnd + 1 cwnd = cwnd + 32 cwnd = cwnd * (1-1/2) cwnd = cwnd * (1-1/8) Packet loss Packet loss AIMD (Additive Increase Multiplicative Decrease) • AIMD increases cwnd by a larger number, say 32, instead of 1 per RTT. • After a packet loss, AIMD decreases cwnd by 1/8, instead of 1/2 TCP Packet loss Packet loss cwnd Slow start Congestion avoidance Time (RTT)

14. Response Function of AIMD • TCP: • AIMD: The throughput of AIMD is always about 13 times larger than that of TCP

15. Properties of AIMD Bandwidth Scalable • Bandwidth Scalability The ability to achieve 10Gbps with a reasonable packet loss probability NOT TCP Friendly • TCP-Friendliness The ability to share bandwidth with TCP connections on low-speed networks

16. cwnd = cwnd + 1 cwnd = cwnd + 0.01*cwnd cwnd = cwnd * (1-1/2) cwnd = cwnd * (1-1/8) Packet loss Packet loss STCP (Scalable TCP) • STCP adaptively increases cwnd, and decreases cwnd by 1/8. TCP Packet loss Packet loss cwnd Slow start Congestion avoidance Time (RTT)

17. cwnd = cwnd * (1-1/2) cwnd = cwnd * (1-dec(cwnd)) cwnd = cwnd + 1 cwnd = cwnd + inc(cwnd) Packet loss Packet loss HSTCP (High Speed TCP) • HSTCP adaptively increases cwnd, and adaptively decreases cwnd. • The larger the cwnd, the larger the increment, and the smaller the decrement. TCP Packet loss Packet loss cwnd Slow start Congestion avoidance Time (RTT)

18. Response Functions of HSTCP and STCP Bandwidth Scalable • HSTCP: • STCP: HSTCP and STCP are both bandwidth scalable and TCP friendly TCP Friendly

19. Outline • Motivation • Related Work • BIC-TCP • Why another one? • Protocol Design • Parameter Setting • Simulations & Experiments • Future Work

20. RTT Fairness • Different connections may have quite different round-trip times, and a good protocol should allocate bandwidth fairly among those connections • RTT fairness index = throughout ratio of two flows with different RTTs

21. RTT Fairness on Low-Speed Networks • For a protocol with the following response function, where c and d are protocol-related constants. • The RTT Fairness Index (or the throughput ratio of two flows) on low-speed networks is • On low speed networks, different protocols have the same RTT fairness Lisong Xu, Khaled Harfoush, and Injong Rhee, "Binary Increase Congestion Control for Fast Long-Distance Networks", in Proceedings of IEEE INFOCOM 2004, March, 2004, HongKong

22. RTT Fairness on High-Speed Networks • For a protocol with the following response function, where c and d are protocol-related constants. • The RTT Fairness Index (or the throughput ratio of two flows) on high-speed networks is • On high speed networks, the RTT fairness of a protocol depends on the exponent d in the response function Lisong Xu, Khaled Harfoush, and Injong Rhee, "Binary Increase Congestion Control for Fast Long-Distance Networks", in Proceedings of IEEE INFOCOM 2004, March, 2004, HongKong

23. Slope Determines the RTT Fairness • If the response function is: then the RTT Fairness is: • The figure is shown in log-log scale, so exponent d in the response function is the slope of the line in the figure • The slope of the line determines the RTT fairness of a protocol on high-speed networks

24. Simulation Results of RTT Fairness Throughout ratio of two flows on a 2.5Gbps Link Simulation setup: BDP Buffer, Drop Tail, Reverse Traffic, Forward Background Traffic (short-lived TCP, Web Traffic)

25. Simulation Result of 2 STCP Flows Throughput of two STCP flows with different RTTs on a 2.5Gbps link Simulation setup: 2.5Gbps, BDP Buffer, Drop Tail, Reverse Traffic, Forward Background Traffic (short-lived TCP, Web Traffic)

26. Design Goal Scalability, RTT Fairness TCP Fairness

27. Outline • Motivation • Related Work • BIC-TCP • Why another one? • Protocol Design • Parameter Setting • Simulations & Experiments • Future Work

28. cwnd = cwnd + 1 cwnd = cwnd + f(cwnd, history) cwnd = cwnd * (1-1/2) cwnd = cwnd * (1-1/8) Packet loss Packet loss BIC (Binary Increase Congestion control) • BIC adaptively increase cwnd, and decrease cwnd by 1/8 TCP Packet loss Packet loss cwnd Slow start Congestion avoidance Time (RTT)

29. A Search Problem • A Search Problem • We consider the increase part of congestion avoidance as a search problem, in which a connection looks for the available bandwidth by comparing its current throughput with the available bandwidth, and adjusting cwnd accordingly. • Q: How to compare R with A? R = current throughput = cwnd/RTT A = available bandwidth • A: Check for packet losses • No packet loss: R <= A • Packet losses : R > A • How does TCP find the available bandwidth? • Linear search while (no packet loss){ cwnd++; }

30. Available Bandwidth Linear Search

31. BIC: Binary Search with Smax and Smin • BIC - Binary search while (Wmin <= Wmax){ inc = (Wmin+Wmax)/2 - cwnd; if (inc > Smax) inc = Smax; else if (inc < Smin) inc = Smin; cwnd = cwnd + inc; if (no packet losses) Wmin = cwnd; else break; } • Wmax: Max Window • Wmin: Min Window • Smax: Max Increment • Smin: Min Increment

32. Smin Smax Binary Search with Smax and Smin Available Bandwidth Wmax Wmin

33. Binary Increase Congestion Control (BIC) Binary search increase Additive increase

34. Outline • Motivation • Related Work • BIC-TCP • Why another one? • Protocol Design • Parameter Setting • Simulations & Experiments • Future Work

35. Setting Smax • Response Function of BIC on high-speed networks Bandwidth scalability • Bandwidth scalability of BIC depends only on Smax • RTT Fairness of BIC on high-speed networks is the same as that of AIMD

36. Setting Smin • Response Function of BIC on low-speed networks • TCP-friendliness of BIC depends only on Smin TCP friendliness

37. Response Functions Bandwidth scalability RTT Fairness TCP-Friendliness

38. Outline • Motivation • Related Work • BIC-TCP • Why another one? • Protocol Design • Parameter Setup • Simulations & Experiments • Future Work

39. Bandwidth Scalability NS-2 Simulation (100 sec) • Link Capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps, • Drop-Tail Routers, 0.1BDP Buffer • 5 Connections, 100ms RTT, 1000-Byte Packet Size Ns-2: capacity = 155Mbps, 622Mbps, 2.5Gbps, 5Gbps, 10Gbps 100 sources, 100 ms round trip propagation delay, 1500-Byte Packet Size

40. TCP - Friendliness Simulation setup: 20Mbps, BDP Buffer, Drop Tail, Reverse Traffic 2 TCP flows, 2 high-speed flows, and some background traffic

41. RTT Fairness • Throughput ratio of two flows with different RTTs on a 2.5Gbps link Simulation setup: BDP Buffer, Drop Tail, Reverse Traffic, Forward Background Traffic (short-lived TCP, Web Traffic)

42. Buffer Size and Throughput

43. Queue Size Vs. Buffer Size

44. Packet loss vs. Buffer Size

45. Simulation Summary

46. SLAC Experiments • Evaluation of Advanced TCP Stacks on Fast Long-Distance Production Networks by SLAC (Stanford Linear Accelerator Center) • Linux TCP • Parallel TCP • HighSpeed TCP • Scalable TCP • HighSpeed TCP Low Priority • Hamilton TCP • BIC TCP • “BIC TCP overall performs very well in our tests” • http://www-iepm.slac.stanford.edu/bw/tcp-eval/

47. Outline • Motivation • Related Work • BIC-TCP • Future Work

48. Conclusion and Future Work • Conclusion • Designed a congestion control protocol, which is scalable, TCP-friendly, and has a good RTT fairness. • "Binary Increase Congestion Control for Fast Long-Distance Networks", in Proceedings of IEEE INFOCOM 2004, March, 2004, HongKong • Our work has been reported by some newspapers. • NBC (US) • The Washington Times (US) • The Times of India (India) • Sina (China) • Future Work • Further improve the performance of BIC • Solve other problems in high-speed networks

49. CUBIC

50. More information: http://www.csc.ncsu.edu/faculty/rhee/export/bitcp Thank you! Any questions?