Bridging Router Performance and Queuing Theory

1. Bridging Router Performance and Queuing Theory Dina Papagiannaki, Intel Research Cambridge with Nicolas Hohn, Darryl Veitch and Christophe Diot

2. Motivation • End-to-end packet delay is an important metric for performance and SLAs • Building block of end-to-end delay is through router delay • We measure the delays incurred by all packets crossing a single router

3. Overview • Full Router Monitoring • Delay Analysis • Modeling • Delay Performance: Understanding and Reporting • Causes of microcongestion

4. Measurement Environment

5. Full Router Monitoring • Gateway router • 2 backbone links (OC-48), 2 domestic customer links (OC-3, OC-12), 2 Asian customer links (OC-3) • 13 hours of trace collection on Aug. 14, 2003 • 7.3 billion packets – 3 TeraBytes of IP traffic • Monitor more than 99.9% of all through traffic • μs timestamp precision

6. Packet matching

7. Packet matching (cntd)

8. Overview • Full Router Monitoring • Delay Analysis • Modeling • Delay Performance: Understanding and Reporting • Causes of microcongestion

9. Not part of the system Store & Forward Datapath • Store: storage in input linecard’s memory • Forwarding decision • Storage in dedicated Virtual Output Queue (VOQ) • Decomposition into fixed-size cells • Transmission through switch fabric cell by cell • Packet reconstruction • Forward: Output link scheduler

10. Delays: 1 minute summary

11. Minimum Transit Time Packet size dependent minimum delay Δ(L), specific to router architecture and linecard technology

12. Not part of the system Δ(L) FIFO queue Store & Forward Datapath • Store: storage in input linecard’s memory • Forwarding decision • Storage in dedicated Virtual Output Queue (VOQ) • Decomposition into fixed-size cells • Transmission through switch fabric cell by cell • Packet reconstruction • Forward: Output link scheduler

13. Overview • Full Router Monitoring • Delay Analysis • Modeling • Delay Performance: Understanding and Reporting • Causes of microcongestion

14. Modeling

15. Modeling

16. Model Validation

17. Model validation

18. Error as a function of time

19. Modeling results • Our crude model performs well • Use effective link bandwidth (account for encapsulation) • The front end Δ only matters when the output queue is empty • The model defines Busy Periods: time between the arrival of a packet to theempty system and the time when the system becomes empty again.

20. Overview • Full Router Monitoring • Delay Analysis • Modeling • Delay Performance: Understanding and Reporting • Causes of microcongestion

21. Delay Performance • Packet delays cannot be inferred from output link utilization • Source of large delays: queue build-ups in output buffer • Busy Period structures contain alldelay information • Busy Period durations and idle duration contain all utilization information

22. Reporting BP Amplitude

23. Reporting BP Duration

24. Report BP joint distribution

25. Busy periods have a common shape

26. Reporting Busy Periods • Answer performance related questions directly • How long will a given level of congestion last? • Method: • Report partial busy period statistics A and D • Use “triangular shape”

27. Understanding Busy Periods

28. Reporting Busy Periods

29. Summary of modeling part • Results • Full router empirical study • Delay modeling • Reporting performance metrics

30. Overview • Full Router Monitoring • Delay Analysis • Modeling • Delay Performance: Understanding and Reporting • Causes of microcongestion

31. Causes of microcongestion • Reduction in link bandwidth from core to the access. • Multiplexing of multiple input traffic streams toward a single output stream. • Degree and nature of burstiness of input traffic stream(s).

32. Stretching and merging Queue Buildup!

33. Causes of microcongestion • Reduction in link bandwidth from core to the access. • Multiplexing of multiple input traffic streams toward a single output stream. • Degree and nature of burstiness of input traffic stream(s).

34. Multiplexing

35. Causes of microcongestion • Reduction in link bandwidth from core to the access. • Multiplexing of multiple input traffic streams toward a single output stream. • Degree and nature of burstiness of input traffic stream(s).

36. Traffic Burstiness • Duration and amplitude of busy periods depends on the spacing of packets at the input. • Highly clustered packets at the input are more likely to form busy periods.

37. A ts tA D Busy periods Maximum amplitude: 5 ms Maximum duration: 15 ms 120,000 busy periods > 1 ms

38. Methodology • Run semi-experiments • Simulate busy periods and measure their amplitude A(S, μ) under two different traffic scenarios, one that contains the effect studied and one that does not • Define a metric to quantitatively capture the studied effect

39. Reduction in Bandwidth

40. Amplification factor • Reference stream: • ST: traffic from a single OC-48 link • Output link rate: μi • Test stream: • Ss: traffic from a single OC-48 link • Output link rate: μo

41. Amplification factor (2)

42. Link multiplexing

43. Link multiplexing • Reference stream: • ST: output link traffic • Output link rate: μo • Test stream: • Si: traffic from a single OC-48 link • Output link rate: μo

44. Link multiplexing (2)

45. Flow burstiness Non-bursty flow Bursty flow

46. Flow Burstiness • Reference stream: • ST: input traffic stream from a single OC-48 link • Output link rate: μo • Test stream: • Sj: top 5-tuple flow OR the set of ALL bursty flows • Output link rate: μo

47. Flow burstiness

48. Summary • Methodology (and metrics) to investigate impact of different congestion mechanisms • In today’s access networks: • Reduction in link bandwidth plays a significant role • Multiplexing has a definite impact since individual links would not have led to similar delays • Flow burstiness does NOT significantly impact delay (bottleneck bandwidths too small to dominate the backbone) • Congestion may be the outcome of network design!

49. Thank you!

50. References • K. Papagiannaki, S. Moon, C. Fraleigh, P.Thiran, F. Tobagi, C. Diot.Analysis of Measured Single-Hop Delay from an Operational Backbone Network.In IEEE Infocom, New York, U.S.A., June, 2002. • N. Hohn, D. Veitch, K. Papagiannaki, C. Diot.Bridging router performance and queuing theory.To appear in ACM Sigmetrics, New York, U.S.A., June, 2004. • K. Papagiannaki, D. Veitch, and N. Hohn.Origins of Microcongestion in an Access Router.In Passive & Active Measurement Workshop, Antibes, France, April, 2004.