Objectives

2. Objectives • Provide an intuitive non-mathematical presentation of QoS-related queuing phenomena • Explain the available solution approaches and associated trade-offs • Give guidelines on how to match applications and solutions

3. Outline • Basic concepts • Source models • Service models (demo) • Single-queue systems • Priority/shared service systems • Networks of queues • Hybrid simulation (demo)

4. Outline • Basic concepts • Performance measures • Solution methodologies • Queuing system concepts • Stability and steady-state • Little’s law • Causes of delay and bottlenecks • Source models • Service models(demo) • Single-queue systems • Priority/shared service systems • Networks of queues • Hybrid simulation (demo)

5. Performance Measures • Delay • Delay variation (jitter) • Packet loss • Efficient sharing of bandwidth • Relative importance depends on traffic type (audio/video, file transfer, interactive) • Challenge: Provide adequate performance for (possibly) heterogeneous traffic

6. Solution Methodologies • Analytical results (formulas) • Pros: Quick answers, insight • Cons: Often inaccurate or inapplicable • Explicit simulation • Pros: Accurate and realistic models, broad applicability • Cons: Can be slow • Hybrid simulation • Intermediate solution approach • Combines advantages and disadvantages of analytical and explicit simulation

7. Examples of Applications

8. Queuing System Concepts: Arrival Rate, Occupancy, Time in the System • Queuing system • Data network where packets arrive, wait in various queues, receive service at various points, and exit after some time • Arrival rate • Long-term number of arrivals per unit time • Occupancy • Number of packets in the system (averaged over a long time) • Time in the system (delay) • Time from packet entry to exit (averaged over many packets)

9. Stability and Steady-State • A single queue system is stable if packet arrival rate < system transmission capacity • For a single queue, the ratio packet arrival rate / system transmission capacity is called the utilization factor • Describes the loading of a queue • In an unstable system packets accumulate in various queues and/or get dropped • For unstable systems with large buffers some packet delays become very large • Flow/admission control may be used to limit the packet arrival rate • Prioritization of flows keeps delays bounded for the important traffic • Stable systems with time-stationary arrival traffic approach a steady-state

10. Little’s Law • For a given arrival rate, the time in the system is proportional to packet occupancy N =  T where N: average # of packets in the system : packet arrival rate (packets per unit time) T: average delay (time in the system) per packet • Examples: • On rainy days, streets and highways are more crowded • Fast food restaurants need a smaller dining room than regular restaurants with the same customer arrival rate • Large buffering may cause large delays

11. Explanation of Little’s Law • Amusement park analogy: people arrive, spend time at various sites, and leave • They pay $1 per unit time in the park • The rate at which the park earns is $N per unit time (N: average # of people in the park) • The rate at which people pay is $ T per unit time (: traffic arrival rate, T: time per person) • Over a long horizon: Rate of park earnings = Rate of people’s payment or N =  T

12. Delay is Caused by Packet Interference • If arrivals are regular or sufficiently spaced apart, no queuing delay occurs Regular Traffic Irregular but Spaced Apart Traffic

13. Burstiness Causes Interference • Note that the departures are less bursty

14. Burstiness ExampleDifferent Burstiness Levels at Same Packet Rate Source: Fei Xue and S. J. Ben Yoo, UCDavis, “On the Generation and Shaping Self-similar Traffic in Optical Packet-switched Networks”, OPNETWORK 2002

15. Packet Length Variation Causes Interference Regular arrivals, irregular packet lengths

16. High Utilization Exacerbates Interference As the work arrival rate: (packet arrival rate * packet length) increases, the opportunity for interference increases

17. Bottlenecks • Types of bottlenecks • At access points (flow control, prioritization, QoS enforcement needed) • At points within the network core • Isolated (can be analyzed in isolation) • Interrelated (network or chain analysis needed) • Bottlenecks result from overloads caused by: • High load sessions, or • Convergence of sufficient number of moderate load sessions at the same queue

18. Bottlenecks Cause Shaping • The departure traffic from a bottleneck is more regular than the arrival traffic • The inter-departure time between two packets is at least as large as the transmission time of the 2nd packet

19. Bottlenecks Cause Shaping Incoming traffic Interarrival times Outgoing traffic Interdeparture times Bottleneck 90% utilization # of packets # of packets Exponential inter-arrivals Fixed packet length sec sec Transmission time

20. Incoming traffic Interarrival times Outgoing traffic Interdeparture times # of packets Bottleneck 90% utilization Small sec Medium Large

21. Packet Trains Histogram of inter-departure times for small packets # of packets sec

22. Outline • Basic concepts • Source models • Poisson traffic • Batch arrivals • Example applications – voice, video, file transfer • Service models (demo) • Single-queue systems • Priority/shared service systems • Networks of queues • Hybrid simulation (demo)

23. Poisson Process with Rate l • Interarrival times are independent and exponentially distributed • Models well the accumulated traffic of many independent sources • The average interarrival time is 1/ l (secs/packet), so l is the arrival rate (packets/sec)

24. Batch Arrivals • Some sources transmit in packet bursts • May be better modeled by a batch arrival process (e.g., bursts of packets arriving according to a Poisson process) • The case for a batch model is weaker at queues after the first, because of shaping

25. State 0 State 1 OFF ON Markov Modulated Rate Process (MMRP) • Extension: Models with more than two states and/or stochastic transmission process Stay in each state an exponentially distributed time Transmit according to a deterministic process at each state

26. Source Types • Voice sources • Video sources • File transfers • Web traffic • Interactive traffic Different application types have different QoS requirements • Delay • Jitter • Loss • Throughput

27. Source Type Properties

28. Typical Voice Source Behavior

29. MPEG1 Video Source Model • The MPEG1 MMRP model can be extremely bursty, and has “long range dependency” behavior due to the deterministic frame sequence Diagram Source: Mark W. Garrett and Walter Willinger, “Analysis, Modeling, and Generation of Self-Similar VBR Video Traffic, BELLCORE, 1994

30. Outline • Basic concepts • Source models • Service models • Single vs. multiple-servers • FIFO, priority, and shared servers • Demo • Single-queue systems • Priority/shared service systems • Networks of queues • Hybrid simulation (demo)

31. Device Queuing Mechanisms • Common queue examples for IP routers • FIFO: First In First Out • PQ: Priority Queuing • WFQ: Weighted Fair Queuing • Combinations of the above • Service types from a queuing theory standpoint • Single server (one queue - one transmission line) • Multiple server (one queue - several transmission lines) • Priority server (several queues with hard priorities - one transmission line) • Shared server (several queues with soft priorities - one transmission line)

32. Single Server FIFO • Single transmission line serving packets on a FIFO (First-In-First-Out) basis • Each packet must wait for all packets found in the system to complete transmission, before starting transmission • Departure Time = Arrival Time + Workload Found in the System + Transmission time • Packets arriving to a full buffer are dropped

33. Multiple Servers • Multiple packets are transmitted simultaneously on multiple lines/servers • Head of the line service: packets wait in a FIFO queue, and when a server becomes free, the first packet goes into service

34. Priority Servers • Packets form priority classes (each may have several flows) • There is a separate FIFO queue for each priority class • Packets of lower priority start transmission only if no higher priority packet is waiting • Priority types: • Non-preemptive (high priority packet must wait for a lower priority packet found under transmission upon arrival) • Preemptive (high priority packet does not have to wait …)

35. Priority Queuing • Packets are classified into separate queues • Based on source/destination IP address, source/destination TCP port, etc. • All packets in a higher priority queue are served before a lower priority queue is served • Typically in routers, if a higher priority packet arrives while a lower priority packet is being transmitted, it waits until the lower priority packet completes

36. Shared Servers • Again we have multiple classes/queues, but they are served with a “soft” priority scheme • Round-robin • Weighted fair queuing

37. Round-Robin/Cyclic Service • Round-robin serves each queue in sequence • A queue that is empty is skipped • Each queue when served may have limited service (at most k packets transmitted with k = 1 or k > 1) • Round-robin is fair for all queues (in terms of packet transmission rate) • Round-robin cannot be used to enforce bandwidth allocation among the queues.

38. Fair Queuing • This scheduling method is inspired by the “most fair” of methods • Transmit one bit from each queue in cyclic order (bit-by-bit round robin) • Skip queues that are empty • To approximate the bit-by-bit processing behavior, for each packet • We calculate upon arrival its “finish time under bit-by-bit round robin” and we transmit by FIFO within each queue • Transmit next the packet with the minimum finish time • Important properties • Priority is given to short packets • Equal bandwidth is allocated to all queues that are continuously busy

39. Weighted Fair Queuing • Fair queuing cannot be used to implement bandwidth allocation and soft priorities • Weighted fair queuing is a variation that corrects this deficiency • Let wk be the weight of the kth queue • Think of round-robin with queue k transmitting wk bits upon its turn • If all queues have always something to send, the kth queue receives bandwidth equal to a fraction wk / Si wi of the total bandwidth • Fair queuing corresponds to wk = 1 • Priority queuing corresponds to the weights being very high as we move to higher priorities • Implementation: For each packet • Calculate its “finish time” (under the weighted bit-by-bit round robin scheme) • Transmit the packet with the minimum finish time

40. Weighted Fair Queuing Illustration Weights: Queue 1 = 3 Queue 2 = 1 Queue 3 = 1

41. A Practical Combination (e.g. Cisco) • Example – voice in PQ, guaranteed b/w traffic in WFQs (all at middle priority), and best effort traffic in low priority queue

42. Demo: Comparing FIFO, WFQ, and PQ • 2 traffic streams mixing on a common interface • Video • FTP • Apply different service schemes • FIFO • PQ • WFQ • Run simulation and compare queuing delays

43. Demo: FIFO FIFO Bottleneck 90% utilization

44. Demo: FIFO Queuing Delay Applications have different requirements • Video • delay, jitter • FTP • packet loss Control beyond “best effort” needed • Priority Queuing (PQ) • Weighted Fair Queuing (WFQ)

45. Demo: Priority Queuing (PQ) PQ Bottleneck 90% utilization

46. PQ FTP FIFO PQ Video Demo: PQ Queuing Delays PQ FTP FIFO PQ Video

47. Demo: Weighted Fair Queuing (WFQ) WFQ Bottleneck 90% utilization

48. PQ FTP WFQ FTP FIFO WFQ/PQ Video Demo: WFQ Queuing Delays

49. Queuing: Summary Points • Choice of queuing mechanism can have a profound effect on performance • To achieve desired service differentiation, appropriate queuing mechanisms can be used • Some of the queuing mechanisms are complex, and may require simulation techniques to analyze behavior • Improper configuration (e.g., queuing mechanism selection or weights) may impact performance of low priority traffic

50. Outline • Basic concepts • Source models • Service models (demo) • Single-queue systems • M/M/1……M/M/m/k • M/G/1……G/G/1 • Demo: Analytics vs. simulation • Priority/shared service systems • Networks of queues • Hybrid simulation (demo)