Adaptive CPU Allocation for Software based Router Systems

1. Adaptive CPU Allocation forSoftware basedRouter Systems Puneet Zaroo

2. Software based routers • Implement packet forwarding/processing in software. • E.g a PC with multiple NICs. • Provide value added services like encryption, network address translation esp. at the network edge. • Issues • Software architecture. • Per flow threads / per-packet threads • Division of input, forwarding and output functions • CPU scheduling. • How to determine CPU shares • How to enforce CPU shares.

3. Objective • Leverage the advantages of a component based software router system. • Flexibility in designing routers • Reusability of software components • Dynamic addition of element modules • Overlay a QoS provisioning mechanism on top of the component based system. • Develop an adaptive QoS system • Adaptive to varying input rate and per-packet processing costs.

4. Some Software Router Systems • Router Plugins : ETH Zurich, Uwash St. Louis • Per flow code modules or plugins. • Implemented in the NetBSD kernel. • Click Modular router : MIT • Routers made of elements composed into a flow graph. • ANTS • Programmable and customizable networks. • Customizable applications acting on packets / packets carrying code as well as data. • X-kernel : University of Arizona • Object oriented interface to protocols. • Can be used on end systems as well as routers. • Scout : University if Arizona, Princeton University • Communication oriented OS based on x-kernel. • Path based abstraction. • Advanced CPU scheduling.

5. OS support for CPU scheduling • Scout • Proportional scheduling. • CPU balance (extension of work on livelock) • Resource Containers : Rice University • Decoupling of protection domain/resource domain. • Proper accounting of resources to processes. • Resources include threads as well as kernel data structures and memory,bound to containers. • E.g a web server serving multiple connections. • Processor Capacity reserves : CMU • Provides support for both time-sharing and real-time systems. • The OS enforces the reservations (cpu share, time period). • Applications free to change their reservations subject to admission control. • Nemesis : Cambridge • OS does low level resource multiplexing. • Avoiding QoS cross-talk • Support for I/O in user level libraries.

6. Click • Composable flow-graphs from router elements • Packets travel along graph edges • Element based processing (push/pull). • Element based scheduling. • Multithreaded SMP Click • Issues in flow level QoS on top of an element based architecture • Flow level accounting and scheduling. • CPU balance b/w input, output and processing. • CPU conservation of idle elements.

7. CROSS/Linux – Resource reservation with containers • Containers • Group of related elements • Elements doing per flow processing. • Container – CPU resource reservation unit. • Why use containers and not flows ? • Types of Containers • Input • Output • Forwarding • Best Effort • QoS - Packet rate reservations

8. Example Router Configuration

9. CROSS/Linux - CPU scheduling • Three level scheduler • Linux schedules CROSS • Linux process scheduler • CROSS schedules Containers • Proportional (Dynamic stride scheduling) • Containers schedule Elements • Simple Round Robin scheduling

10. CROSS/Linux – Architectural Enhancements to Click • CPU conservation through sleep/wakeup • Elements tested for scheduling eligibility • Containers tested for scheduling eligibilty • Notifier Queues - wake up elements (make eligible for scheduling) • Delayed wakeup • Network interface Input Element • Switching between polling and interrupt • Based on a threshold packet input rate to reduce programmed I/O overhead • Topology discovery • Discovering input/output queues for a container

11. CROSS/Linux – Enhancements to Click • virtual Interface queues – especially for interface statistics gathering • Linux /proc interface – • One directory for each container • Directory provides information about • Container tickets • CPU cycles consumed • Packet rate/drop rate • Elements • Input/Output queues • Set container tickets

12. CROSS/Linux – Share adaptation • Why ? • Inability to do a-priori CPU share calculation • Variations in packet input rate • Variations in per-packet processing cost • How ? • Scheduler for each container keeps track of • Packet input rate. • Packet drop rate. • CPU cycles used. • Recomputes container shares to remove packet drops.

13. CROSS/Linux – Share adaptation • Statistics maintained by Queues • Packet rates • Packet drop rates • Queues used to connect containers • Packet pass/drop rates at Queues indicate the difference between the required and the actual CPU shares for the container

14. Share adaptation Algorithm • Invoked every 1 second • Notation used • T – Ticket share • C – Current CPU share • p – Input packet rate • d – packet drop rate • m – maximum input rate • General idea • Increase ticket share of a container so that the drop rate is removed at all the containers

15. Input Container share adaptation (Issues) • Pass as many packets as possible upto a maximum. • How to arrive at this maximum? • Forwarding more than the maximum adversely affects the effective router throughput. • Reduce share on observing over allocation.

16. Input Container – Share adaptation(Algorithm) if p > m /* Input rate too high */ /* reduce share */ T = C * (m/p) else if d > 0 /* Increase share to */ /*remove packet drops */ drate = min(d + p,m) T = C * (drate/p) else if (T – C) >= delta /* Over allocation */ /* reduce share */ T = T – eps

17. QoS container – Share adaptation(Issues) • Always forward till reserved rate. • Target a forwarding rate range. • Reduce share in case of over allocation

18. QoS container – Share adaptation(Algorithm) If p ε[ R – Dt, R + Dt] /* No change */ return if p > R + Dt /* Reduce share */ T = C * (R/p) else if d > 0 /* Increase share */ drate = min(p + d,R) T = C * (drate/p) else if (T-C) >= delta /* Reduce share */ T = T – eps

19. Output Container – Share adaptation (Issues) • Try to forward all that is received • Throttling if any has happened upstream • Reduce share in case of over allocation

20. Output Container – Share adaptation (Algorithm) if d > 0 /*Increase share */ T = C * (1 + d/p) else if (T – C) >= delta / * Reduce Share */ T = T - eps

21. Best Effort Container – Share adaptation • No action taken • System makes no guarantees

22. Discussion • Packet rate based reservation • Reservations based on packet rates more intuitive • CPU shares may vary for the same packet rates • C (Actual share) - How is it calculated? • Input container • Only include CPU cycles used in packet processing as opposed to idle polling. • Other containers • Easy to calculate since no idle polling. • m (Maximum forwarding rate) • Constant determined at router initialization • Evaluated at each iteration

23. Evaluation • Using a simulator • Calculates the forwarding rate , drop rate based on the CPU shares. • Mimics the actions of the adaptive algorithm • Eases loading the “router” and testing of diverse workloads • Using a real implementation • CROSS/Linux on 866 Mhz Pentium III CPU.

24. Adaptive vs. Non Adaptive(Experimental setup) • Input (2 µs), Output (2 µs) , Best Effort Container (6 µs). • Router – 1 MHz CPU => max forwarding = 100,000 packets/s • Static ticket assignment = 1:1:1 • Input varied for 0 to 110,000 packets/s in increments of 10,000 packet/s every 10s.

25. Adaptive vs. Non Adaptive(Variation with time)

26. Adaptive vs. Non Adaptive(Maximum loss free forwarding rate)

27. Variable packet processing time(Experimental Setup) • Input (2µs), Best Effort/QoS (6µs), Output Container (2µs) • Observe different convergence behavior for QoS / Best Effort • Router – 1 MHz CPU => max forwarding rate initially = 100,000 packets/s • Constant input = 50,000 packets/s • Per packet processing cost increased by 2 µs every 10 secs. • Max. forwarding rate = 50,000 packets/s at t=50s.

28. Variable packet processing time(Adaptive vs. Non Adaptive)

29. Variable packet processing time-(Best Effort vs. QoS)

30. Adaptation in m • Hard to determine m at router initialization • May vary with variations in per packet processing costs. m = maxi (TOTAL_CPU_CPS/cpu_cpp(ci)) where ciε C • TOTAL_CPU_CPS - Total CPU cycles per second available to the router • cpu_cpp(ci) - cycles/packet being used by the flow serviced by container ci cpu_cpp(ci) = cpu_cpi() + cpu_cycles(ci)/num_packets(ci) + cpu_cpo() • C - The set of containers servicing active flows

31. Fixed vs adaptive m - (Experimental setup) • Input (8µs), Best Effort/QoS (1µs), Output Container (1µs) • Router – 1 MHz CPU => max forwarding rate, initially = 100,000 packets/s • Constant input = 50,000 packets/s • Per packet processing cost increased by 2 µs every 5 secs • Max forwarding rate = 50,000 packets/s at t=30 s.

32. Fixed vs adaptive m - (Effective Best Effort Forwarding)

33. Fixed vs. adaptive m(Effective QoS forwarding)

34. Fixed vs. Adaptive m(Best Effort, QoS , Theoretical maximum)

35. Advanced Adaptation in m • Previous algorithm gives too much stress to the least expensive flow. • Fine if all packets destined for that flow. • The packet rate to different flows can be variable. • m =(TOTAL_CPU_CPS/weighted_cpu_cpp) • weighted_cpu_cpp = Σ (cpu_cpp(ci) * rate(ci))/ (Σ rate(ci)) where ciεC

36. Adaptive m vs. advanced adaptive m(Experimental Setup) • Input container (5 µs), Output Container(5 µs) • Router (1 MHz CPU) • 2 flows • QoS container (50,000 p/s,30 µs) => max forwarding rate achievable = 25,000 packets/s • Best Effort container (3 µs) => max forwarding rate achievable = 77,000 packets/s • Input rate to best effort container = 500 packets/s • Input rate to QoS container varied from 15,000 packets/s to 50,000 packets/s in increments of 5,000 packets/s every 5 s.

37. Adaptive m vs. advanced adaptive m(Forwarding rate vs. time)

38. Evaluation on a Router • CROSS/Linux software router platform • P III 866 MHZ pc. • 3 network interface cards.

39. QoS Forwarding (Experimental setup) • 866 MHz , PIII router • Input Container(4.5 µs) , Best Effort Container(3 µs),QoS container (32,000 packets/s), Output Container (4.9 µs) • 3 different per – packet processing costsfor the QoS container • 3, 9.7 and 15.2 µs • Input to QoS => 32,000 packets/ • Input to Best Effort => 27,000 packets/s

40. QoS Forwarding (Forwarding rate)

41. QoS Forwarding (Ticket Share)

42. QoS forwarding (Ticket Shares)

43. QoS forwarding (CPU Shares)

44. Effective Forwarding rate(Experimental setup) • Input (4.5 µs), best effort (8.3 µs) and output (4.9 µs) • Maximum forwarding rate = 57,000 p/s • 3 different scenarios • No Adaptation • CPU share Adaptation and m = 65000 packets/s • CPU share Adaptation and m = 110000 packets/s

45. Effective Forwarding rate

46. Future Work • Conjoint CPU – Buffer Allocation • Insufficient CPU share => always packet drops • Once sufficient CPU shares, more buffering => more efficiency • More buffering => higher packet delays and packets getting dropped at line cards. • Share adaptation between Linux/CROSS • Can use the SFQ scheduler already implemented

47. Conclusion • Provide a QoS provisioning layer on top of a component based system. • Adaptive in response to variable packet input and processing costs.

48. THANK YOU