In VINI Veritas Realistic and Controlled Network Experimentation

1. In VINI VeritasRealistic and Controlled Network Experimentation Andy Bavier Nick Feamster* Mark Huang Larry Peterson Jennifer Rexford Princeton University *Georgia Tech Modified and presented by Arjumand Younus and Moonyoung Chung

2. Outline • Motivation • Scientific Value and Importance of VINI • Fundamental Philosophy of VINI • Design Requirements • Strategy for building VINI • PL-VINI: prototype on PlanetLab • Experimental results • Conclusion

3. Motivation - How to Validate an Idea? Emulation VINI • Fixed, shared among many experiments • Runs real routing software • Exposes realistic network conditions • Gives control over network events • Carries traffic on behalf of real users Simulation Small-scale experiment Live deployment

4. Realism has Scientific Value The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ (I found it!) but ‘That’s funny …’ -- Isaac Asimov • Discover unanticipated interactions • Can only simulate / emulate known properties • Build richer models • Test robustness and scalability of ideas

5. Traffic Synthetic or traces Real clients, servers Arbitrary, emulated Synthetic or traces Actual network Real clients, servers Topology Traffic Inject faults, anomalies Observed in operational network Network Events Fundamental Philosophy “Controlled Realism” • Control: • Reproduce results • Methodically change or relax constraints • Realism: • Long-running services attract real “customers” • Forward high traffic volumes (Gb/s) • Robustly handle unexpected events

6. Overview • VINI requirements • Fixed, shared infrastructure • Flexible network topology • Expose/inject network events • External connectivity and routing adjacencies • PL-VINI: prototype on PlanetLab • Experimental results • Conclusion

7. Fixed Infrastructure VINI nodes embedded in Abilene

8. Shared Infrastructure Experiments given illusion of dedicated hardware

9. Shared Infrastructure Experiments given illusion of dedicated hardware

10. Flexible Topology VINI supports arbitrary virtual topologies

11. Flexible Topology VINI supports arbitrary virtual topologies

12. Network Events VINI can inject network failures

13. Network Events virtual link fails real link fails VINI can expose network failures (coming soon)

14. External Connectivity c s Experiments can carry traffic for real end-users

15. BGP BGP c BGP BGP External Routing Adjacencies s Experiments can participate in Internet routing

16. PlanetLab  VINI • Build VINI from PlanetLab, a global testbed for distributed services • Begun in 2002 • 700 nodes at 336 sites in 35 countries • 600 projects and 2500 researchers • Serves 3-4 TB/day to ~1M clients • MyPLC: PlanetLab software distribution • Anyone can run their own private PlanetLab

17. Node Mgr Local Admin VM1 VM2 VMn … Virtual Machine Monitor (VMM) (Linux++) PL-VINI: Prototype on PlanetLab • PlanetLab: testbed for planetary-scale services • Simultaneous experiments in separate VMs • Each has “root” in its own VM, can customize • Can reserve CPU, network capacity per VM PlanetLab node

18. Resource Isolation • Issue: Forwarding packets in user space • PlanetLab sees heavy use • CPU load affects virtual network performance

19. PL-VINI: Prototype on PlanetLab • Enable experiment: Internet In A Slice • XORP open-source routing protocol suite (NSDI ’05) • Click modular router (TOCS ’00, SOSP ’99) • Clarify issues that a VINI must address • Unmodified routing software on a virtual topology • Forwarding packets at line speed • Illusion of dedicated hardware • Injection of faults and other events

20. Synthesis of Software Components to enable PL-VINI XORP – Open Source Routing Protocol Suite especially developed for network research community. [9, NSDI ’05] Click – Software Architecture for Building Flexible and Configurable routers [10] UML – Linux-Kernel running as regular Linux-User-Process without Root-Privileges. OpenVPN – Open source SSL VPN solution.

21. XORP: Control Plane XORP (routing protocols) • BGP, OSPF, RIP, PIM-SM, IGMP/MLD • Goal: run real routing protocols on virtual network topologies

22. User-Mode Linux: Environment UML XORP (routing protocols) • Interface ≈ network • PlanetLab limitation: • Slice cannot create new interfaces • Run routing software in UML environment • Create virtual network interfaces in UML eth0 eth1 eth2 eth3

23. Click: Data Plane UML XORP (routing protocols) • Performance • Avoid UML overhead • Move to kernel, FPGA eth0 eth1 eth2 eth3 Control Data Packet Forward Engine Click

24. Click: Data Plane UML XORP (routing protocols) • Performance • Avoid UML overhead • Move to kernel, FPGA eth0 eth1 eth2 eth3 Control Data Packet Forward Engine UmlSwitch element Click

25. Click: Data Plane UML XORP (routing protocols) • Performance • Avoid UML overhead • Move to kernel, FPGA • Interfaces  tunnels • Click UDP tunnels correspond to UML network interfaces eth0 eth1 eth2 eth3 Control Data Packet Forward Engine UmlSwitch element Tunnel table Click

26. Injecting Link Failures UML XORP (routing protocols) • Performance • Avoid UML overhead • Move to kernel, FPGA • Interfaces  tunnels • Click UDP tunnels correspond to UML network interfaces • Filters • “Fail a link” by blocking packets at tunnel eth0 eth1 eth2 eth3 Control Data Packet Forward Engine UmlSwitch element Tunnel table Click Filters

27. Life of a packet in IIAS

28. Experiments • Microbenchmarks • Capacity • Behavior • Intra domain Route changes • Failure and recovery

29. Overlay Efficiency IIAS’s user-space Click forwader vs. in-kernel forwarding

30. Performance is bad • User-space Click: ~200Mb/s forwarding • Can do a lot with 200Mb/s • 20 experiments can have dedicated 10Mb/s nationwide networks • Improving performance is ongoing work • Allow experiments to load custom Click modules into the VINI kernel

31. Overlay on PlanetLab on dedicated hardware (DETER) vs. shared platform (PlanetLab) and PL-VINI

32. Overlay on PlanetLab PL-VINI and IIAS together provide a close approximation of the underlying network’s behaviors.

33. Intra-domain Route Changes s 2095 856 700 260 233 1295 c 639 548 366 846 587 902 1893 1176 Watch OSPF route convergence on Abilene

34. Intra-domain Route Changes s 2095 856 700 260 233 1295 c 639 548 366 846 587 902 1893 1176 Watch OSPF route convergence on Abilene

35. Intra-domain Route Changes s 2095 856 700 260 233 1295 c 639 548 366 846 587 902 1893 1176 Watch OSPF route convergence on Abilene

36. Intra-domain Route Changes s 2095 856 700 260 233 1295 c 548 366 846 587 902 1893 1176 Watch OSPF route convergence on Abilene

37. Intra-domain Route Changes s 2095 856 700 260 233 1295 c 639 548 366 846 587 902 1893 1176 Watch OSPF route convergence on Abilene

38. Link down Ping During Link Failure

39. Link down Link up Ping During Link Failure

40. Routes converging Link down Link up Ping During Link Failure

41. Link down TCP Throughput

42. Link down Link up TCP Throughput

43. Link down Link up Zoom in TCP Throughput

44. Slow start Arriving TCP Packets

45. 2.45 Packet received 2.4 2.35 Slow start 2.3 Megabytes in stream 2.25 2.2 Retransmit lost packet 2.15 2.1 17.5 18 18.5 19 19.5 20 Seconds Arriving TCP Packets

46. Slow start Retransmit lost packet Arriving TCP Packets PL-VINI enables a user-space virtual network to behave like a real network on PlanetLab

47. Conclusion • VINI: Virtual Network Infrastructure • Controlled and realistic experimentation • New routing, forwarding, and addressing • Initial experiment • Intradomain routing convergence in XORP • Later, perhaps run commercial router code?  • Long term goal • Shared platform for networking experiments • Real user traffic and upstream ISP connections