Customizing Virtual Networks with Partial FPGA Reconfiguration

1. Customizing Virtual Networks with Partial FPGA Reconfiguration Dong Yin, Deepak Unnikrishnan, Yong Liao Lixin Gao and Russell Tessier Funded by National Science Foundation Grant CNS-0831940 Electrical and Computer Engineering University of Massachusetts, Amherst USA

2. Outline • Network virtualization • FPGA as a network virtualization platform • Customizing virtual routers with Partial FPGA reconfiguration • Dynamic virtual router allocation • Results • Conclusions

3. Network Virtualization • Many logical networks share a physical network infrastructure • Reduces costs • Independent routing policies • Key Challenges • Isolation • Performance • Scalability • Flexibility • Usability

4. Network Virtualization Techniques • Software • Full/Container virtualization • ASIC/Network Processors • Supercharging PlanetLab • Juniper E series

5. FPGA as a Virtualization Platform • High performance • Flexible hardware through reconfiguration • Full reconfiguration • Reprogram entire chip • Chip shut down during reconfiguration • Partial reconfiguration • Reprogram ‘part’ of the chip • Non-reconfigured regions can still operate during reconfiguration

6. FPGA as a Virtualization Platform • Key Challenges • Resource isolation between virtual routers • Scalability - Limited on-chip logic • In this paper, • Use partial reconfiguration for better resource isolation between virtual routers • Combine software virtual routers with hardware routers to create a scalable system

7. System Overview OpenVZ VRouter OpenVZ VRouter OpenVZ VRouter NIC NIC Software bridge Linux Kernel driver PCI 1G Ethernet I/F SDRAM SRAM PHY HW VRouter SDRAM HW VRouter SRAM NetFPGA

8. CPU Transceiver MAC RXQ MAC TXQ CPU RXQ CPU TXQ Output Queues Input Arbiter Design Select PRR1 MAC RXQ MAC TXQ Fwd Logic CPU RXQ CPU TXQ Fwd Table MAC RXQ MAC TXQ PRR2 CPU RXQ CPU TXQ Fwd Logic MAC RXQ MAC TXQ VIP TYPE VID Fwd Table x.x.x.x HW 0 CPU RXQ CPU TXQ y.y.y.y SW 1 CONTROL OpenVZ Virtual Router PCI I/F PCI I/F Bridge Bridge OpenVZ Virtual Router Architecture Static FPGA Reconfigurable Region Linux

9. CPU Transceiver MAC RXQ MAC TXQ CPU RXQ CPU TXQ Output Queues Input Arbiter Design Select PRR1 MAC RXQ MAC TXQ Fwd Logic CPU RXQ CPU TXQ Fwd Table MAC RXQ MAC TXQ PRR2 CPU RXQ CPU TXQ Fwd Logic MAC RXQ MAC TXQ VIP TYPE VID Fwd Table x.x.x.x HW 0 CPU RXQ CPU TXQ y.y.y.y SW 1 CONTROL OpenVZ Virtual Router PCI I/F PCI I/F Bridge Bridge OpenVZ Virtual Router Architecture Static FPGA Reconfigurable Region Linux

10. Dynamic Virtual Router Management • Virtual router requirements change over time • Varying bandwidth/latency requirements • Different routing policies -> Different h/w resources • Solution - Exploit heterogeneity • High throughput routers -> Hardware • Low throughput routers -> Software • Dynamically migrate virtual routers between s/w and h/w

11. Dynamic Virtual Router Management • Allocation • Try allocation in software (if b/w permits) • Else try allocation in hardware (if b/w and resources permit) • Removal • S/W - Destroy OpenVZ container • H/W - Program blank bitstream to reconfigurable region • Upgrades • Greedily migrate virtual routers between FPGA and Software based on bandwidth requirement

12. Evaluation • 2 Partially reconfigurable virtual routers on Virtex II • Software virtual routers - 3Ghz AMD X2 2GB RAM • Metrics • Throughput • Reconfiguration time • Packet generation • NetFPGA packet generator Source NetFPGA Pktgen Virtual Router Sink NetFPGA Pktcap 1Gbps 1Gbps

13. Partial Bitstream Generation Virtual router 2 (Verilog) Virtual router 1 (Verilog) Xilinx Early Access Partial Reconfiguration Partial bitstream repository PlanAhead JTAG

14. Throughput of Partially Reconfigurable Virtual Router • Consistent line rate (1Gbps) across packet sizes

15. H/W Virtual Router A MAC Qs MAC Qs Output Qs Design Select H/W Virtual Router B Source Sink OpenVZ eth eth S/W Bridge S/W Bridge OpenVZ Traffic Isolation and Reconfiguration Time • Full Reconfiguration Approach Fully Reconfigure FPGA FPGA H/W Virtual router B’ Linux

16. Traffic Isolation and Reconfiguration time Full reconfiguration 12 seconds Partial reconfiguration 20x reduction in downtime

17. Throughput per virtual router • Partial reconfiguration benefits at higher reconfiguration frequencies

18. Dynamic Virtual Network Allocation • Evaluation Model • 1000 virtual networks • Bandwidth distribution from PlanetLab nodes. • Poisson arrivals and Poisson lifetimes • Mean arrival period = 2hrs • Mean lifetime = ~2.5 days (64 hrs) • Bandwidth of live network changes according to Uniform distribution between 0%-X% from initial allocation b/w • All networks can be either allocated in FPGA or Software virtual routers

19. Benefit of Virtual Network Migration • Upto 20% more upgrade requests serviced by dynamic assignment • Reallocation benefits over wide fluctuations in bandwidth

20. Resource Usage and Power Consumption • Power management via Clock gating • Shutdown virtual routers when not in use • Saves 10% overall power

21. Conclusion • A novel heterogeneous network virtualization platform • Enhanced resource isolation and reconfiguration time • With partial FPGA reconfiguration • Scalable • Combines fast FPGA virtual routers with software virtual routers • Dynamic network allocation and migration • Towards a green virtualization platform • Integrates power management techniques

22. Future Work • Usability • Higher level abstractions for creating and managing partial bitstreams • Explore non-JTAG interfaces for reconfiguration (PCI Express/ICAP)

23. Thank you Questions?