1 / 48

Decomposing Overlay Applications

Decomposing Overlay Applications. Achieving Extensibility with High Performance. Yitzchak Gottlieb Princeton University. Constant Innovation. 1725 RFCs since 1995 50 Obsolete old RFCs 307 Update ~1400 New ideas 897 Standards track 142 Experimental. Similar to existing More recipients

daire
Télécharger la présentation

Decomposing Overlay Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Decomposing Overlay Applications Achieving Extensibility with High Performance Yitzchak Gottlieb Princeton University

  2. Constant Innovation • 1725 RFCs since 1995 • 50 Obsolete old RFCs • 307 Update • ~1400 New ideas • 897 Standards track • 142 Experimental

  3. Similar to existing More recipients Broadcast Multicast Any one recipient Anycast Truly new Scalable routing Scalable object location Content-based addressing Fault tolerance Automatic management Types of Innovations Hard to change current networks to enable these services

  4. Overlay Networks • On end hosts • Application-level networks • Advantage • Ease deployment • Problems • Limited performance • User space • Proxy path

  5. Performance Limits: Proxy Path 1500 B packets 100 Mbit/s 8 000 pkt/s 10 Gbit/s 800 000 pkt/s

  6. PlanetLab “Planetlab is an open platform for developing, deploying, and accessing planetary-scale services”

  7. PlanetLab • Nodes are PCs • Centrally managed • Distributed world-wide • 366 Nodes • 152 Institutions • ~500 Projects • Overlays are user-level programs • Executing in vservers on Linux

  8. PlanetLab • PlanetLab is for overlays • Very successful overlays • e.g. Routing overlays • Traffic increases • Inefficiencies interfere How to achieve high performance with overlays?

  9. Inside Overlay Nodes Maintain network Forward packets Process queries … OS

  10. Computation Barrier Computation Domains • Where computation is • Separated by barriers • Address space • Protection level • Processor • Differences • Performance • Trust

  11. Hypothesis • We can decompose overlay applications • We can map components onto different computation domains • This will improve performance and efficiency

  12. Outline • Introduction • Decomposition • Software Architecture • Examples

  13. Decomposing Overlay Networks • Five forms • Forms are: • Semantically equivalent • All equally viable • Refinement steps • Different • Performance • Trust

  14. Lessons of Internet • High speed IP forwarding • Separate concerns • Routing protocol • Program forwarding table • Packet forwarding • Decrement TTL • Replace layer-2 address

  15. Control and Data • Control • Complex • Processed locally • Data • Simple • Processed remotely

  16. Decomposition—Form I • State of the art • Monolithic application • Few network connections • All communication to application • Application sees all traffic Application

  17. Two code modules Control Maintain network Offer service Overlay Forwarder Forward to other hosts Two modes in overlay Local delivery Forwarding Control Application C Forwarder Decomposition—Form II

  18. In separate domains User-space control Overlay Forwarder In kernel On line card Why trust forwarder? SimpleVerifiably safe 100–200 instructions No backward branches Flow isolation Decomposition—Form III Control C Forwarder

  19. Two modules Overlay Daemon Connectivity Routing Application File serving Query Processing Two control modes Maintain network Offer service Control Application C Daemon Forwarder Decomposition—Form IV

  20. F F Forwarder Decomposition—Form V Application • Application-specific forwarding extension • Recording • Redirecting • Modifying C Daemon • Applications need data stream

  21. Daemon Application Overlay Daemon Application F F Application Forwarder Overlay Forwarder Decomposing Overlay Applications General Application Control Data

  22. F F F F F F F F NIC High Performance Overlays • Decompose overlays • Place overlay forwarder in • Leave rest in user space Application Daemon kernel router line card user space User Kernel Router

  23. Forwarders and Extensions

  24. Outline • Introduction • Decomposition • Software Architecture • Examples

  25. C Plug Board

  26. Plug Board • Software architecture for decomposition • Features • Single powerful classifier • Multiple computation domains • Multiple forwarders • Fixed forwarders • Extensible forwarders

  27. Plug Board • Miscellaneous features • Modularity • Functions grouped by protocol • Encapsulation • Input functions • Output functions • Wrap forwarders

  28. Implementation • Point implementations • Three domains • User space Linux • Kernel space Linux • Leverage SILK • Intel IXP1200 EEB • Leverage Vera architecture

  29. Outline • Introduction • Decomposition • Application • Examples

  30. Decomposition Examples • Applications • Peer-to-peer networks • Other network • Each is decomposable • Four components

  31. Peer-to-Peer Networks • Peers are clients, servers, and routers • Examples • Pastry • P2P application toolkit • Chord • Scalable lookup service • Gnutella • Unstructured, file-sharing network

  32. Pastry • Peer-to-peer application toolkit • Nodes have 128 bit ID • Treated as number in base 2b • Forward to node with closer ID • Digit by digit • Logarithmic forwarding

  33. F Pastry • Overlay Daemon • Connectivity • Overlay Forwarder • Find next longest prefix match • Application • PAST, Scribe, etc. • Application Forwarder • Application dependent Daemon Application F

  34. Pastry Implementation • User-level library • Kernel-level forwarding • IXP Forwarding • ~50 instructions

  35. Chord • Scalable object location • 128 bit node addresses • Circular address space • Forward to farthest 2-ith of circle • Logarithmic forwarding • Iterative • Recursive 0 7 1 6 2 5 3 4

  36. 0 7 1 6 2 F 5 3 4 Chord • Overlay Daemon • Chord invariants • Overlay Forwarder • Recursive lookup • Application • Node location • Application Forwarder Daemon Application

  37. Gnutella • Overlay Daemon • Connectivity • Overlay Forwarder • Flood • Record route • Application • Application Forwarder • Redirects successful queries

  38. Other Network Services • IP Forwarding • Filters, Firewalls • Proxies • Splicing TCP connections • Ethernet bridging

  39. Summary • Overlays limit performance • Improve by applying classic techniques • Decompose overlay • Control/Data • General/Application-specific • Map onto computational domains

  40. Conclusions • Overlay applications can perform well • Separate applications and networks • Optimize networks • Offload applications • Some overlays are applications • Too complex for network • Limited performance

  41. Credits • SILK/Scout • Larry Peterson, Andy Bavier, Oliver Spatschek, Ian Murdoch, Xiahu Qie, Aki Nakao • Vera • Scott Karlin • IXP1200 • Scott Karlin, Tammo Spalink And many more…

  42. Thank you

  43. URLs • PlanetLab • http://www.planet-lab.org/ • Princeton CS Network Systems Group • http://www.cs.princeton.edu/nsg

  44. Interfaces • Overlay Daemon and Overlay Forwarder • Routing table interface (Add, Delete route) • Application and Application Forwarder • Application specific • Use bit pipe or shared memory • Overlay Daemon Application • Overlay specific

  45. Function Types • Predicates (P), Forwarders (F) • Extensions (E) • Extensible Forwarders (EF) • Extensible Extensions (EE) • Input (I) and Output (O) • Encapsulation (F, I, OF) • Installation (P,F())

  46. M TCP Connection Splicing • Overlay Daemon • Installs splicer • Overlay Forwarder • Splicer • Application • Application Forwarder • URL rewriter A B

  47. IP Forwarding • Overlay Daemon • Routing protocols (e.g. BGP) • Overlay Forwarder • IP forwarding • Application • Many examples • Application Forwarder

  48. B B B B B B 3 1 5 6 2 4 Ethernet Bridging • Overlay Daemon • Computes Minimum Spanning Tree • Overlay Forwarder • Multicast to open ports • Application • Control learning algorithm • Application Forwarder • Record source and destination

More Related