Peer-to-Peer Networks

1. Peer-to-Peer Networks

2. Overlay Network A logical network laid on top of the Internet Logical link AB Logical link BC B A C Internet

3. The Formal Model Let V be a set of nodes. The functions id : V  Z+ assigns a unique id to each node in V rs : V  {0, 1}* assigns a random bit string to each node in V (optional) A family of overlay networks ON : F  G, where F is the set of all triples λ= (V; id; rs) and G is the set of all directed graphs. A unique directed graph ON(λ)∈ Gwith each labeled set λ = (V; id; rs) of nodes. Each node contains one or more objects. One important objective is SEARCH: any node must be able to access any object as quickly as possible

4. Structured vs. Unstructured Overlay networks Unstructured Structured Network topology satisfies specific invariants. Examples: Chord, CAN, Pastry Skip Graph etc No restriction on network topology. Examples: Gnutella, Kazaa, Bittorrent, Skype etc.

5. Gnutella The Gnutella network is a fully distributed alternative to the centralized Napster. Initial popularity of the network was spurred on by Napster's legal demise in early 2001.

6. What is Gnutella? A protocol for distributed search object1 object2 peer peer No central authority.

7. Remarks Gnutella uses the simple idea of searching by flooding, but scalability is an issue, since query flooding wastes bandwidth. Uses TTL to control flooding. Sometimes, existing objects may not be located due to limited TTL. Subsequently, various improved search strategies have been proposed.

8. Searching in Gnutella The topology is dynamic, i.e. constantly changing. How do we model a constantly changing topology? Usually, we begin with a static topology, and later account for the effect of churn. Modeling topology -- Random graph -- Power law graph (measurements provide useful inputs)

9. Random graph: Erdös-Rényi model A random graph G(n, p) is constructed by starting with a set of n vertices, and adding edges between pairs of nodes at random. Every possible edge occurs independently with probability p. Q. Is Gnutella topology a random graph?

10. Gnutella topology Gnutella topology is almost a power-law graph. (Also called scale-freegraph) What is a power-law graph? The number of nodes with degree k = c.k - r (Contrast this with Gaussian distribution where the number of nodes with degree k = c. 2 - k. ) Many graphs in the nature exhibit power-law characteristics. Examples, world-wide web (the number of pages that have k in-links is proportional to k- 2), Thefraction of scientific papers that receive k citations is k-3 etc.

11. AT&T Call Graph How many telephone numbers receive calls from k different telephone numbers? # of telephone numbers from which calls were made # of telephone numbers called 4

12. data 2 10 power-law fit t = 2.07 1 10 proportion of nodes 0 10 0 1 10 10 number of neighbors Gnutella network power-law link distribution summer 2000, data provided by Clip2 5

13. A possible explanation Nodes join at different times. The more connections a node has, the more likely it is to acquire new connections (“Rich gets richer”). Popular webpages attract new pointers. It has been mathematically shown that such a growth process produces power-law network 7

14. Search strategies • Flooding • Random walk / • - Biased random walk/ • - Multiple walker random walk • (Combined with) • One-hop replication / • Two-hop replication • k-hop replication

15. On Random walk Let p(d) be the probability that a random walk on a d-D lattice returns to the origin. In 1921, Pólya proved that, (1) p(1)=p(2)=1, but (2) p(d)<1 for d>2 There are similar results on two walkers meeting each other via random walk

16. Search via random walk Existence of a path does not necessarily mean that such a path can be discovered

17. Search via Random Walk Search metrics Delay = discovery time in hops Overhead = total distance covered by the walker Both should be as small as possible. For a single random walker, these are equal. K random walkers is a compromise. For search by flooding, if delay = hthen overhead = d + d2 + … + dhwhere d = degree of a node.

18. A simple analysis of random walk Letp = Population of the object. i.e. the fraction of nodes hosting the object T = TTL (time to live)

19. A simple analysis of random walk Expected hop count E(h) = 1.p + 2.(1-p).p + 3(1-p)2.p + …+ T.(1-p)T-1.p = 1/p. (1-(1-p)T) - T(1-p)T With a large TTL, E(h) = 1/p With a small TTL, there is a risk that search will time out before an existing object is located.

20. K random walkers Assume they all k walkers start in unison. Probability that none could find the object after one hop = (1-p)k. The probability. that none succeeded after T hops = (1-p)kT. So the probability that at least one walker succeeded is 1-(1-p)kT.A typical assumption is that the search is abandoned as soon as at least one walker succeeds. Using these, one can derive a new value of E(h) As k increases, the overhead increases, but the delay decreases. There is a tradeoff.

21. Increasing search efficiency Major strategies Biased walk utilizing node degree heterogeneity. Utilizing structural properties like random graph, power-law graphs, or small-world properties Topology adaptation for faster search Introducing two layers in the graph structure using supernodes

22. One hop replication Each node keeps track of the indices of the files belonging to its immediate neighbors. As a result, high capacity / high degree nodes can provide much better clues to a large number of search queries. Where is

23. Biased random walk P=5/10 P=2/10 P=3/10 Each node records the degree of the neighboring nodes. Search easily gravitates towards high degree nodes that hold more clues.

24. 67 63 54 1 power-law graph number of nodes found 94 6 2 Deterministic biased walk 9

25. The next step This growing surge in popularity revealed the limits of the initial protocol's scalability. In early 2001, variations on the protocol improved the scalability. Instead of treating every node as client and server, some resource-rich nodes were used as ultrapeers or “supernodes,” containing indices of the objects in the local neighborhood. Search requests and responses were routed through them leading to faster response.

26. The KaZaA approach Where is ABC? download ABC Supernode Supernode Powerful nodes (supernodes) act as local index servers, and client queries are propagated to other supernodes. Two-layered architecture.

27. The Chord P2P Network Some slides have been borrowed from the original presentation by the authors

28. Main features of Chord -- Load balancing via Consistent Hashing • Small routing tables per node: log n • Small routing delay: log n hops • Fast join/leave protocol (polylog time)

29. Consistent Hashing -- Assigns both nodes and objects an m-bit key. -- Order these nodes around an identifier circle (what does a circle mean here?) according to the order of their keys (0 .. 2m-1). This ring is known as the Chord Ring. An object with key k is assigned to the first node whose key is ≥ k (called the successor node of key k)

30. Nodes and Objects on the Chord Ring Key 5 K5 Node 105 N105 K20 Circular 7-bit ID space N32 N90 K80 A keyk is stored at its successor (node with key ≥ k)

31. Consistent Hashing [Karger 97] Property 1 If there are N nodes and K object keys, then with high probability, each node is responsible for (1+ )K/N objects. Property 2 When a node joins or leaves the network, the responsibility of at most O(K/N) keys changes hand (only to or from the node that is joining or leaving. When K is large, the impact is quite small.

32. The log N Fingers (0) 1/4 1/2 Distance of N80’s neighbors from N80 Circular (log N)-bit ID space 1/8 1/16 1/32 1/64 1/128 N80 Each node knows of only log N other nodes.

33. Finger i points to successor of n+2i N120 112 ½ ¼ 1/8 1/16 1/32 1/64 1/128 N80

34. Chord Finger Table N32’s Finger Table (0) N113 33..33 N40 34..35 N40 36..39 N40 40..47 N40 48..63 N52 64..95 N70 96..31 N102 N102 N=128 N32 N85 N40 N80 N79 N52 Finger table actually contains ID and IP address N70 N60 Node n’s i-th entry: first node  n + 2i-1

35. N70’s Finger Table Lookup 71..71 N79 72..73 N79 74..77 N79 78..85 N80 86..101 N102 102..5 N102 6..69 N32 Greedy routing (0) N113 N32’s Finger Table N102 33..33 N40 34..35 N40 36..39 N40 40..47 N40 48..63 N52 64..95 N70 96..31 N102 N80’s Finger Table N32 N85 81..81 N85 82..83 N85 84..87 N85 88..95 N102 96..111 N102 112..15 N113 16..79 N32 N40 N80 N52 N79 N70 N60 Node 32, lookup(82): 32  70  80  85.

36. New Node Join N20’s Finger Table (0) N113 N20 1 21..21 2 22..23 3 24..27 4 28..35 5 36..51 6 52..83 7 84..19 N102 N32 N40 N80 N52 N70 N60 Assume that the new node N20 knows one of the existing nodes.

37. New Node Join (2) N20’s Finger Table (0) N113 N20 21..21 N32 22..23 N32 24..27 N32 28..35 N32 36..51 N40 52..83 N52 84..19 N102 N102 N32 N40 N80 N52 N70 N60 Node 20 asks that node to locate the successors of 21, 22, …, 52, 84.

38. The Join procedure The new node id asks a gateway node n to find the successor of id n. find_successor(id) if id  (n, successor] then return successor else forward the query around the circle fi Needs O(n) messages for a simple Chord ring. This is slow.

39. Steps in join Linked list insert n n id id Successor(n) Finally But the transition does not happen immediately

40. A More Efficient Join // ask n to find the successor of id if id  (n, successor] then return successor else n’= closest_ preceding_node (id) return n’.find_successor(id) fi // search for the highest predecessor of id n. closest_preceding_node(id) for i = log N downto 1 if (finger[i]  (n,id) return finger[i]

41. Example N20 wants to find out the successor of key 65 (0) N113 N20 N102 N32 N40 N80 N52 N70 N60 K65

42. After join move objects (0) N20’s Finger Table N113 N20 21..21 N32 22..23 N32 24..27 N32 28..35 N32 36..51 N40 52..83 N52 84..19 N102 N102 D114..20 N32 N40 N80 N52 Notify nodes that must include N20 in their table. N113[1]=N20, not N32. N70 N60 Node 20 moves documents from node 32.

43. Three steps in join Step 1. Initialize predecessor and fingers of the new node. Step 2. Update the predecessor and the fingers of the existing nodes. (Thus notify nodes that must include N20 in their table. N113[1] = N20, not N32. Step 3. Transfer objects to the new node as appropriate. (Knowledge of predecessor is useful in stabilization)

44. Concurrent Join n1 n1 New node n’ New node n’ New node n New node n n2 n2 [Before] [After]

45. Stabilization Periodic stabilization is needed to integrate the new node into the network and restore the invariant. n1 n1 New node n New node n n2 n2 Predecessor.successor(n1) ≠ n1, so n1 adopts predecessor.successor(n1) = n as its new successor

46. The complexity of join With high probability, any node joining or leaving an N-node Chord network will use O(log2N) messages to re-establish the Chord routing invariants and finger tables.

47. Chord Summary • Log(n) lookup messages and table space. • Well-defined location for each ID. • Natural load balance due to consistent hashing. • No name structure imposed. • Minimal join/leave disruption.

48. Chord Advanced issues

49. Analysis Theorem. Search takes O (log N) time 2m = key space, N= number of nodes Proof. After log N forwarding steps, distance to key is at most (N= 2log N). Number of nodes in the remaining range is O(log N) with high probability (property of consistent hashing). So by using successors in that range, it will take at most an Additional O(log N) forwarding steps.

50. Analysis (contd.) O(log N) search time is true if finger and successor entries correct, But what if these entries are wrong (which is possible during join or leave operations, or process crash?)