Structured P2P overlay networks

1. Structured P2P overlay networks Yann Busnel Maitre de conférence – Université de Nantes F2R401b - Architectures des systèmes dynamiquesTelecom Bretagne – Rennes Structured P2P overlay networks

2. Recall on unstructured P2P • Unstructured P2P architectures • Direct application of P2P paradigm • Each peer: • Is both client and server • Is autonomous in term of storage and resources • Has only a knowledge of its logical neighborhood • Problems : • Numerous limitations due to • The number of exchanges messages • The efficiency of localization process • Blind approach … Structured P2P overlay networks

3. Polemic • P2P paradigm is realistic ? • Nodes from a network are they all really equivalent? • Is it reasonnable to not indexing resources in order to protect autonomy? Structured P2P overlay networks

4. Constat 1 • All peers are not equivalents: • Bandwidth heterogeneity • A gap from 1 to 3 of the available bandwidth [Sar02] • Storage capabilities heterogeneity • Processor heterogeneity • Behavior heterogeneity • In file sharing system, 70% of users are free ridersi.e. users that don’t share any file [Ada00] • 50% of replies are produced from 1% of peers [Ada00] Finally, all peers are not equals! Motivation for HYBRIDES P2P ARCHITECTURES Structured P2P overlay networks

5. Constat 2 • Resources indexation remains (until now) the best way to guarantee resource localization Finally, in order to maintain a good localization service QoS, it is not reasonnable to avoid resource indexes! Motivation for STRUCTURED P2P ARCHITECTURES Structured P2P overlay networks

6. Outline • Introduction • Hybrid P2P • Structured P2P • DHT • Interface • Example system • CHORD • PASTRY • CAN Structured P2P overlay networks

7. Hierarchicalpeers • Remove the peer’sequalityconstraint • Aims: • Give to somepeers a specificrole • Representationrole • Routingrole • Removelowbandwidthnodesfrom the localizationprocess: ask for otherpeers to representthem • Strategies • Creation of peer’s data index on a subset of peers=> Super-peer Structured P2P overlay networks

8. Super-peer • Super-peer / Peer • Super-Peer : Nodeswith a large bandwidth • Peer : Nodewith a lowbandwidth are clusteredaround a super-peer • Role: • To substitute itself to simple peers in the localizationprocess • Indexation • Super-peers have an index of resourcesprovided by representedpeers. • Note that • Napster : 1 super-peer / n pairs • KaZaA : msuper-peers / n pairs Structured P2P overlay networks

9. i i i i i Hierarchical network • Super-Peers are organized in P2P fashion • All Peers are linked to a super-peer in client/serveur mode pair super-pair cluster C / S P2P Index des ressources du cluster Structured P2P overlay networks

10. Peer architecture Resource Request Super-Peer Peer User Interface Download service Download C/S Communication Request Super-peer interogation system Result Super-peer connection system Index Locales resources Index of stored data @ super peer Structured P2P overlay networks

11. Super peer architecture Request Resource Request Result Message management Super-Peer User Interface Download Request management Result transmitter Request handler Downloading service Resources locales Index of local data and represented peers’ data C/S Comm. C/S communication server Super-Peer Neighborhood table Result handler Request forwarder Results Results Peer Request Request Request Structured P2P overlay networks

12. Faulttolerance • Redondant Super-peer • Possible enhancement, choose ksuper-peers (partners) instead of one in a cluster • Each partner is connected to each client and owns an index for all their resources • Raise join cost of new node by a factor k • Clients send their requests to their partner according to a « round robin » principle • Permit to decrease the load of a partner by a factor k Structured P2P overlay networks

13. Example of redondant super-peer C/S P2P Structured P2P overlay networks

14. Remarks • R1 : increase the cluster size raiseindividualload (in term of bandwidth) of super-peers but reduce the global load • Large clusters => Hugeconsequence in term of failure, attacks, … • R2 : super-peer redondance is positive but costful • Good trade-offbetween global loadbalancingusing large cluster withlowindividualload and a good aulttolerance Structured P2P overlay networks

15. Summary • Avantages • Remove low resource peers (bandwidth, processing power, …) from the information localization process • Faster resource localization • Disadvantages • Creation of an index on super-peer likely to resolve queries • Centralization, even local, infers a lower super-peer failure tolerance • Costly super peer redondance Structured P2P overlay networks

16. Example of hierarchical P2P • Super-peer : • KaZaA • 4 500 000 simultaneous connections • Connexions duration: few hours • Number of available files: 106 • … • Gnutella (0.6-*) • Ultrapeers Structured P2P overlay networks

17. Outline • Introduction • Hybrid P2P • Structured P2P • DHT • Interface • Example system • CHORD • PASTRY • CAN Structured P2P overlay networks

18. Objectives • Give value to efficiency of localization process • Data indexation • Guarantee to find any available data in the network Required a decentralized index ! Structured P2P overlay networks

19. Structure networks • Properties • Data or data reference are placed on specificnodes • Allocation are made using a hash function • Searchingisproceedaccording to a key • Challenge • Findfastly an objectstartingfromitskey in a large scale network withoutcentralized structure • Chord, CAN, Pastry… Structured P2P overlay networks

20. 1-P2P routing infrastructure • Overlay: network abstraction on top of IP • Basic functionality: distributed hash table • CAN relies on a multidimensional cartesian space • Chord, Pastry, Tapestry: generalized hypercube routing based on prefix matching • Exact-match interface • Applications • Content-delivery networks • Storage systems, Caching • Naming services • Multicast Structured P2P overlay networks

21. Distributed Hash Table (DHT) containers k1,v1 k2,v2 k3,v3 Operations: insert(k,v) lookup(k) k4,v4 k5,v5 k6,v6 Table of containers • Store <key,value> pairs • Efficient access to a value given a key • Mapping key-value ensured by the table of containers Structured P2P overlay networks

22. P2P Distributed Hash Table nodes k1,v1 k2,v2 k3,v3 P2P overlay network Operations: send(m,k) k4,v4 k5,v5 k6,v6 • Message sent to keys: implementation of a DHT • P2P infrast. ensures mapping between keys and physical nodes • Fully decentralized: peer to peer communication paradigm Structured P2P overlay networks

23. DHT capabilities • Scalability: • Resources agregation • Storage • Computing power • Bandwidth • Load balancing • Availability • Self-organizing • Failure resilience Structured P2P overlay networks

24. P2P DHT principle 91 40 18 75 66 52 32 24 83 Structured P2P overlay networks 24

25. P2P DHT principle Asia Europe South America Australia 91 40 18 Overlay network 75 Asia 66 52 32 North America Europe Asia 24 83 North America logical address space Structured P2P overlay networks

26. nodeId key Mapping Identifier Space • An identifier is associated to each node (nodeId) • Large identifier space (keys and nodeId) • A node is responsible for closest key to its nodeId Structured P2P overlay networks

27. nodeId key Sending messages to keys destination source • Each node maintains a routing table (nodeid, IP adress) • At each routing hops, the message gets closest to the key • Infrastructure P2P: mapping between key and node Structured P2P overlay networks

28. K V K V K V K V K V K V K V K V K V K V K V Distributed Hash Table (K1,V1) insert(K1,V1) retrieve (K1) Structured P2P overlay networks

29. Routing Interface • Route (key,node,msg) • Send (node,msg) • NextHopSet = lookup (key, num) • Upcalls • Deliver(key,msg) • Forward (key,msg, NexHopNode) • Update (neighbour set) Structured P2P overlay networks

30. Routing Interface • Route (key, node, msg)Route the msg to the node (alive) responsible of the key. Non blocking, (best effort), the message may be lost or duplicated • nodeparameter is optional and is interpreted as the first hop (using a good heuristic might deliver the message in one single hop) • msg: application-specific • key: numerical key (160 bits) • node: encapsulates the @ of the last node known associated to that key Structured P2P overlay networks

31. Routing Interface • nextHopSet = lookup (key,num)Returns a set of nodes (num) from the routing table as potential next steps for the routing • key: numerical 160 bit key • num: number of nodes required in the set • Result: the resulting set of nodes Structured P2P overlay networks

32. Routing Interface • Send(node,msg)sends a message to a specific node; fails if the node is not available, blocking and reliable • Msg: msg is application-specific • Node: encapsulate the @ of the destination node • Result: true if the msg is delivered false otherwise Structured P2P overlay networks

33. Routing Interface • Deliver (msg,key)Upcall delivering the message to application. Assumes one application per node msg: message application-specific key: numerical 160 bit key Structured P2P overlay networks

34. Routing Interface • Forward (msg,key,nextHopNode)upcall invoked at each node on the message route (source and destination included) to inform an application that the msg and the key are about to be forwarded to NextHopNode. The application can modify msg, key, nextHopNode or terminate the message by setting nextHopNode to null • Msg: application specific msg • Key: numerical 160 bit key • nextHopNode: @ of the node to which the message is forwarded after the upcall has returned Structured P2P overlay networks

35. Routing Interface • Update(neighborSet)upcall invoked to inform the application of a change in the neighborhood of the local node. • Neigborset: set of k nodes Structured P2P overlay networks

36. Outline • Introduction • Hybrid P2P • Structured P2P • DHT • Interface • Example system • CHORD • PASTRY • CAN Structured P2P overlay networks

37. Pastry / Chord node Id space key NodeId = 128 bits Nodes and key place in a linear space (ring) Mapping : a key is associated to the node with the numerically closest nodeId to the key (different placement in Pastry and Chord) Structured P2P overlay networks

38. Chord • DHT-based on a ring that permit place 2m nodes • Node allocation • Hashing of node IP • Each node is allocated on the ring according to its IP hash m = 4  16 emplacements Structured P2P overlay networks

39. Resource allocation • Resource hash • H(resource) = key • The key is placed on the node with ID greater or equal than key value ( clé=0000 , contenu=F1 ) ( clé=0010 , contenu=F2 ) ( clé=1001 , contenu=F3 ) Structured P2P overlay networks ( clé=0110 , contenu=F4 )

40. Naiveresourcesearching • Each node store ID of its successor • To handle a query, a peer forward the request to its successor • The result follow the backward path • Linear searching according to the number of peers Structured P2P overlay networks

41. Example of naivesearching Recherche(0000) ( clé=0000 , contenu=F1 ) Structured P2P overlay networks

42. Required a routing table more complete:  Finger nodes table Use a successor mechanism not limited to the direct one on the ring Principle For each node Ni with hash value j : Succ[k]=first node on the ring that verify (j + 2k-1) mod 2m, 1≤ k ≤ m Use of predecessor for ring maintenance Researchenhancement Structured P2P overlay networks

43. « Fingernodes » table Structured P2P overlay networks

44. If the key exists locally Yes  return associated value Else Search in the routing table a node with a value lower or equal than the searched key Forward the query to the selected node Apply this recursively  Average number of hops: O(log2(N)) Searchingalgorithm Structured P2P overlay networks

45. Searching in Chord Storedkeys {00000,01111,10000,11111} Routing table (Fingernodes) Look for key 00001 from peer N2 N1 N6 0 15 1 N7 14 2 N8 13 3 N10 Storedkeys {00111,10111} Routing table (Fingernodes) 12 4 N4 5 N3 11 6 10 9 N5 7 8 N2 Structured P2P overlay networks 00001

46. Combining elementary operations: N.JOIN(N’) Node N tells to node N’ its arriving int the network and ask for a successor N.STABILIZE() Periodically launch, permit to N and its successor to check if they form a correct couple (no new node inserted between them) N.MAJENTRY() : Periodically launch, permit to initialize routing table of new nodes, or update existing ones N.TESTPREC() : Periodically launch, check if predecessor remains connected Node insertion Structured P2P overlay networks

47. 01010 , 110010 Insertion of node N11 at position 9 9 >9 9 9 >9 1) Contact successor N11 2) Key retrival (join) 01000 , 11000 , 11001 3) Predecessor retrival (stabilize) Structured P2P overlay networks 4) Routing table update

48. 01010 , 110010 Insertion of node N11 at position 9 N11 N11 N11 N11 9 N11 N11 , N11 1) Contact successor 01000 , 11000 , 11001 N11 insertion generates modification on 5 nodes 2) Key retrival (join) N5, N4, N10, N6 3) Predecessor retrival (stabilize) Structured P2P overlay networks 4) Routing table update

49. Guarantee that any sequence of join interlace with stabilize converges to a stable state (all nodes are reachable) Even in presence of (limited) joins, searching cost remains in O(log(N)) A search can fail if the network is not completelly stabilized (it’s possible to send again the query a little bit later) Insertion algorithmproperties Structured P2P overlay networks

50. The algorithm relies of successor notion (Failure on one node cut the network) A solution consists to manage a set of r successor (cf. Pastry) Failuretolerance Structured P2P overlay networks