1 / 20

presented by Anwitaman Datta

NCCR - MICS [IP5]. Route maintenance overheads in DHT overlays. presented by Anwitaman Datta Joint work with Karl Aberer and Manfred Hauswirth {Karl.Aberer, Anwitaman.Datta, Manfred.Hauswirth}@epfl.ch. EPFL-I&C-LSIR [P-Grid.org]. Workshop on Distributed Data and Structures ’04. P2P.

delora
Télécharger la présentation

presented by Anwitaman Datta

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. NCCR-MICS [IP5] Route maintenanceoverheads in DHT overlays presented byAnwitaman Datta Joint work with Karl Aberer and Manfred Hauswirth{Karl.Aberer, Anwitaman.Datta, Manfred.Hauswirth}@epfl.ch EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures’04

  2. P2P • Central index • Unstructured • Flooding • Super-peer • Power-law networks … ContentXYZ Need XYZ Network P2P is more than just File sharing or Pirate to Pirate!

  3. A Case for P2P • Goals: • Efficient, scalable and reliable resource discovery/name resolution in a decentralized internet scale system • Content addressable network - disentangle the underlying network from the applications and services • A reliable substrate for other distributed applications • Semantic Web, Grid computing, Web services, large scale event notification, P2P web-search, …

  4. Challenges • Unreliability of autonomous participants (peers) • Unreliability of the communication network • Lack of global knowledge and coordination • Scalability and robustness (fault tolerance) • Performance

  5. Approach (& issues) • Distributed indexing and routing in an overlay (disentangled from the underlying network) • Just like the world wide web, we can realize an overlay (but more structured) on top of the internet infrastructure e.g., Distributed Hash Table/Distributed Indexing • P-Grid, Chord, DKS, CAN, Pastry, Kademlia, … • Peers (re-)joining/leaving the overlay => Maintenance of the overlay routes is required • Self-healing while routing • Flux in overlay => system operates in dynamic equlibrium • Two forces: • Changes in the network making routing information unusable • Maintenance operations repairing routing information

  6. Prefix Routing in P-Grid 0 1 00 01 10 2 0 : 1,1410 : 11,13 12,13,14 14 1 : 2,1200 : 9,4011: 3,10 4,5 7 1 : 12, 1301 : 5,14001: 9,4 0,1 1 : 8, 1300 : 7,9011: 3,10 5 4,5 10 1 : 6,800 : 1,7010: 5,14 6,7 6 0 : 1,311 : 2,12101: 8,13 8,9 1 : 6,1301 :10,14000: 1,7 4 2,3 peer identifier ID Peer 9 holds keys with prefix 001, so we call, its path is 001 1 : 12, 13 routing table entry query(101) @ 7 11 000 001 010 011 100 101 Replicas 12 0 : 5,710 : 6,13 12,13,14 3 1 : 11,1200 : 1,9010: 5,14 6,7 11 0 : 4,711 : 2,12101: 8,13 8,9 0 : 5,911 : 2,12100: 6,11 13 10,11 1 1 : 12, 1301 : 5, 10001: 9,4 0,1 9 1 : 8,201 : 3, 10000: 1,7 2,3 8 10,11 0 : 4,911 : 2,12100: 6,11

  7. P-Grid Self-referential directoryimplemented by P-Grid lookup IP address in case of failure routing based on logical address routing based on logical address lookup IP address directory (logical ID <-> IP address) Self-healing, self-referential directory • Any distributed access structure (such as P-Grid) requires mapping of a logical ID (associated data key) to physical ID (IP address) • This mapping cannot by static in the presence of dynamic IP addresses (hence requires a directory service) • A very important problem for the implementation of any P2P system

  8. Example Presently online ID ID Presently offnline Routingentriesrepaired Up-to-date cache 1 :2 ,12 Stale cache 2 0 : 1,1410 : 11,13 12,13,14 Key as 4 bits for ID (2=0010 etc.) 14 1 : 2,1200 : 9,4011: 3,10 4,5 1 : 12, 1301 : 5,14001: 9,4 7 1 5 1 : 8, 1300 : 7,9011: 3,10 4,5 1 : 6,800 : 1,7010: 5,14 10 6,7 0 : 1,311 : 2,12101: 8,13 6 8,9 1 : 6,1301 :10,14000: 1,7 4 2,3 offline query(01*) @ 7 …query(0101) @ 7 (for stale entry 5, cycle -> abort) …query(1110) @ 7 (for stale entry 14, forward to 12 or 13) …query(1110) @ 12 (is offline) …query(1110) @ 13 (for stale entry 2) ……query(0010) @ 13 (forward to 5) ……query(0010) @ 5 (forward to 7) ……query(0010) @ 7 (forward to 9) ……query(0010) @ 9 (new entry for 2 found !) …query(1110) @ 2 (new entry for 14 found !) query(01*) @ 14 (finally ) 0 1 Stores mappings of peers ID 00 01 10 11 000 001 010 011 100 101 12 0 : 5,710 :6,13 12,13,14 3 1 : 11,1200 : 1,9010: 5,14 6,7 11 0 : 4,711 : 2,12101: 8,13 8,9 13 0 : 5,911 :2,12100: 6,11 10,11 1 : 12, 1301 : 5, 10001: 9,4 1 1 9 1 : 8,201 : 3, 10000: 1,7 2,3 8 10,11 0 : 4,911 : 2,12100: 6,11

  9. Possible strategies • Eager - Correction on Use (CoU) • While using a routing table, try correcting stale entries even if the present query can be routed using alternate routes (available locally). • Some entries of a particular level of routing table are unusable, but other entries of the same level are still usable. • Lazy - Correction on Failure (CoF) • While using a routing table, try correcting stale entries only if no alternate routes for the present query is available locally. • All entries of a particular level of routing table are unusable, but other levels may still be usable.

  10. Performance of overlays in flux • Static resilience • Given a state of the network, and no more changes, how does the network perform? • P-Grid, Chord, various topologies … • Dealing with network churn • Given flux in the network, what maintenance cost is required to maintain a certain state. • e.g., Lower bound (MIT/Chord) • Simulations … (many groups) • Dynamic equilibrium • Given any flux in the network, and any maintenance strategy, what equilibrium state will the network operate in, and what will the maintenance cost and performance in the equilibrium state be?

  11. Eager recursion a.k.a. Correction on Use ‘CoU’ Dynamic equilibrium equation • LHS • Rate at which repair of stale routing entries occur • rup changes per 1-rup queries • Nrec – 1 additional recursive queries • Repair makes sense only if the routing entry to be repaired corresponds to an online peer • A repair is possible only if recursive query succeeds • RHS • Rate of entries turning stale • rup changes • 1-pdyn probability of non-stale references (only these can turn stale) • r references at each peer for each of log2n levels

  12. Try to rectify stale references only when none of the references in a given level are usable Not all routing entries are treated uniformly (unlike in CoU). The number of stale entries for each routing level at each peer defines the state of that level. Markovian model. Dynamic equilibrium equation determined by equating inflow and outflow for each state At dynamic equilibrium, the number of routing levels with given number of stale entries over the whole system should not change Lazy Repair Strategy (Correction on Failure ‘CoF’ ) ID change ID change ID change ID change 0 ref stale 1 ref stale 2 ref stale r ref stale … repairs N.B. We distinguish stale entries from offline peers

  13. Analysis vs. Simulations (Lazy recursion)

  14. Overhead with varying pon

  15. Contour: Zone of operation for a maximum cost (Lazy)

  16. CoU (eager) vs. CoF (lazy)

  17. Reactivestrategies Our approaches Taxonomy of route maintenance mechanisms

  18. Summary • Self-referential decentralized directory with self-healing routing • Dynamic equilibrium of overlay network in flux (model & analysis) • Route maintenance mechanisms • Correction on Use • Correction on Failure • Taxonomy of maintenance mechanisms

  19. Other/open issues • Security/DDoS/… • Identity/Authentication • Authorization/Privacy • Reputation/Trust • Quorums/Web-of-trust • Garbage collection of references • Generic analysis (for various DHTs) • Sensor networks or MANETs and overlays

  20. References • Efficient, self-contained handling of identity in Peer-to-Peer systems, Karl Aberer, Anwitaman Datta, Manfred Hauswirth; IEEE Transactions on Knowledge and Data Engineering16(7), July 2004 & other papers @ http://www.p-grid.org Questions?

More Related