Distributed Object Location in a Dynamic Network

1. Distributed Object Location in a Dynamic Network Kirsten Hildrum, John D. Kubiatowicz, Satish Rao and Ben Y. Zhao

2. Object Object 1 DOLR(Decentralized Object Location and Routing)

3. Requirements • Deterministic location • Routing locality – routes should have low stretch • Minimal storage and computational load • Dynamic membership

4. Main Results

5. Additional Results • Handle deletes • Simultaneous insertions • Algorithm with polylog stretch for general metrics. • Similar to Bourgain’s use of subsets to embed a general metric into L2

6. Outline • Background and previous work • PRR/Tapestry • Insertion • Maintaining deterministic location • Low stretch: A nearest neighbor algorithm • Related Work • Conclusion

7. Neighbor Map For “2175” (Octal) NodeID 2158 NodeID 2163 2170 210x 20xx 0xxx 2171 211x 2175 1xxx 3 2172 212x 22xx 2175 3 2173 213x 23xx 3xxx Ø 214x 24xx 4xxx 3 2175 215x 25xx 5xxx NodeID 2543 NodeID 2175 2 Ø 216x 26xx 6xxx 2177 2175 27xx 7xxx 4 2 4 3 2 1 2 Routing Levels NodeID 2170 NodeID 2421 Neighbor Table 1 NodeID 2245

8. 3 4 2 NodeID 2143 NodeID 2172 NodeID 2234 NodeID 0xEF34 NodeID 2175 1 4 NodeID 2173 NodeID 2163 3 2 1 3 NodeID 0775 4 4 3 2 NodeID 2732 NodeID 2161 3 NodeID 2170 4 NodeID 2056 NodeID 2105 2 NodeID 0777 2 3 2 1 2 3 NodeID 0x2345 NodeID 3456 NodeID 0732 NodeID E324 1 Basic Tapestry Mesh(from PRR97)

9. Use of Mesh for Object Location

10. Neighbor Map For “2175” (Octal) 2170 210x 20xx 0xxx 2171 211x 2175 1xxx 2172 212x 22xx 2175 2173 Ø 23xx 3xxx Ø 214x 24xx 4xxx 2175 215x 25xx 5xxx Ø 216x 26xx 6xxx 2177 2175 27xx 7xxx 4 3 2 1 Routing Levels Surrogate Routing • Neighbor table will have holes • Must be able to find unique root for every object • Our solution: try next highest. • For 2176, try 2177 • For 213? Try 214? • Need: all Ø really are Ø -> root unique.

11. Outline Again

12. Neighbor Map For “2175” (Octal) 2170 210x 20xx 0xxx 2171 211x 2175 1xxx 2172 212x 22xx 2175 2173 213x 23xx 3xxx Ø 214x 24xx 4xxx 2175 215x 25xx 5xxx Ø 216x 26xx 6xxx 2177 2175 27xx 7xxx 4 3 2 1 Routing Levels Insertion • Build own neighbor table • Want: Closest for each entry (note: cannot copy from nearby node) • Has to add self to other tables • Fill in holes • May replace further node 1

13. Finding the Closest Node • Building table means finding nearest neighbor for each entry. • Our algorithm: find nearest level-i node for every i in process of finding overall nearest • Easily extend to find other entries

14. Network Assumption • Nearest neighbor is hard in general metric • Assume the following: • Ball of radius 2r contains only a factor of c more nodes than ball of radius r. • Also, b > c2 • [Both assumed by PRR] • Start knowing one node; allow distance queries

15. Algorithm Idea • Call a node a level i node if it matches the new node in i digits. • The whole network is contained in forest of trees rooted at highest possible imax. • Let list[imax] contain the root of all trees. Then, starting at imax, while i > 1 • list[i-1] = getChildren(list[i]) • Certainly, list[i] contains level i neighbors.

16. 3 4 2 NodeID 0xEF97 NodeID 0xEF32 NodeID 0xE399 NodeID 0xEF34 1 4 NodeID 0xEF37 NodeID 0xEF44 3 2 1 NodeID 0x099F 4 3 NodeID 0xE530 NodeID 0xEF40 3 NodeID 0xEF31 NodeID 0xE555 NodeID 0xEFBA NodeID 0x0999 2 2 1 2 NodeID 0xE932 NodeID 0xFF37 NodeID 0x0921 NodeID 0xE324 1 We Reach The Whole Network NodeID 0xEF34

17. The Real Algorithm • Simplified version ALL nodes in the network. • But far away nodes are not likely to have close descendents •  Trim the list at each step. • New version: while i > 1 • List[i-1] = getChildren(list[i]) • Trim(list[i-1])

18. <2r r How to Trim • Consider circle of radius r with at least one level i node. • Level-(i-1) node in little circle must must point to a level-i node in the big circle • Want: list[i] had radius three times list[i-1] and list[i –1] contains one level i

19. new Animation

20. True in Expectation • Want: list[i] had radius three times list[i-1] and list[i –1] contains one level i • Suppose list[i-1] has k elements and radius r • Expect ball of radius 4r to contain kc2/b • Ball of radius 3r contains less than k nodes, so keeping k all along is enough. • To work with high probability, k = O(log n)

21. Steps of Insertion • Find node with closest matching ID (surrogate) and get preliminary neighbor table • If surrogate’s table is hole-free, so is this one. • Find all nodes that need to put new node in routing table via multicast • Optimize neighbor table • w.h.p. contacted nodes in building table only ones that need to update their own tables • Need: No fillable holes. Keep objects reachable

22. Need-to-know nodes • Need-to-know = a node with a hole in neighbor table filled by new node • If 1234 is new node, and no 123s existed, must notify 12?? Nodes • Acknowledged multicast to all matching nodes

23. Acknowledged Multicast Algorithm Locates & Contacts all nodes with a given prefix • Create a tree based on IDs as we go • Nodes send acks when all children reached • Starting node knows when all nodes reached The node then sends to any 5430?, any 5431?, any 5434?, etc. if possible 543?? 5431? 5434? 54340 54345

24. Outline Again

25. Related Work • Many peer-to-peer systems • Not locality-aware • Chord Stoica, Morris, Karger, Kaashoek, Balakrishnan • CAN Ratnasamy, Francis, Handley, Karp, Shenker • Pastry Rowstron and Druschel • Polylog stretch, general metric • Awerbuch and Peleg 1991 (not dynamic) • Rajaraman, Richa, Vöcking, Vuppuluri 2001 • Constant stretch, special metric • Plaxton, Rajaraman, Richa 1997 (not dynamic) • Nearest Neighbor • Karger and Ruhl’s 2002 • Thorup and Zwick 2001 • Clarkson 1997

26. Future Work • Getting stretch is low in practical systems • Transit stub networks • Network structure inside stub • Periodic optimization • Better handling of inserts in presence of deletes

27. Extra/Backup Slides

28. Related Work

29. Maintaining Object Availability • Object requests continue • New node: resends requests for object it does not have via surrogate • Established node: always checks that that it is final destination. • Sometimes-checking would require state. • May get cycle for non-existent objects, but can be detected.

30. Body • Finding the nearest neighbor • Can query for distance between self and any given node. • Assume a low expantion property (low dimentional) • Karger and Ruhl, May 2002. • Our algorithm not as good for general problem, but solves our particular problem better • No additional storage needed • Finds all levels of table

31. new Picture

32. Introduction • Object Location/DOLR slide • Related Work table • Not much done with stretch • Our results • Add insertion to PRR/Tapestry • Deletes • Parallel inserts • Solve nearest neighbor problem inprocess • Similar to Karger and Ruhl • Result for general metric spaces