1 / 21

Advances in Distributed Search Systems: A Rendezvous Approach by Christof Leng

This presentation by Christof Leng, from the Department of Computer Science at Technische Universität Darmstadt, delves into enhancing distributed search systems using a rendezvous approach. It discusses traditional methods like distributed key-value lookups and dynamic search functionalities. The benefits of a "black box" approach are highlighted, promoting flexibility, scalability, and robust query processing. Various applications, from P2P networking to sophisticated information retrieval, are examined, emphasizing the need for expertise in developing these systems.

euclid
Télécharger la présentation

Advances in Distributed Search Systems: A Rendezvous Approach by Christof Leng

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Search withRendezvous Search Systems Christof Leng Dipl.-Inform. Christof Leng Databases & Distributed Systems Department of Computer Science TechnischeUniversität Darmstadt http://www.dvs.tu-darmstadt.de cleng@dvs.tu-darmstadt.de

  2. Distributed Search: A Traditional View • Implement distributed key-value lookup • e.g., a distributed hash table (DHT) • scalable& simple • Build everything else on top of it • keyword search, XPath, range queries • map every operation to a number of lookups • Downside: • implement every query language yourself • query processing not portable between DHTs • unforeseen side-effects & interactions • Experts needed for every application • application domain • P2P networking • information retrieval … need sophisticated queries … Application Developer Information Retrieval Expert P2P Networking Expert Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  3. Rendezvous Search • “Every query will meet each data item somewhere in the network” • don’t worry how it gets there • black box approach • Implement the match functionality • as if it was local • re-use existing libraries • Advantages: • separation of concerns • flexible • scalable (O(√n) search cost) • robust (O(√n) replicas) … need sophisticated queries … glue code Application Developer Information Retrieval Library Rendezvous Search System Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  4. Example Application Rendezvous Middleware Lucene Keyword Search P2P Wiki Overlay Network Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  5. Rendezvous Search Genealogy Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  6. Data Center Solutions Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  7. Grid Pro: • simple • efficient • flexible Con: • not fault-tolerant • not size-adaptive • not load-adaptive • two-dimensional structure Reference: L. A. Barroso, J. Dean, and U. Hölzle. Web Search for a Planet: The Google Cluster Architecture. IEEE Micro, 23(2):22–28, 2003. Data D Data D D D Q Q Q R D Q Q Q Q Q Q R Q D Q Q Q Q Q Query Query Rendezvous Node Rendezvous Node D D D D Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  8. ROAR – Rendezvous on a Ring Pro: • simple mapping to 1 dim. • size-adaptive • load-adaptive • fault-tolerant Con: • centralized or • inefficient Reference: C. Raiciu, F. Huici, M. Handley, and D. S. Rosenblum. ROAR: Increasing the Flexibility and Performance of Distributed Search. In Proceedings of SIGCOMM’09, pages 291–302, 2009. ACM. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  9. Structured Peer-to-Peer Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  10. Bit Zipper Pro: • z-order mapping • DHT-based = decentralized • fast tree-based replication Con: • relies on DHT’s reliability • no replica maintenance Reference: W. W. Terpstra, S. Behnel, L. Fiege, J. Kangasharju, and A. Buchmann.Bit Zipper Rendezvous—Optimal Data Placement for General P2P Queries.In EDBT’04 Workshop on Peer-to-Peer Computing and Databases, 2004. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  11. Deetoo Pro: • 2 rings for projection • includes replica maintenance Con: • maintenance overhead for 2 rings • no responsibility ranges(probabilistic approach) • additional overhead Reference: T. W. Choi and P. O. Boykin. Deetoo: Scalable unstructured Search built on a Structured Overlay. In HOTP2P - International Workshop on Hot Topics in Peer-to-Peer Systems, Los Alamitos, CA, USA, 2010. IEEE Computer Society. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  12. Unstructured Peer-to-Peer Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  13. Ferreira et al. – Random Walks Pro: • very resilient topology • proven correctness Con: • probabilistic • additional overhead • slow & fragile random walks • no topology protocol Reference: R. A. Ferreira, M. K. Ramanathan, A. Awan, A. Grama, and S. Jagannathan. Search with Probabilistic Guarantees in Unstructured Peer-to-Peer Networks. In Proceedings of P2P’05, pages 165–172, Washington, DC, USA, 2005. IEEE Computer Society. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  14. BubbleStorm Pro: • resilient& fast • size-adaptive • load-adaptive • topology & replica maintenance included Con: • probabilistic • additional overhead Reference: W. W. Terpstra, J. Kangasharju, C. Leng, and A. P. Buchmann. BubbleStorm: Resilient, Probabilistic, and Exhaustive Peer-to-Peer Search. In Proceedings of SIGCOMM’07, pages 49–60, New York, NY, USA, 2007. ACM. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  15. Hautakorpi & Schultz Pro: • unstructured solution on top of DHT • incremental deployment possible Con: • probabilistic • additional overhead • fragile Reference: J. Hautakorpi and G. Schultz. A Feasibility Study of an Arbitrary Search in Structured Peer-to-Peer Networks. In Proceedings of 19th International Conference on Computer Communications and Networks (ICCCN), pages 1–8. IEEE, Aug. 2010. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  16. Semi-Structured Peer-to-Peer Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  17. SplitQuest Pro: • combines unstructured resilience with structured efficiency Con: • probabilistic • lacks analysis and prototype evaluation Reference: P. Lopes and R. A. Ferreira. SplitQuest: Controlled and Exhaustive Search in Peer-to-Peer Networks. In Proceedings of the 9th International Conference on Peer-to-Peer Systems, IPTPS’10, Berkeley, CA, USA, 2010. USENIX Association. Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  18. Comparison Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  19. Rendezvous Search & Cloud Computing • similar problems • similar requirements • less research history could benefit from • size adaptivity • traffic adaptivity • fault tolerance • algorithm diversity D D Replication Data D D Q Q Q M Q Q Q Q Q Q R Q D Q Q Q Q Q Query Sharding Rendezvous Node D Key-Value Store Scalability D Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  20. Conclusion • Rendezvous Search is a powerful abstraction • scalable • flexible • robust • A wide range of solutions is out there • data center • unstructured P2P • structured P2P • semi-structured P2P • Ever needed decentralized but powerful search in a project? • don’t try to reinvent the wheel! Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

  21. Thank you! ? Questions http://www.bubblestorm.net http://www.dvs.tu-darmstadt.de Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"

More Related