NDN Repository Storage and Query Algorithm for Efficient Data Retrieval

1. Index-based NDN Repository Junxiao Shi, 2014-06-17

2. Overall Structure • Storage: read-write storage for id=>Data packet • Index: sorted associative container for Name=>entry • Query algorithm

3. Storage API • typedef<implementation-defined> Id; • Id put(const Data& data); • Data get(Id id); • forward iterator for (Id, Data) – minimalorforward iterator for (Id, fullName, KeyLocatorHash) – preferred • fullName: full Name including implicit digest • KeyLocatorHash: SHA256 hash of KeyLocator

4. Storage in SQL Database: Table Structure • Indexes • PRIMARY KEY id

5. Index API • sorted container • entry type: (Id, fullName, KeyLocatorHash) • order by fullName • bidirectional iterator • lower_bound algorithm

6. Query Algorithm – Helper Routines • Query type • interest • KeyLocatorHash: SHA256 hash of PublisherPublicKeyLocator selector, or null • minComponents • maxComponents • boolmatchesSimpleSelectors(Query, Index::iterator) • determines whether the Data described by the index entry can satisfy MinSuffixComponent, MaxSuffixComponent, PublisherPublicKeyLocator, Exclude

7. Query Algorithm – leftmost for (it = index.lower_bound(interest.getName()); it != index.end(); ++it) { if (!interest.getName().isPrefixOf(it->fullName)) return index.end(); if (matchesSimpleSelector(interest, it)) return it; }

8. Query Algorithm – rightmost boundary = index.lower_bound(interest.getName()); if (boundary == index.end() || !interest.getName().isPrefixOf(boundary->fullName)) return index.end(); last = index.lower_bound(interest.getName().succ()); while (true) { prev = last; --prev; if (prev == boundary) return matchesSimpleSelectors(interest, prev) ? prev : index.end(); first = index.lower_bound(prev->fullName.getPrefix(interest.getName().size() + 1); match = std::find_if(first, last, matchesSimpleSelectors); if (match != last) return match; last = first; }

9. Query Example – rightmost, non-exact • Dataset: /A/B/d001, /A/C/D/E/d002, /A/C/F/d003, /A/C/G/d004, /A/C/H/I/d005 • Interest /A rightmost MaxSuffixComponents=3 • boundary=/A/B/d001, last=end (one past last entry) • iteration 1: first=/A/C/H/I/d005, last=end, no match • iteration 2: first=/A/C/D/E/d002, last=/A/C/H/I/d005, match=/A/C/F/d003

10. Query Example – rightmost, exact • Dataset: /A/d000, /A/d000/d001, /A/d002 • Interest /A/d000 rightmost MaxSuffixComponents=0 • boundary=/A/d000, last=/A/d002 • iteration 1: first=/A/d000/d001, last=/A/d002, no match • iteration 2: prev=/A/d000, equals boundary, matchesSimpleSelectorsis true, returns /A/d000

11. Query Algorithm – rightmost, alternate • start with index.lower_bound(interest.getName().succ()) • iterate to the left, until a match is found • continue to the left, until fullName.at(interest.getName().size()) changes; return the last match • This algorithm is less efficient when • The dataset is a versioned file, where each version has many segments • The Interest wants to fetch the first segment of the last version • This algorithm must iterate over every segment of the last version • This algorithm is more efficient for finding the last segment in a sequence, because it calls index.lower_bound() only once