1 / 13

FAWN: A Fast Array of Wimpy Nodes

FAWN (Fast Array of Wimpy Nodes) presents innovative techniques for improving distributed data environments. This presentation highlights methods such as consistent hashing that allow systems to tolerate node failures without data loss. Key components include the architecture for data logs that optimize memory usage and performance with append-only writes. The approach integrates virtual nodes, which distribute workloads effectively, mitigating overload issues during failures. Learn how FAWN leverages minimal memory for robust data storage and efficient data management through split, merge, and compact strategies.

manon
Télécharger la présentation

FAWN: A Fast Array of Wimpy Nodes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FAWN:A Fast Array of Wimpy Nodes D. G. Andersen1 J. Franklin1 M. Kaminsky2A. Phanishayee1 L. Tan1 V. Vasudevan1 1CMU 2Intel Labs

  2. Warning • This is not a complete presentation: it just explains some items that were left out of the authors' presentation of FAWN • Topics such as • Consistent hashing • Data Log architecture

  3. Consistent hashing (I) • Technique used in distributed hashing schemes to tolerate the loss of one or more machines • Traditional approach: • Each node corresponds to a single bucket • If a node dies, • We lose a bucket • Must update the hash function forall nodes

  4. Go next! Consistent hashing (II) • Have several buckets per node • Organize all buckets into a ring • Each bucket has a successor • If a bucket is unavailable • Select next bucket

  5. Consistent hashing (III) • Potential problem • Neighboring bucket will become overloaded • Not if we associate with each physical node a set of random disjoint buckets: • AKA virtual nodes • When a physical node fails, its workload get shared by several other physical nodes

  6. Data log architecture • FAWN nodes have two constraints • Small memory • Poor performance of flash drives forrandom writes • FAWN data log architecture • Minimizes its RAM footprint • All writes are append-only

  7. Mapping a key to a value • Through an in-memory hash table • FAWN uses 160-bit keys: • i least significant bits are index bits • next 15 low-order bits are key fragment • Index bits are used to select a bucket • Key fragment is stored in bucket entry • 15 bits + valid bit + 32-bit pointer to address in data log = 48 bits = 6 bytes

  8. The data log • One data log per virtual node • Data log entries consist of • A full 160-bit key • A length field • The actual data

  9. Basic data store functions • Store: • Adds an entry to the log and updates corresponding hash table entry • Lookup: • Locates a data log entry and checks full key • Invalidate: • Marks hash table entry invalid and adds a delete entry to the log (for durability)

  10. Store

  11. Maintenance functions • Split: • Splits a data store between existing virtual node and a new virtual node • Merge • Merge two data stores into one • Compact: • Compacts data log and updates all hash table entries

  12. Split

  13. Split

More Related