Download
file systems for clusters n.
Skip this Video
Loading SlideShow in 5 Seconds..
File Systems for Clusters PowerPoint Presentation
Download Presentation
File Systems for Clusters

File Systems for Clusters

259 Vues Download Presentation
Télécharger la présentation

File Systems for Clusters

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. File Systems for Clusters Peter J. Braam Carnegie Mellon University

  2. Plan • 4 advanced systems • Coda • InterMezzo (prototype) • GFS • Lustre (presently a design) • compare • features • data paths • protocols • applicability Distributed File Systems for Clusters

  3. Features • Coda: • mobile use, server replication, security • GFS: • shared storage file system, logical volumes • InterMezzo: • Smart “replicator”. Exploits disk fs. • Lustre: • shared storage file system • likely best with smarter storage devices Distributed File Systems for Clusters

  4. Data transport • Usually several transport mechanisms: • file data, directory data, inode meta data • Coda Storage Format: • not compatible with local file system • not identical on client & server • NFS: • does not transport directory data • InterMezzo & Coda: • have persistent cache Distributed File Systems for Clusters

  5. File data Data Paths Inode meta data Directory data InterMezzo, NFS Coda, Lustre Client FS Objects Server FS Objects InterMezzo Lustre, Lustre Client Buffers Server Buffers InterMezzo, Coda GFS, GFS, GFS Coda Client cache (Server) Disk NFS Distributed File Systems for Clusters

  6. Mechanisms • GFS: lock and get/put blocks • NFS: remote execution almost everything • Lustre: • talks to disk with “inode methods” • Coda: • maintain client cache, access through Venus • InterMezzo: • act as filter/journal driver for local fs Distributed File Systems for Clusters

  7. InterMezzo Lento: Cache Manager & Server Other lento’s Ship when full mkdir... create... rmdir... unlink... link…. no VFS Filter: data fresh? Local file system Presto Kernel Update Journal Distributed File Systems for Clusters

  8. Protocols • Callbacks: guarantee for currency • Tokens: exclusivity for updates • Versions: validation of cached data • Tradeoffs (see. Amiri, Gibson etal) Distributed File Systems for Clusters

  9. Conflicts • Coda is optimistic: • Allow conflicting updates • May require human interaction to fix • Unsuitable when: • daemons exploiting network file system • InterMezzo so simple that: • sharing semantics can be tuned Distributed File Systems for Clusters

  10. GFS & Lustre • GFS • Allows for heavy write/write sharing • Can exploit storage arrays • Has disk based locks • Lustre • Between Calypso (IBM) & GFS • Will use a DLM, more complicated protocol Distributed File Systems for Clusters

  11. Use scenarios • Coda: • clients without much write sharing • security, disconnected operation • reasonably robust; needs more speed & scale • GFS: • shared storage devices (need Dlocks) • write sharing, logical volumes. • Still needs recovery & cluster management Distributed File Systems for Clusters

  12. Near future scenarios: • CMU Nasd: • ask Garth Gibson • InterMezzo: (guess: 6 months) • replication of file trees • good semantics, recovery, disconnected op • local disk speed, • redundancy without shared storage Distributed File Systems for Clusters