1 / 4

Lightning Talks

Lightning Talks. Automated Fixity Checker for Disparate Hardware . Jacob Farmer, CTO Cambridge Computer. Our Project – Defining Best Practices for File Management. Inspiration for our project comes from SRB/IRODS Bring parts of the SRB/IRODS vision to reality

aleta
Télécharger la présentation

Lightning Talks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lightning Talks Automated Fixity Checker for Disparate Hardware Jacob Farmer, CTO Cambridge Computer

  2. Our Project – Defining Best Practices for File Management • Inspiration for our project comes from SRB/IRODS • Bring parts of the SRB/IRODS vision to reality • Define a general purpose feature set • Intuitive user interface • Simplified API • Inspiration also comes from numerous home grown solutions in our client base. • The paradigm: • Stat() your file systems • Make database records for each file and/or directory • Relate metadata to the file and directory records • Report and/or take action

  3. In-Band and Out-of-Band

  4. Building a Fixity Checker Into Starfish • Out of the box, Starfish has most of the key components of a fixity checker • Hash calculations and comparisons are fundamental to the system: deduplication, integrity checking, versioning, content addressing • Walking file systems and object stores is fundamental. • Super fast, parallelized crawler and file processor • Starfish holds any kind of metadata • Position of files on tape • Logging of our activity • Starfish takes metadata from other systems through API • If another system does a fixity check, we can store the history and results • Starfish rules engine works on a schedule and uses metadata as a trigger for activity • We query the metadata catalog and then “do stuff” • The “stuff” could be a hash compare

More Related