SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

SCAPEdev1 Workshop with Andy Jackson at The British Library

sacha-vazquez avatar
sacha-vazquez
11 diapositives · 208 Views · Nov 12, 2014

Join the SCAPEdev1 Workshop with Andy Jackson at The British Library to discuss Preservation Component integration plans, major platforms, and technical coordination. Get to know each other, explore preservation tools, and understand the challenges in ensuring reproducible tool invocation across contexts. Learn about using Hadoop, HBase, and Taverna for preservation workflows.

#scapedev1 workshop with andy jackson at the british library #scapedev1 workshop with andy jackson at the british library ppt #powerpoint #presentation
SCAPEdev1 Workshop with Andy Jackson at The British Library - slide 1
01 / 11SCAPEdev1 Workshop with Andy Jackson at The British Library
1 / 17

Télécharger la présentation

SCAPEdev1 Workshop with Andy Jackson at The British Library
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Welcome First SCAPE Developers’ Workshop Andy Jackson The British Library SCAPEdev1 AIT, Vienna - 6th – 7th June 2011

  2. SCAPEdev1 – The Goals • Get to know each other. • Get familiar with the major platforms. • Outline and discuss the initial Preservation Component (a.k.a. tool) integration plan.

  3. Getting to know me • Andrew Jackson, at the British Library • Technical Coordinator for SCAPE, which means… • Someone to go to when you get stuck • Someone who will look for cross-work-package confusion or integration problems • Someone who will propose solutions if necessary • Chair of the Technical Coordination Committee • Raising and resolving technical integration issues, etc. • Open – let me know if you want to sit in or raise an issue • Will meet via Skype monthly

  4. Getting to know each other • Round-the-room • Let’s get it out of the way… • And… • Talk together • Debug together • Eat together • And finally, put your picture on the SCAPE wiki and/or Sharepoint… • …if you don’tmind.

  5. Where are Preservation Components used? • SCAPE Testbeds (TB) • Taverna designs workflows that run the tools. • SCAPE Platform (PT) • Executing tools and workflows at scale. • Preservation Planning & Watch (PW) • PLATO uses tools on sample files during planning. • And beyond… • Integration into tools, repositories, institutions, command-line interfaces (CLI) & scripting, etc…

  6. Testbeds & Taverna • SCAPE Testbeds are generating scenarios and preservation workflows that explore them • Uses Taverna to build workflows that invoke our tools • WSDL/SOAP calls mature • RESTful style maturing rapidly • Core integration of CLI plugin as of 2.3 • We’ll play with it this morning

  7. The SCAPE Platform • Initially, vanilla Hadoop, HDFS and HBase • Pure Java preferred • Local CLI okay too • Later on, running Taverna workflows • Pure Java, local CLI • Or locally deployed web services if necessary • Later on, may invoke services from inside VMs • e.g. when tools need a particular OS • Web services would make integration easier

  8. Why HBase? • Hadoop+HDFS provides massive fault-tolerant file system and processing • But HDFS does not cope well with lots of 'small' files • See here for information:http://www.cloudera.com/blog/2009/02/the-small-files-problem • HBase architecture is a good fit for our use cases • HBase is used in many places… • Including web archiving at SCAPE partners • We’ll play with Hadoop and HBase this afternoon

  9. PLATO & Wider Integration • PLATO Planning Tool (PW) • CC, PA and QA during planning process • Stable API web services required (WSDL or REST) • Repository Integration • Local CLI for some (e.g. ePrintsPLATO integration [*]) • RESTful for others • Web Integration • RESTful style preferred

  10. The Challenges • Reproducible tool invocation across all contexts: • CLI, Java, SOAP / REST • But ease of development and deployment is critical • Interoperable data formats and consistent semantics across contexts where required • So clients can understand tool outputs correctly • Tomorrow we’ll explore the proposed integration plan

  11. Questions?