1 / 11

Kepler + MeDICi - Service-Oriented Scientific Workflow Applications

Kepler + MeDICi - Service-Oriented Scientific Workflow Applications. Jared Chase, Ian Gorton, Chandrika Sivaramakrishnan, Justin Almquist, Adam Wynne, George Chin, Terence Critchlow IEEE 2009 International Workshop on Scientific Workflows (SWF 2009) Los Angeles, CA, July 6-10, 2009.

carlyn
Télécharger la présentation

Kepler + MeDICi - Service-Oriented Scientific Workflow Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Kepler + MeDICi - Service-Oriented Scientific Workflow Applications Jared Chase, Ian Gorton, Chandrika Sivaramakrishnan, Justin Almquist, Adam Wynne, George Chin, Terence Critchlow IEEE 2009 International Workshop on Scientific Workflows (SWF 2009) Los Angeles, CA, July 6-10, 2009

  2. Origins of MeDICi … • Started using Mule to build some applications • Successful, but … • XML/Mule configuration and concepts complex • Limited for orchestrating long running jobs • MeDICi addresses these: • Simplified Java API • Integration with BPEL • DWF – our simplified version • Retains Mule’s robustness/flexibility 2

  3. MeDICi MeDICi Workflow Apps • Requirements • Fast for streaming apps • Orchestrate long running jobs/workflow apps • Scalable through distribution • Robust • Simple concurrency model • Highly flexible integration • Protocols • Languages • Platforms • Download from: • http://medici.pnl.gov DWF BPEL MeDICi Streaming Apps MeDICi Integration Framework Mule 2.0 API 3

  4. MeDICi Integration Framework • Java-based integration platform • Low friction • Component-based API for creating analytical pipelines • Asynchronous component model with multi-language support • Scalable through replication/partitioning • Configurable communications • Simple programming model for copying large data sets • Built on robust Java technologies • Service-Oriented Architecture (Mule open source ESB) • Java Messaging Service (e.g., JBoss, ActiveMQ, SonicMQ) • ehcache 4

  5. MIF Component Builder 5

  6. BBHRP Workflow in DWF 6

  7. Use Case: BBHRP Application ARM Project

  8. Kepler • Kepler is an established technology for building scientific workflows. • Kepler has many actors (or components) in a library that can be reused in various workflows. • Components Include: Ssh, remote command execution, job launching to various queuing systems, etc. • Workflow designers can drag and drop actors onto a canvas and connect them to form complex workflows.

  9. Kepler and MeDICi Integration • MIF components expose a Web service interface that must be invoked asynchronously by Kepler. • Kepler exposes a web service that MIF components call to restore control flow back to the workflow. • Intermediate files stored in a shared file system • Only file references are passed between MIF components and to/from Kepler

  10. Conclusions • Advantages for using Kepler with MeDICi to execute the BBHRP workflow include: • Split the BBHRP processing into simple, independent components rather than a large monolithic code • View the workflow at a higher level of abstraction and construct it using a simple drag and drop design workbench • Execute parts of workflow in parallel by simply changing a configuration. • Simply modify parts of the workflow, such as using a different MIF component • Asynchronous communication between Kepler and MIF eliminates the need for repeated polling for results from the workflow engine, which reduces overheads and improves scalability. • Still issues to address, eg: • Exception handling • Notifications • ???

  11. Acknowledgment Funding for this research is provided by the U. S. Department of Energy.

More Related