Component Frameworks and Programming Approaches for Application Web Services

1. Component Frameworks and Programming Approaches for Application Web Services Rainer Schmidt, Austrian Research Centers – ARC rainer.schmidt@univie.ac.at VGE-CCA

2. The VGE-CCA Component Framework VGE and VGE-CCA extensions CCA libraries, Infrastructure Elements, Client Environment Application Component Libs (clibs) Plugged into the Web Service Container Distributed Application Coordination Programming Model Component Creation and Composition, Handling State Data Staging, Coupling, Infrastructure Services The Planets Interoperability Framework A Service-oriented Architecture for Digital Preservation Component and Programming Model Configuration and Execution Environment Outline

3. CCA is designed for high-performance application Facilitates import of scientific code into frameworks. Provides a plug-and-play environment 3 party composition capabilities, based on Interactions CCA Fundamental Concepts Components: Units of software that can be composed Frameworks: Environment for components to run and interact CCA Specification defines behavior for component and framework interaction Platform and language neutral. does not prescribe implementation details Applicable to parallel and distributed/Grid environments. Common Component Architecture

4. HPC Service provision framework Encapsulates native HPC applications on parallel hardware Offers common set of services (conversational) job execution, monitoring, file transfer, QoS, error recovery Negotiable QoS support for time-critical applications Advance Reservation, Perf. Models Dynamic Configuration, WSLA Standard Web service technology AXIS, SOAP attachments, WS-Addressing Security: TLS, WS-Security, X.509 certificates, End-to-End Application services, Registries, High-level client API EU Projects: GEMSS, Aneurist CA Client Service Registry VGE - On-Demand Supercomputing

5. Distributed CCA framework atop a Web service based Grid environment Models VGE application services as distributed CCA components. Adds port-based composition capabilities to Web services. Plugin mechanism for application-specific libraries Model for Distributed Grid Programming Actively interacting Grid application components sequentially, concurrently Co-scheduling concurrently running Grid components. Client composition and computational steering. VGE-CCA

6. Service-side CCA Libraries No modification of VGE core functionality (still a valid VGE service) Adds Builder service (creation and connection) Provide CCA framework to the application service. Transparently utilize remote framework services (registry, proxy) Component Libraries (clibs) Plugin Mechanism Extend VGE comp. with app. specific. logic, ports, dependencies Method for distributed application coordination. Framework Web Services Component Registry, Proxy Registry, Negotiation Service Component-based Client API Distributed programming by interlinking parallel app. components. The VGE–CCA System

7. app. specific comp. code App. Dsc. Anatomy of a VGE-CCA Component binaries, files OS-levelinteraction VGE Ports (App, QoS, Recovery) App.Handler … VGE libs Creation + Connection Port service specific comp. code connected Services App. Specific Provides Ports Port+ Conn. Table CCA Types CCA Services CCA lib Fw. Web Service (component, proxy repository)

8. Component Creation by Contextualization Conversational Identifier, WS - Addressing Different Types of Connections stateful + stateless interactions sequential + concurrent running applications Port Connections to Infrastructure Services Connect Grid component to Infrastructure Web service E.g. CRL-Service, Logging Service Connect using port mechanism visible dependencies Configured descriptively or dynamically found Creation and Composition

9. Coupling Application Parts Co-scheduled application components. VGE Advance Reservation Selection: QoS Negotiation Service Port-based Interactions of concurrently running Components interact as needed Application Specific Component Plugins vgeJob uSignal C pSignal vgeJob uSignal pSignal Application Coupling Ant Colony Optimization: - Component signals current best solution

10. ACO algorithm for solving a VRP. MPI-based application withmixed parallelization strategy Several colonies of ants cooperate in finding good solutions. Cooperating colonies periodically exchange single solutions. Example: Ant Colony Optimization Basic Decomposition Approach: • Partition each colony of ants into n (# processors) sub-colonies (fine grained) • Distributed coarsely grained families. • Provide Ant Colony as service. • Exchange information between colonies

11. Distributed Coordination Ant Application Components exchange solutions as required. Client Application remains unaware of application specific interaction logic Client is not involved into computation, may go offline. Only required for application steering (start, stop, …) Idea: Aggregate Distributed Resources

12. Application Specific Plugins • Pluggable Component Libraries (clibs) • hooks needed to extend an application service • intercept port invocations • add custom ports and dependencies • application specific component interactionusing the service-side CCA Framework • "Signal" Interface added for Ant components • Provides and uses Port • Implemented as one-to-many connection • Daemon keeps track of local optimum andsignals if required.

13. VGE-CCA Programming Model Abstract Specification (ports, apps, quality) registry and negotiation Spatial Composition port-based connections, dynamic proxy Control-Flow (Computational Steering, Monitoring, Data Staging) Client workflow orchestration Separates applications composition from runtime control flow steering VGE-CCA provides an extensible API for basic CCA constructs and specialized components. Programming Model

14. ACO Client Application //component description cmpDsc.addProperty(CConst.APP, "ACO"); qosDsc.setBeginTime(beginTime); qosDsc.setWallTime(wallTime); cmpDsc.addProperty(CConst.qosDsc, qosDsc); //creation and application assembling VgeComponent ant1 = builder.createInstance(cmpDsc); ... AntComponentGroup antComposite = new AntComponentGroup(ant1, ant2,ant3); //client workflow antComposite.upload(vrp_infile); antComposite.start(); while(antComposite.getStatus != ...) {

15. Integrated Services for Digital Preservation IST FP6 Project - IP €14m Consortium of Libraries, Archives, Universities, Tech. Companies http://www.planets-project.eu OGF Preservation Environments RG Focused on Data Grids Sub-Project: Interoperability Framework Defines a SOA for Digital Preservation builds atop Application Web Service Model Interoperability, Virtualization Planets Project

16. Integration of heterogeneous environment Existing Applications, 3rd Party Services, Repositories Massive Data Amounts, Complex Object Types Results in Complex Grid Applications and Workflows Strong focus on data provenance Develop a Common Data Model Ensure repository consistency Provide Transaction model Component-based Programming Model Abstract Complexity Configuration and Execution Plug-and-Play Environment Distributed Digital Preservation System

17. Generic, distributed portal framework Ties together the involved components Migration, Emulation, Characterization Services Preservation Planning, Format Registries, Repositories Common Services: User Management, Service Registry, Workflow Enactment, Security Provide Programming and Execution Environment Currently based on EJB/Web Service Model Add Independent DP Component Model Programming and Workflow features Planets Interoperability Framework

18. IF Architecture

19. App. Service - Abstraction Levels Level 0: Application Level 1 – Web Services Level 2 – Planets Components A B C Workflow - Level Planets IF - Instance Planets IF - Instance

20. Abstraction based on layered approach Level 1: Any network accessible applications (services) Level 2: Compostable HL Abstractions (components) API/Type System for digital preservation components High-level with explicit context dependencies Defined interfaces and composition Exchange/manipulate a common data model. Life cycle, exception handling, … Abstracts data model on component level Implicitly maintain information about provenance of data Automate Repository Interaction Consistency, Transactions, Abstraction Components Model

21. Levels of application abstraction and construction Different user groups with different expertise: PA -> Component developers; PP -> Workflow developers; Archivists, Libarians -> User. Workflow Templates Assembled from high-level preservation components Independent from underlying services/middleware Rely on provider concept Workflow Configuration and Instantiation Automatically by Preservation Planning Services OR Based on Graphical Interface (Portal) Process archive consists of template and associated view. Programming and Execution

22. Light-weight App. Service Containers No changes to underlying application (legacy apps.) Advanced Grid Services (SLA, interactions) Provide High-Level Programming Model Req. Extensibility: Based on Common Independent Concepts Abstract and Independent from underlying Middleware Allow Encapsulate Application Complexity Distributed Coordination Implicitly interact with Infrastructure Services Great Potential for Cloud Computing Bridge SOA and Component Technologies Composition, Dynamic Resource Selection Conclusions