1 / 24

AstroGrid’s Common Execution Architecture

AstroGrid’s Common Execution Architecture. Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developers SC4DEVO-1, Pasadena, July 2004. AstroGrid. UK national virtual-observatory project See cover slide for participants and funding

davida
Télécharger la présentation

AstroGrid’s Common Execution Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AstroGrid’s Common Execution Architecture Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developers SC4DEVO-1, Pasadena, July 2004

  2. AstroGrid • UK national virtual-observatory project • See cover slide for participants and funding • Tasked to produce a VO toolkit to be reused by service providers • Collective author of a SOA using web services • Common Execution Architecture: • A major part of AstroGrid architecture • Part of the service-oriented architecture • A way of making and using web services • Some important applications to data mining AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 2

  3. SOA for data mining How can we carve up the distributed application into services? And where do we put the application logic? AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 3

  4. Distributed app: conservative Desktop Application Application logic in here Data-centre 2 Data-centre 1 Data transform Data source AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 4

  5. Distributed app: browser-based Desktop Browser Application logic in here Data-centre 2 Data-centre 1 Data transform Data source AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 5

  6. Distributed app: intragrid Desktop Some application logic in here; Rest still in client or in user Browser/app Node Data source Data transform Node Node Node Data-centre 2 Data-centre 1 Compute grid AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 6

  7. Portal Distributed app: scripted App logic in here Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 7

  8. Portal Distributed app: embedded Little app logic here Desktop Script Browser Workflow engine Most app logic here Data source Data transform Application Data mine Data-centre 1 Data-centre 2 AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 8

  9. Portal Distributed app: data grid Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 9

  10. Boring, technical: How to write apps as w/s? How to write w/s clients? How to connect apps to storage? How to register functions? Interesting, scientific: Who has useful algorithms? How to match algorithms to data? Best place to run computations? How to share code? Provenance? How to share kudos? Is more science possible? Issues to be addressed AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 10

  11. AstroGrid architecture and products • AstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate on the science issues. • AstroGrid supports: • Browser-based app • Scripted app • Data grid • Embedded app (pre-installed exes) • Common Execution Architecture lets all parts but exe for embedded app be reusable code. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 11

  12. Portal Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 Common Execution Architecture (1) “CEC” = Common Execution Connector Web service (CEC) MySpace CEC configuration Application (local exe) Data mine AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 12

  13. Portal Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 Common Execution Architecture (2) Scripts stored here, as files MySpace Page presentation Workflow definition Workflow executor (JES) Portal AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 13

  14. Common execution architecture (3) Common language throughout system allows s/w reuse. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 14

  15. How to write an app using CEA? • Don’t write a web service! • Write a command-line exe • Configure a CEC to run your app: • I.e. describe parameters in the CEA language • Or: get your app configured on someone else’s CEC AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 15

  16. How to publish an app using CEA? • Take the <ApplicationDefinition> that informs the CEC. • Wrap it in a <VOResource> of sub-type CeaApplicationType. • Publish the <VOResource> in IVO registry. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 16

  17. How to call an app-service using CEA? • Look it up in the resource registry. • Get the <ApplicationDefinition> from its <VOResource>. • Give the <ApplicationDefinition> to a CEA delegate => define SOAP call. • Add parameters from UI according to <ApplicationDefinition>. • Bake in pre-heated SOAP engine @ HTTP 1.1. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 17

  18. CEA language example <ApplicationDefinition> <Parameters> <ParameterDefinition name="DetectionImage" type="binary"> <UI_Name>Detection Image</UI_Name> <UI_Description>The image that is used to detect sources. Basic position, shape and size information is derived from this image</UI_Description> </ParameterDefinition> <ParameterDefinition name="PhotoImage" type="binary"> <UI_Name>Measurement Image<UI_Name> <UI_Description>The Image that is used to measure photometric parameters</UI_Description> </ParameterDefinition> <ParameterDefinition name="config_file" type="text"> … AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 18

  19. CEA language schema (1) AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 19

  20. CEA language schema (2) AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 20

  21. JES CEA Delegate Other Delegate PAL CEC HyperZ CEC SIAP CEC CEA facilities so far Other service MySpace AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 21

  22. Future evolution • Add facilities to CEC: • Asynchronicity • Security • More parameter types • New facility: field-programmable CEC (PCEC) AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 22

  23. PCEC (1): local code libraries Client Select exe PCEC Load exe Code library Data mine AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 23

  24. PCEC (2): distributed code-library VOSpace Client Code library Upload exe Select exe PCEC Download exe Sandbox Run exe Data mine AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 24

More Related