1 / 0

Science Gateways for improving the uptake of Grid and Cloud Computing Infrastructures

Science Gateways for improving the uptake of Grid and Cloud Computing Infrastructures. Tamas Kiss, Stephen Winter, Gabor Terstyanszky , Thierry Delaitre University of Westminster Wibke Sudholt , CloudBroker GmbH Peter Kacsuk , Zoltan Farkas , SZTAKI

hosea
Télécharger la présentation

Science Gateways for improving the uptake of Grid and Cloud Computing Infrastructures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Science Gateways for improving the uptake of Grid and Cloud Computing Infrastructures

    Tamas Kiss, Stephen Winter, Gabor Terstyanszky, Thierry Delaitre University of Westminster WibkeSudholt, CloudBroker GmbH Peter Kacsuk, ZoltanFarkas, SZTAKI Jens Krüger, Sandra Gesing, University of Tübingen
  2. gsissh -p 2222 grid-compute.oesc.ox.ac.uk globus-job-submit grid-compute.oesc.ox.ac.uk/jobmanager-pbs  -q RXXXXX /bin/hostname -f globus-job-status <uid> globus-job-get-output <uid> globus-job-clean <uid> Using Distributed Computing Infrastructures is not so easy or is it? ! ?
  3. SCIentific Gateway Based User Support SCI-BUS is a 3-year EU FP7 project Started on 1stOctober 2011 Build around 30 science gateways to various user communities in Europe 10 academic and 5 commercial partners from 10 European countries.
  4. Main objectives of SCI-BUS Create a generic-purpose gateway technology Elaborate an application-specificgateway building technology and customisation methodology Establish production gateway services both for NGIs and various user communities Provide seamless access to major computing, data and networking DCIs and services including supercomputers, clusters, grids and clouds Create and maintain a Liferayportlet repository that enables the quick creation of user specific customised gateways Provide gateway development and application development support Develop business models to guarantee the sustainability of the gateway services and to enable the commercial exploitation
  5. Value brought by the project SCI-BUS will serve large number of user communities by providing generic-purpose and customized scientific gateways for them. SCI-BUS will also create and provide access to many different portlet and application repositories to help user communities in sharing and running these applications via the SCI-BUS gateways
  6. The SCI-BUS user communities Project partners: International seismology community Helio-physics community Swiss systems biology community of the SystemsX.ch project German MoSGrid computational chemistry and bioinformatics community Biomedical researchers community of the Academic Medical Centre of the University of Amsterdam Astrophysics community PireGrid SMEs community Business process modelling community involving a wide range of areas such as finance, healthcare, government, production, robotics and emergency Blender rendering community Citizen web-2 community Public application developer community Subcontractors: 6 sub-contractors to be selected representing additional user communities Hugely competitive – over 3 times as much applications Additional user communities supported by the project
  7. The SCI-BUS architecture The gateway services will enable unified access to all the major DCI infrastructures used in Europe: Local and remote clusters accessed by queuing systems (PBS, LSF, Condor, SGE) Local and remote supercomputers Local and volunteer Desktop Grids (BOINC, XtremWeb, OurGrid) Grids (ARC, gLite, Globus, UNICORE) Academic Public and Private Clouds (OpenNebula, Eucalyptus, etc.) Public Commercial Clouds (Amazon EC2, IBM Cloud, etc.)
  8. The SCI-BUS Infrastructure
  9. NationalGrid portals (UK, Ireland, Croatia, Turkey, Spain, Belgium, Malaysia , Kazakhstan, Switzerland, Australia, ClGrid, etc.) HP-SEE/SEE-GRID-SCI, BalticGrid US Open Science Grid, TeraGrid IDGF (International Desktop Grid Federation) SHIWA Simulation Platform Economy-Grid, Swiss BioGrid, Bio ésBiomed Virtual Organisations, MathGrid, etc. HP-SEE The baseline technologyP-GRADE Portals around the globe
  10. 2010 2009 2008 2011 Open source since 2008 Open source since 2011 P-GRADE Portal software family GEMLCA Grid Legacy Code Arch. P-GRADE Portal 2.4 P-GRADE Portal 2.5 GEMLCA, storage concepts NGS P-GRADE portal WS-PGRADEPortal 3.1 P-GRADE Portal 2.8 Basicconcepts WS-PGRADEPortal 3.2 P-GRADE Portal 2.9.1 WS-PGRADEPortal 3.3 P-GRADE Portal 2.10 WS-PGRADEPortal 3.4
  11. Motivations of creating WS- PGRADE/gUSE To overcome (most of)the limitations of P-GRADE portal: To provide better modularity  to replace any service To improve scalability  to millions of jobs To enable advanced dataflow patterns To interface with wider range of resources To separate Application Developer view from Application User view WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment) and gUSE(GridUserSupportEnvironment) architecture
  12.               WS P-GRADE in a nutshell General purpose, workflow-oriented portal.Supports the development and execution of workflow-based applications Based on Liferay Servicessupported by the portal: New functionalities Web services DB connectors Embedded workflows Job level PS Conditional jobs Recursive graph Multi-generator Multi-collector CROSS product PS DOT product PS  
  13. WEB-UI(HTML) WS WS GMLCA Grid Glite Grid Glite Grid GT2 Grid GMLCA Grid GT4 Grid GT4 Grid LCG Grid LCG Grid GT2 Grid GT2 submitter WS (Axis) submitter GEMLCA submitter GT4 submitter Glite submitter local submitter LCG submitter Inside of the SystemgUSE tires Backend WFInterpreter WF Storage Liferay WS-PGRADE portal DCI-Bridge File Storage WF Graph editor ApplicationRepository Information System
  14. Concrete Workflow Graph Template Workflow Instance Repository Item Algorithms,Resource references,Inputs Jobs,Edges,Ports Constraints,Comments,Form Generators Running state,Outputs Application ORProject OR,Workflow part(G,T,CW) WS P-GRADE workflow concept Legend:aba must reference baba may reference b
  15. ASM Application Specific Module A new module called “Application Specific Module” has been developed for gUSEthatease the creation of ApplicationSpecified Portlets The moduleprovides programmingAPI to control applications developed using gUSE The moduleprovides programmingAPI to access inner gUSE services Any kind of Graphical interface can be developed on top of this module GUI written in JSP, JSTL (Java Servlet Tag Library ) and Java GUI can be customized for the grid application GUI can be customized for end users
  16. ASM - Application Specific Module GridApplicationDevelopercreatesownGridApplicationviaws-PGrade and export ittothe local repository ThentheGridPortalDevelopercreates a newweb-application (mainly an web-interface) whichfitstotheusercommunity'sneeds, and connectsittothegUSEusing ASM API Finallytheend-userscreateownapplicationfromtheexportedone, parameterize and executeitusingthedevelopedportlet gUSE Developed web-interface ws-PGrade GridApplication ASM << import >> Storage Workflow Interpreter Repository User space GridApplication Information GridApplication Submitter Workflow Storage DCI systems
  17. CloudBroker Platform Easy, scalable, secure, integrable and pay-per-use access to scientific and technical applications in the cloud High performance computing application store and marketplace with accounting and billing Deployment and execution of applications in the cloud with the same parameters and files as for local execution Accessible through the internet via a web browser UI Integration possibilities into third party tools via web service-based APIs Public version available under https://platform.cloudbroker.com Using infrastructure as a service (IaaS) from cloud providers Offering platform as a service (PaaS) for software vendors Providing software as a service (SaaS) to end users CloudBroker GmbH
  18. Platform Architecture R&D End Users and Software Vendors CLI WebBrowserUI GenericWorkbenches Domain-Specific Gateways Web Service API CloudBroker Platform CloudBroker Integration ChemistryAppli-cations …Appli-cations BiologyAppli-cations MedicineAppli-cations Engineering Appli-cations …Cloud AmazonCloud IBMCloud CloudBroker GmbH
  19. CloudBroker Business Model Free access to the platform after registration Each organization manages its own users Prepayment or monthly billing Platform usage costs =resource usage costs +software usage costs + CloudBroker usage surcharges Usage $ Resources Applications $ $ CloudBroker GmbH
  20. Clouds and Applications Currently supported cloud infrastructures: Amazon Web Services IBM SmartCloud Enterprise Adapters to further clouds planned Open source-based private cloud tools will be interfaced within SCI-BUS Current applications: Chemistry (GAMESS, Gromacs) Biology (BLAST, X! Tandem, Rosetta) Medicine (AutoDock) Engineering (OpenFOAM) Further applications in preparation Own applications can be easily added by users CloudBroker GmbH
  21. CloudBroker Platform Integration in Liferay/WS-PGRADE/gUSE IntegratetheSaaScapabilities of CB intothegUSE/WS-PGRADE system. Enable WS-PGRADE userstorunsoftwares (SaaS) exposedbythe CB platform. Job ConfigurationPortlet CloudBrokerUI Portlet UserMappingPortlet WS-PGRADE CloudBrokerplugin DCI-BRIDGE ExposetheCloudBroker UI through WS-PGRADE interfaceby proxy. gUSE CB Java API API UI SaaS (Softwaresas a Service) Rosetta Gamess ... Blast Xtandem Autodock Gromacs Open Foam
  22. MosGrid Science gateway for molecular simulations Requirement: Workflow enabled grid portal for UNICORE based on Liferay Solution: Adaptation of WS-PGRADE from GridSphere to Liferay Development of UNICORE Submitter Support for SAML Development of ASM (Application Specific Module) support
  23. MosGrid Science gateway - architecture
  24. MosGrid Science gateway - architecture
  25. MosGrid Science gateway – application domains Quantum Chemical Calculations based on approximated solutions of the Schrödinger equation Average scalability Gaussian, NWChem, Turbomole, … Molecular Dynamics based on forcefields describing molecular interactions Good scalability Gromacs, NWChem, Amber, … Docking Based on simplified forcefields Excellent scalability CADDSuite, FlexX, …
  26. MosGrid Science gateway – application domains – molecular dynamics

  27. Thank you for your attention!

    Any questions? http://www.sci-bus.eu/
More Related