1 / 39

CS Communication & Systèmes Applied Grid For SMEs

CS Communication & Systèmes Applied Grid For SMEs. romain.nougarede@c-s.fr. +3,300 employees (70% engineers) Headquarters and Regional offices in France 6 subsidiaries in Europe Subsidiaries in Canada, USA, Brazil and Chile 2004 Consolidated Sales : € 348 Millions.

nate
Télécharger la présentation

CS Communication & Systèmes Applied Grid For SMEs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS Communication & Systèmes Applied Grid For SMEs romain.nougarede@c-s.fr

  2. +3,300 employees (70% engineers) Headquarters and Regional offices in France 6 subsidiaries in Europe Subsidiaries in Canada, USA, Brazil and Chile 2004 Consolidated Sales : € 348 Millions • Long-term partner for our customers in design, integration and operations of critical applications, systems and infrastructures • Focus on applications at the heart of our customers' business : expertise, commitment, innovation, sense of service and ability to manage large and complex projects

  3. Business skills adapted to our clients needs Industrial & Critical Applications Critical Systems Integration Outsourcing& Transitioning IS Network Services SystemsandApplications Infrastructure Services • Critical Real Time software • Mission and Control Centres • Simulation and Scientific Computing • Technical Information Systems • (PLM, EDMS, EAI, Web-services, Virtual Reality) • Security & Business Continuity • Subsidiaries • SCOT • Germany • Canada • Romania • Air Traffic Control Centre • Simulator • Communication • Systems • IntelligenceC4I • Logistic Information • Systems • Toll Systems & • Contact less toll collection • Traffic Management • Subsidiaries • United States • Chile • Croatia Distributed Systems Networks End-to-End Production DataMigration Systems Conversion Application Maintenance Architectureand Design Integration and Deployment Maintenance and Supervision Security Digital Video Surveillance Space and Civil Aviation Aeronautics and Automotive Defence and Military Aviation Public Administrations Energy, Utilities and Telecom Banking and Finance Intelligent Transport Systems Cross-Disciplinary Offers Application Management / Security / Linux & Open-source Software / Grid Computing

  4. Applied Grids for SMEs

  5. SMEs Needs and Challenges • Computing and storage facilities • Security and confidentiality • Capacity to use a set of dedicated applications closed to their industrial needs • Costs reduction by mutualisation and computing on demand • Externalization of IT tasks • ‘User friendly’ interfaces to HPC infrastructures • Business model matching to their financial capacities • Industrial Applications driven Grids • RUGBI : To SMEs of Biotechnology Industry • Openplast : To SMEs of Polymer Processing Industry

  6. Application driven Grid Challenges • Needs • Non computing specialist users. • Light Client with grid frontier (portal). • Information System for user request, management and exploitation. • Technologies Web compliant. • Quality Of Services (QoS) • Transparency. • Security / Confidentiality. • Performance / Permanence. • Easy business oriented increase and scalability. • Interoperability, Evolution compliant, Portability. • Administration / Exploitation / Maintenance. • Distributed Architecture. • Open source and Open file format solutions. • Initial state of the art • DataGrid, E-Toile, GRASP, …

  7. Openplast RUGBI Funded by the French Ministry of Research IT Services company Academic institution Software Editor Infrastructure provider Business oriented interface Two Grid projects gathering:

  8. Client Actors driven Architecture Academics & IT services company: Computing cluster MPI Academics & IT services company: Storage bay IT services company: Controller server Technological park & Editors & Academics: Web Portal Business oriented IT services company: Exploitation server SMEs : End - User

  9. Web Portal server Controller server Academic institutions Exploitation Server Client Computing Cluster Client IBCP [Lyon] ECP [Paris] Storage Bay PEP [Oyonnax] ECP [Paris] High speed networks CC-IN2P3 [Lyon] High speed network CC-IN2P3 [Lyon] INSA [Oyonnax] CS SI [Grenoble] CS SI [Grenoble] BioPôle [Clermont limagne] Client GridsSites Infrastructures Openplast RUGBI LPC [Clermont Ferrand]

  10. The grid portal must provide a user friendly interface for grid management, exploitation and easy use. Grid management User management Sites management Grid Portal Resources management Exploitation services Workflow management Files management Jobs management

  11. Resources management services : • Applications (commercial or not): • Creation. • Deployment. • Configuration. • Update. • Databases (commercial or not): • Creation. • Deployment. • Configuration. • Update. • Storage resources with specific roles: • Transit (all uploaded data) on portal. • Repository (all permanent data) on storage element. • Working (for job submission) on computing element.

  12. Web Portal Soft & DB deployment User Resources Rights Execution/ Workflow Location/ Allocation Files management Exploitation DBMS XML-DB XML Files Information System Data Business oriented applications Business oriented applications Controller Grid Services Globus Batch Manager Operating System Generic layers Specific layers CS GRID Controller: multi-layer architecture • Controllers provide services to manage users, resources and users rights on resources. • Controllers provide allocation services to allocate jobs execution. • Controllers provide exploitation services : accounting, monitoring, …

  13. Technologicalchoices • Globus 2.4, 3.2, and now 4.0 • Java (servlet, CoG) • Web Services SOAP • OpenSource schedulers(PBS, Condor, …) • Exchange format : XML • Information System hosted on DBMS Oracle / Mysql / Xindice wrappers

  14. Exploitation Client Computing Portal Controller Storage Client external Communication Grid internal Communication Portal-Controller Communication Grid Network Architecture • WS-SOAPServer • MyProxy Server • NTPServer • CAServer • MailClient FIREWALL SOAP [8080] MyProxy [7521] FIREWALL SOAP [8080] FIREWALL GridFtp [2811] HTTPS [8443] FIREWALL GRAM-SOAP [8080] GridFtp [2811] • GridFtpServer • GRAMServer • SOAPServer • Client Web • Client SOAP • Client CoG Globus • GridFtp, MyProxy • Client Mail DMZ FIREWALL SOAP [8080] • GridFtpServer • WS-SOAP Server • CoG Globus Gram, • GridFtp, MyProxy Clients • SGDB-XML Server • Web Server • Client WS-SOAP • GridFtpServer FIREWALL GRAM-SOAP [8080] GridFtp [2811]

  15. DMZ • Intranet CS Network • Internet provider • Storage and Computing Network CS Node Network Architecture compliant with internal and external grids

  16. Computing cluster MPI Storage bay Controller server Computing cluster MPI Storage bay Authorization Authentication Exploitation server Site A Connection proxy creation certificate creation Grid basic job submission principles Web Portal server Site B User portal

  17. Computing cluster MPI Storage bay Workflow settings Workflow execution File File File File transfer File upload Job submission Grid basic job submission principles Computing cluster MPI Storage bay Controller server Web Portal server Workflow generation Exploitation server Site A Job execution Site B User portal

  18. Computing cluster MPI Storage bay Updates portal / send mail Files transfer to user space Workflow execution File result Grid basic job submission principles Computing cluster MPI Storage bay Controller server Web Portal server Worflow engine informs user Exploitation server Site A Job execution Site B User portal

  19. Screen Shots of generic grid functions

  20. Intuitive Grid Portal/Controller installation & configuration • Examples : • Installation packages • Controller configuration

  21. User friendly interfaces • Examples : • Grid-client Files Management • User management

  22. Exploiting and Monitoring • Subscription forms • Statistical use diagrams

  23. Specific tools

  24. Tools Sequence alignment Blast Fasta Multiple sequence alignment Clustalw Protein 2D structure prediction Predator Sympa96 GorIV … Docking of compounds to proteins Autodock Metabolic Pathway Analysis RUGBI Tools and databases Automatically Updated Databases Protein knowledgebase Swissprot Trembl 3D protein structure data PDB Encyclopedia of genes and genomes KEGG nucleotide sequence database EMBL compounds diversity set NCI

  25. RUGBI alignment tool:the BLAST example

  26. XML XML XML XML Study view CAD Mesh Data settings Molding Analysis Defects Project Subproject Study Project view Case XML XML XML XML Platform for polymer injection molding simulation • Project management : case-studies oriented • Allows to specify all the stages of polymers injection modeling • Qualifying and Validating each stage of the process • Data enrichment Data settings Post-treatment Mesh Soft. Solver Analysis Plast3D Plast2D PlastTherm3D PlastPart2D PamView PamRTM Geomesh PlastPre PlastDefects CATIA CAD CAD + Mesh CAD + Mesh + DS [LC, IC, FC] CAD + Mesh + DS + Raw results CAD + Mesh + MED + Raw res. + Analysed res. CAD + Mesh + DS + Raw res. + Analysed res. + Post-treated res. DS : Data settings

  27. Prototype Openplast client modules Study Framework Post-treatment framework

  28. Conclusion • Applied grids for SMEs : • Security based on Globus authentication by proxy and certificate. • Confidentiality managed by a strict use of user directories (temporary, working, storage,…) based on user rights. • Exploitation and Accounting facilities for different user type. • Easy of use for non grid specialists. • Basic set of applications (client and grid side). • Expected Grid End-Users • RUGBI • IBCP portal gets daily 2000 connections that will be transferred on RUGBI nodes. • Different SMEs from Clermont Limagne BioPole and others. • Openplast • Polymer processing industry from which 70% are SMEs. • European Polymer Institute.

  29. For a good industrial accomplishment • Starting point: the SMEs real needs • Business Interface partners • Openplast : European Polymer Institute • RUGBI : BioPôle Clermont Limagne. • In progress: • RUGBI: study market with Lyon Biopôle, Arteb, Théogone … • Openplast: partnership with PEP. • IT service company (IN2P3, CS). • Academic institutions for their up-to-date research knowledge. • Software editors: Openplast: ESI, Fluent, … & RUGBI: Accelrys, Tripos, … • Infrastructure: extend from a regional level to national • RUGBI: Theogone and different biopoles and interoperability with future national french bioinformatics grid. • Openplast: HPC pole - Iles de France(IOLS). • European: EGEE2 (functional test OK with GT2.4, and interoperability with Glite in progress).

  30. Contacts http://rugbi.in2p3.fr http://www.openplast.org Project manager : steve.langlois@c-s.fr

  31. Questions and Answers ?

  32. More Technical slides

  33. Information System 1/3 • Manage users, groups and rights on resources. • Manage logical resources : softwares and databases. • Manage physical resources : Computing / Storage elements, Portals …

  34. Database updating RUGBI serviceallows : • Releases updates • Versions updates • Versions management • Versions deployment XML FILE Available files list FTP server Update Metadata (names, date) • Replication • Registration Comparison Download - Waiting end of jobs on old database - Deletion of the old database Available files list Missing files XML FILE

  35. Computing cluster MPI Computing cluster MPI Storage bay Storage bay Slave Controller server Slave Controller server Web Portal server data Modification notification Web Portal server data Modification notification Site A Site C data modification User portal data updates Grid information system update mechanisms Computing cluster MPI Storage bay Master Controller server Exploitation server Web Portal server Site B

  36. Computing cluster MPI Computing cluster MPI Storage bay Storage bay Slave Controller server Slave Controller server Web Portal server data update notification Web Portal server Site A Site C User portal data updates data updates Grid information system update mechanisms Computing cluster MPI Storage bay Master Controller server Exploitation server Web Portal server Site B

  37. Interface << DataRequestor >> Wrapper Requests Backup MySQL MySQLRequestor Controller DataRequestorFactory Wrapper Dump.xml XIndice XIndiceRequestor Restore Information System 3/3 • Wrappers : MySQL, Oracle, PostGres, XIndice compliant. Backup / Restore throw XML files driven by the master controller.

  38. jobs GRID : End-user interface / Application developer interface GRID job1 Workflow Engine Web Portals + Control ler • Application developer • Submit jobs, transfer • files, invoke grid services, receive notification events, etc. • While: • Monitoring execution • Handling errors • Recovering workflow failures • Freeing allocated resources • etc. job2 End-user Choose applications, input files, arguments, etc. execution environment

  39. Execution sequence example Execution sequence executed by the workflow engine <graph wf:NAME="graph-8433" xmlns:wf="http://www.in2p3.fr/workflow/language" xmlns:x0="http://www.w3.org/2001/XMLSchema"> <wf:job wf:NAME="job-8433" wf:contact="sirius.si.c-s.fr:2119/jobmanager-fork" wf:isBatchJob="false" wf:provider="GT2"> <wf:executable wf:shell="/bin/sh" wf:stageIn="true" wf:tmp="/grid/transit/user/1/projects/8433/"> <![CDATA[/grid/software/BLAST/2.4/bin/blastall -p blastn -i $INPUT -d /grid/db/ecoli/ecoli.aa -o $OUTPUT -v 500 > $OUTPUT]]> </wf:executable> <wf:sandbox wf:intermediary="gsiftp://sirius.si.c-s.fr:2811/grid/vo/CS-VO/cs-user1/projects"> <wf:input wf:VAR="INPUT" wf:source="file:///grid/transit/user/1/projects/8433/input.in" wf:destinationdir="$HOME/tmp/8433" > <wf:output wf:VAR="OUTPUT" wf:destination="file:///grid/transit/user/1/projects/8433/output.out" wf:sourcedir="$HOME/tmp/8433"/> </wf:sandbox> </wf:job> <wf:mail wf:NAME="mail-8433" wf:contact="mail.grid.fr" wf:depends="job-8433" wf:from="exploitant@grid.fr" wf:subject="your job 8433 is finished." wf:to="nicolas.demesy@c-s.fr"> <wf:message> <![CDATA[ Your results are on your transit spaceonCS grid portal CS ... ]]></wf:message> </wf:mail> </graph> Workflow used spaces Portal transient space (input and output) /grid/transit/user/1/projects/8433 Temporary CE space gsiftp://sirius.si.c-s.fr:2811/grid/vo/CS-VO/cs-user1/projects/ Temporary Working node space $HOME/tmp/8433 CE software location /grid/software/BLAST/2.4/bin/blastall : CE database location /grid/db/ecoli/ecoli.aa

More Related