Download
slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Denis Caromel Institut universitaire de France (IUF) OASIS Team PowerPoint Presentation
Download Presentation
Denis Caromel Institut universitaire de France (IUF) OASIS Team

Denis Caromel Institut universitaire de France (IUF) OASIS Team

157 Vues Download Presentation
Télécharger la présentation

Denis Caromel Institut universitaire de France (IUF) OASIS Team

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Programming, Composing, Deploying for the GRID Denis Caromel Institut universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis JAOO, Cannes, May 2004 • 1. Grid principles • 2. Distributed Objects and Components: • ObjectWeb ProActive LGPL environment • 3. Application: 3D Electromagnetism • 4. Towards Peer-To-Peer (P2P)

  2. The GRID PCs : 1 Milliard in 2002 (25 years) Forecast: 2 Milliards in 2008

  3. The Grid idea • GRID = Electric Network • Computer Power (CPU cycles) <==> Electricity • Can hardly be stored, if not used --> Lost • Global management, • Mutual sharing of the resource • But CPU cycles much harder to share than electricity: • Production cannot be adjusted! • Cannot really be delivered where needed! • Not yet interoperability: MultipleAdministrative Domains • 2 important aspects : • Computational + Data Grid

  4. Example Challenge: Scale up • 50 Machines, 1.5 year of computation • 5000 Machines, with only 50 % Efficiency • ==> 10 days • Applications: • Simulate the stock market evolution • Research for an urgent vaccine • Forecast a bush-fire path • Forecast in real time a flooding consequences ... • etc.

  5. Ubiquitous: some numbers • PCs in my lab (INRIA Sophia) : ~ 1500 French Riviera : 1.3 Millions • France : 25 Millions Europe : 108 Millions USA : 400 Millions • World : 1 Milliard in 2002 (25 ans) Forecast: 2 Milliards in 2008 • France : • 36 Millions of cellular phones • 2.2 Millions of laptops • 630 Thousand PDA (sources: ITU, Gartner Dataquest, IDC, 02-03, )

  6. The multiple GRIDs • Scientific Grids : • Parallel machines, Clusters • Large equipments: Telescopes, Particle accelerators, etc. • Enterprise Grids : • Data, Integration: Web Services • Remote connection, Security • Intranet and Internet Grids, (miscalled P2P grid): • Desktop office PCs: Desktop Intranet Grid • Home PC: Internet Grid (e.g. SETI@HOME)

  7. Internet Apache Servlets EJB Databases Enterprise Grids

  8. Large Equipment Internet Parallel Machine Clusters Scientific Grids

  9. Internet Job management for embarrassingly parallel application (e.g. SETI) Internet Grids

  10. The multiple GRIDs • Scientific Grids • Enterprise Grids • Intranet and Internet Grids Strong convergence in process! At least at the infrastructure level, i.e. WS

  11. Grid: from enterprise ... to regional Very hard deployment problems … right from the beginning

  12. Grid: from regional ... to worldwide • Communication Cannes-Los Angeles: 70 ms Light Speed • Challenge: Hide the latency ! Define adequate programming model

  13. Distributed Objects and ComponentsProActiveProgramming

  14. SMP LAN Clusters ProActive:A Java API + Tools for the GRID Parallel, Distributed, Mobile, Activities, across the world ! Desktop • Model: • Remote Objects (Active Objects vs. Java RMI) • Asynchronous Communications, with automatic Futures • Group Communications, Migration (computation mobility) • Environment: • XML Deployment, dynamic class-loading • Various protocols: rsh, ssh, LSF, Globus, BPS, ... • Graphical Visualization and monitoring: IC2D

  15. A V Creating AO and Groups A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param); ... v.bar(); //Wait-by-necessity Group, Type, and Asynchrony are crucial for Cpt. and GRID Typed Group Java or Active Object

  16. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder)

  17. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder)

  18. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct

  19. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct direct

  20. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

  21. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

  22. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

  23. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

  24. Parallel, Distributed, HierarchicalComponentsfor the GridComposing

  25. Component interface My Business Component Facets Receptacles OFFERED REQUIRED Event sinks Event sources Attributes A CORBA Component Courtesy of Philippe Merle, Lille, OpenCCM platform

  26. Building CCM Applications =Assembling CORBA Component Instances Provide + Use, but flat assembly

  27. Controller Content The Fractal model:Hierarchical Component Defined by E. Bruneton, T. Coupaye, J.B. Stefani, INRIA & FT

  28. Controller Content Interface = access point

  29. Controller Content Hierarchical model : composites encapsulate primitives, which encapsulates Java code

  30. Controller Content Binding = interaction

  31. Controller Content Binding = interaction

  32. Component Identity Content Controller LifeCycle Controller Binding Controller Controllers : non-functional properties Controller Content Component = runtime entity

  33. D C ProActiveComponents for the GRID 1. Primitive component Java + Legacy An activity, a process, … potentially in its own JVM 2. Composite component Composite: Hierarchical, and Distributed over machines 3. Parallel and composite component Parallel: Composite + Broadcast (group)

  34. Composing: XML ADL • Primitive-component"cd-player” • implementation = "CdPlayer” // Java class with functional code • Providesinterface "input” … • Requires … • VirtualNode = VNa // Virtual Node name • Composite-component”stereo” • VirtualNode = VNc, vn ... // Virtual Node • Provides …Requires … • Primitive-component”cd-player” • Primitive-component”speaker” • Bindingsbind "cd-player.output" to "speaker.input" • Merging VNa, VNb, ---> VNc // Co-allocation in that case Example of an XML component file:

  35. P A B A A C B D C Group proxy Group proxy Groups in Components A parallel component! Broadcast at binding, on client interface At composition, on composite inner server interface

  36. Migration Capabilityof composites • Migrate sets of components, including composites

  37. Migration Capabilityof composites • Migrate sets of components, including composites

  38. Co-allocation, Re-distribution • e.g. upon communication intensive phase

  39. Co-allocation, Re-distribution • e.g. upon communication intensive phase

  40. Co-allocation, Re-distribution • e.g. upon communication intensive phase

  41. Environment Deploying

  42. Large Equipment Internet Internet Internet Apache Servlets EJB Databases Job management for embarrassingly parallel application (e.g. SETI) Parallel Machine Clusters How to deploy on the Various Kind of Grids ?

  43. Abstract Deployment Model • Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. • A key principle: Virtual Node (VN) + XML deployment file • Abstract Away from source code: • Machines • Creation Protocols • Lookup and Registry Protocols • Protocols and infrastructures: • Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF, ...

  44. JVM on the current Host JVM started using SSH Mapping Virtual Nodes: example (1) • <processDefinition id="linuxJVM"> • <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> • </processDefinition> • <processDefinition id=”sshProcess"> • <sshProcess class="org.objectweb.proactive.core.process.ssh.SSHJVMProcess" • hostname="sea.inria.fr"> • <processReference refid="linuxJVM"/> • </sshProcess> • </processDefinition> Infrastructure informations

  45. Definition of LSF deployment, … Globus Mapping Virtual Nodes: example (2) • <processDefinition id=" clusterProcess"> • <bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" • hostname=”cluster.inria.fr"> • <processReference refid=”singleJVM"/> • <bsubOption> • <processor>12</processor> • </bsubOption> • </bsubProcess> • </processDefinition> Infrastructure informations

  46. XML Deployment (Not in source) VNa VNb VNc = VN(a,b) C C A B A B Separate or Co-allocation

  47. IC2D: Interactive Control and Debugging of Distribution

  48. Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of com- munications

  49. Application

  50. A Parallel Object-Oriented Application for 3D Electromagnetism • Visualize the radar reflection of a plane, medicine on head, etc. • Pre-existing Fortran MPI version: EM3D • Jem3D: • Sequential object-oriented design, modular and extensible (in Java) • Sequential version can be smoothly distributed: --> keeping structuring and object abstractions • Efficient distributed version, large domains, Grid environment