Fabrice Huet inria.fr/oasis/ProActive OASIS Team INRIA - CNRS - I3S

1. ProActive: Distributed and Mobile Objects for the GRID Fabrice Huet www.inria.fr/oasis/ProActive OASIS Team INRIA - CNRS - I3S

2. ProActive: A Java API + Tools for Parallel, Distributed Computing • A uniform framework: An Active Object pattern • A formal model behind: Prop. Determinism, insensitivity to deploy. • Main features: • Remotely accessible Objects • Asynchronous Communications with synchro: automatic Futures • Group Communications, Migration (mobile computations) • XML Deployment Descriptors • Interfaced with various protocols: rsh,ssh,LSF,Globus,Jini,RMIregistry • Visualization and monitoring: IC2D • In the www.ObjectWeb.org Consortium (Open Source middleware) • since April 2002 (LGPL license)

3. model • Active objects : coarse-grained structuring entities (subsystems) • Each active object: - possibly owns many passive objects • - has exactly one thread. • No shared passive objects -- Parameters are passed by deep-copy • Asynchronous Communication (with RDV) between active objects • Future objects and wait-by-necessity. • Full control to serve incoming requests

4. 3 Active object • An active object is composed of several objects : • The object itself (1) • The body: handles synchronization and the service of requests (2) • The queue of pending requests (3) Standard object 1 Object Active object Proxy Object 1 2 Body

5. A call on an active object consists in 2 steps A query : name of the method, parameters… A Reply : the result of the method call A query returns a Future object which is a placeholder for the result The callee will update the Future when the result is available The caller can continue its execution event if the Future has not been updated Wait by necessity foo () { AnyObject r = a.process(); //do other things ... r.toString(); } AnyObject process() { //perform long //calculation return result; } will block if not available

6. Intra-object synchronization class BoundedBuffer extends FixedBuffer implements runActive { runActivity (Body myBody) { while (...) { if (this.isFull()) myBody.serveOldest("get"); else if (this.isEmpty()) myBody.serveOldest ("put"); else myBody.serveOldest (); // Non-active wait myBody.waitARequest (); }}} • Explicit control: • Library of service routines: • Non-blocking services,... • serveOldest (); • serveOldest (f); • Blocking services, timed, etc. • serveOldestBl (); • serveOldestTm (ms); • Waiting primitives • waitARequest(); • etc. Implicit (declarative) control: library classes e.g. : myBody.forbid ("put", "isFull");

7. ProActive MOP ProActive architecture: a simple MOP • MOP (Meta-Object Protocol) • Runtime reflection (for dynamicity) • New semantics for method and constructor calls • Uses the Java Reflection API • ProActive • Implemented on top of the MOP • Other models can be built on top of ProActive or on top of the MOP

8. Static or dynamic (BCEL) generation of stubs: Take place of a reified object Reification of all method calls Sub-class of the reified object: type compatible Stub only depends on the reified object type, not on the proxy Any call will trigger the creation of an object Call that represents the invocation. It will be passed to the proxy: has the semantics to achieve MOP principles Reified object Reified object Stub Proxy

9. Request Receiver Reply Receiver Service Reply Sender Meta-Objects for DistributionAn Active Object Body FuturePool RequestLine Object

10. Active Object Architecture • Multiples Objets • RequestReceiver: Receive the requests • ReplySender: Send back the result to the caller • ReplyReceiver: Receive the future updates • Service: Choose (select) and executes the requests • RequestLine: Pending Requests • FuturePool: Pending Futures

11. Reply Sender Request Receiver Reply Receiver Service Caller Body 1 1 - Call 2 2 - Reception of Request 3 - Selection of Request 5 4 - Execution 5 - Sending back the reply 4 Object 3 Request to an Active Object

12. Group Communications • Method Call on a typed group • A call on a group will give a group of results • Based on the ProActive communication mechanism • Replication of N ‘ single ’ communications • preserve the « rendez-vous »

13. V Construction of a Result Group A ag = newGroup (…) V v = ag.foo(param); v.bar(); A Typed Group Java or Active Object

14. Group as Result of Group Communication • Dynamicaly built and updated • B groupB = groupA.foo(); • Ranking oder property • Synchronization primitive • ProActiveGroup.waitOne(groupB); • ProActiveGroup.waitAll(groupB); • ... waitForAll, ... • Predicates: • noneArrived, kArrived, allArrived, ...

15. Collective Operations : Example • // Build a group of « A » • A ag = (A)ProActiveGroup.newGroup(« A », {a1,a2,b1}) • V v = ag.foo(param); // foo(param) invoked // on each member • // A group v of result of type V is created • class A {… • V foo(P p){...} • } • class B extends A { ...} • A a1=PA.newAct(A,); • A a2=PA.newAct(A,); • B b1=PA.newAct(B,); • ProActiveGroup.waitForAll(v); //bloking -> all • v.bar();//Group call • v.bar(); • // starts bar() on each member of the result group upon arrival V vi = ProActiveGroup.getOne(v); //bloking -> one vi.bar(); //a single call

16. Two Representation Scheme Management of the group getGroup method of class Group Group of objects gA Typed group groupA getGroupByType static method of class ProActive Fonctional use of the group

17. Two Representations (2) • Management operations add, remove,size, … • 2 representations of a same group : Typed Group / Group of objects • ability to switch between those 2 representations • Group gA = ProActiveGroup.getGroup(groupA); • gA.add(new A()); • gA.add(new B()); //B herits from A • A groupA = (A) gA.getGroupeByType();

18. A A C 1 C 1 2 C C 1 2 1 2 2 1 2 Broadcast or Scatter for Parameters • Broadcast is the default behavior • Scatter is also possible • groupA.bar(groupC); // broadcast groupC • ProActiveGroup.setScatterGroup(groupC); • groupA.bar(groupC); // scatter groupC scatter broadcast

19. 1 inter network communication 2 inter network communications Hierarchical Groups • Add a typed group into a typed group • Benefit of the network structure A A Net2 A Net1 Net2 Net1

20. Other features for Groups • Optimization • Parallel calls within a group (latency hiding) • Treatment in the result order (if needed) • Scatter (a group as a parameter to be dispatched in Group. Com.) • A single serialization of parameters • Active Group • A group becomes remotely accessible so: updatable and consistent ---> migration ---> Dynamic changes

21. Abstract Deployment Model • Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. • Abstract Away from source code: • Machines names • Creation/Connection Protocols • Lookup and Registry Protocols • Interfaced with various protocols and Infrastructures: • Cluster: LSF, PBS, SGE , OAR and PRUN(custom protocols) • Internet and LAN: rsh, rlogin, ssh • Grid: Globus, Web services, gsissh • API to query resources from within the application through symbolic names (VirtualNode)

22. Descriptors: based on Virtual Nodes • Virtual Node (VN): • Identified as a string name • Used in program source • Configured (mapped) in an XML descriptor file --> Nodes • Operations specified in descriptors: • Mapping of VN to JVMs (leads to Node in a JVM on Host) • Register or Lookup VNs • Create or Acquire JVMs • Program Source Descriptor (RunTime) • |----------------------------------| |-------------------------------------------| • Activities (AO) --> VN VN --> JVMs --> Hosts

23. VM1 VM2 jem3dNode VirtualNode: jem3dNode Mapping: jem3dNode  VM1 Infrastructure: VM1  SSH frontend then pbsProcess pbsProcess  PBS 10 nodes Local VM VM VM VM localhost frontend VM VM VM App. VM VM VM VM jem3dNode Deployment Descriptor- Examples VirtualNode: jem3dNode Mapping: jem3dNode  VM1, VM2 Infrastructure: VM1  Local VM VM2  SSH host1 thenLocal VM localhost App. host1

24. Virtual Nodes in Programs • 1. Load the descriptor file • Descriptor pad = ProActive.getDescriptor ("file://ProActiveDescriptor.xml"); • 2.Activate the mapping • VirtualNode vn = pad.activateMapping ("Dispatcher"); // Triggers the JVMs • 3.Use the node • Node node = vn.getNode(); • ... • C3D c3d = ProActive.newActive("C3D", param, node); • log ( ... "created at: " + node.name() + node.JVM() + node.host() );

25. Deploying on a Grid • Co-allocation issues: • Book computers on multiple cluster • No central reservation, so issue n requests • Only m ≤ n will be fulfilled • Software issues: • Local services (nfs, dns…) might be down on a particular node • Rogue processes from previous users might prevent your application from starting • It is very hard to predict how many functional nodes will be available when the application starts • Give as much information/control as possible to the application

26. Example of application controlled deployment • Start the application with • Various parameters • Deployment file (optimistic number of nodes) • Inside the initialization loop of the application • Start the deployment • Wait for either all available nodes or a timeout • Decide what to do with available resources • Cancel execution • Reduce problem size if not enough nodes available • Use only necessary nodes for best performance • Very simple to write

27. Jem3D • Solves 3D Maxwell equations • Relies on finite volume approximation method • Works on unstructured tetrahedral discretization of the computation domain • The complexity of the computation is linked to the number of tetrahedra • Test case for performance: propagation of an eigenmode of a cubic metallic cavity

28. Computation intensive Communication intensive Jem3D (2) Initialization Compute mag. field, and update elec. field Compute elec. field, and update mag. field Calculation of the discrete electromagnetic energy t < tmax t = tmax Solution Saving

29. 1 2 1 1 2 4 Communication Area: 3*N2 Communication Area: 2*N2 Distribution of the ComputationExample on 4 nodes N

30. Running Environment • DAS-2 MultiCluster • 200 Nodes on 5 sites • ≥1GB of memory • We use 72 Nodes on 1 site, other evenly distributed • Intra domain nodes linked by 100Mb/s ethernet • Inter domains linked with 1Gb/s • OS • RedHat 7.2 • Java • Sun’s 1.5.0 and IBM’s 1.4.1

31. Jem3D with 51x51x51 mesh

32. Comparison with a Fortran Implementation • We compare: • The same algorithm • Implemented in very different languages (dynamic data structures in Java, static in Fortran) • By different persons • We want to know how much we pay for the Java features • We measure the time taken to perform 100 iterations of the main loop

33. Sequential Version

34. Ibis • Grid programming environment with efficient communications • Java Implementation for portability • Native Implementation for performance • TCP • Myrinet • MPI • Various programming models: Group Method Invocation (GMI), Divide and Conquer (Satin), Remote Method Invocation (RMI) • We focus here on the Java implementation for portability reasons

35. Improved RMI implementation • Source compatible with Sun’s RMI (replace java.rmi.* with ibis.rmi.*) • Reduced overhead through caching of type information • Overall, reduce by 10% the amount of data sent over the wire • Improved serialization • Generation of serialization code for all classes • Specially generated constructor to create empty objects

36. Speedup, 51x51x51

37. 241x241x241 mesh on Grid5000

38. Jem3D - Conclusion • ProActive + Ibis: • Integrated Solution • Allows a better speedup than plain RMI implementation • Gives to the application a better control on its environment • Still slower (3x) than a “close” Fortran/MPI implementation but • Easier to compile • Easier to deploy (Jem3D ran on 150 nodes) • But have to play cats and mouse