On MDE to implement a Component Assembly Compiler for HPC

1. On MDE to implement a Component Assembly Compiler for HPC Journées IDM, June the 7th 2011 Julien Bigot*, Christian Pérez* Laboratoire de l’Informatique du Parallelisme (LIP) ENS Lyon

2. Scientific applications Domain specialist code Long life cycle (~30 years) Computation intensive Hardware Heterogeneous Supercomputers Computing clusters Computing grids Clouds Parallel Short life cycle (~3 years) Programming model Low level, fine-tuning of code Hand made … and remade Context: High Performance Computing ? programming models Structural Mechanics Optics Dynamics Thermal Satellite modeling SAN SAN Cluster Cluster LAN WAN Supercomputer Supercomputer Grid

3. Outline • Context: High Performance Computing (HPC) • High Level Component Model (HLCM) Modeling and Implementation • Feedback on the Model Driven Approach • Conclusion

4. Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components

5. Components black boxes Interactions through well defined interaction points Application Set of interconnected instances High level view of architecture Separation of concern Domain code: components Support: component model Eased reuse Well defined interface & behavior Software components d a b CmpA CmpB c c1 c2

6. Software Component Models • Industrial Component Models (CORBA Component Model, Enterprise JavaBeans, …) • Transparent distribution support (CORBA, JavaRMI, …) • Overhead for same host components • Research Component Models (Fractal, …) • Interesting features (Hierarchy, Connectors, …) • Features handled at runtime, additional costs • High Performance Component Models (CCA, …) • Acceptable overheads • Low level of abstraction => Hardware adaptation complex

7. Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Component implementation choice

8. Distinction type/implementation Type: name + ports Implementation: behavior Choice Depending on available hardware Multiple alternative implementations CmpA I2 I1

9. Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Component implementation choice 1st class Software Connectors with implementation choice

10. 1st class entity A Name Set of roles Multiple alternative implementations Hardware optimization possible Interaction code well identified Multiple versions possible More application specific interactions 1st class software connectors CmpA CmpB ports CmpB CmpA UP user provider roles

11. Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Algorithmic skeletons, generic programming & hierarchy Component implementation choice 1st class Software Connectors with implementation choice

12. Generic programming & hierarchy:Algorithmic skeletons Farm<W, I, O, N> Farm W I O W Disp<I> Coll<O> . . . N times W

13. Programming model requirements • Ease of development • Domain specialist: Domain code / support code separation • Long life cycle: clean code behavior and interface specification • High Performance • Minimal overhead • Support for parallelism • Hardware evolution • Algorithm optimization • Interaction optimization • E.g. distributed memory vs. shared memory Software Components Compilation based approach Algorithmic skeletons, generic programming & hierarchy Component implementation choice 1st class Software Connectors with implementation choice No need for new components, only a new assembly model

14. HLCM Compilation • Reuse an existing component model (L²C) L²C: • Low Level • Primitive components only • Various interaction • C++ • MPI • CORBA • … PIM PSM HLCM: • Genericity • Hierarchy • Connectors • Multi-Implem MDA Hardware description PDM

15. HCLM (Meta)Modeling • Using the Eclipse Modeling Framework (EMF) • PIM • Main classes: ComponentType, PortType, Connector, … • Primitive implementations: abstract • Some patterns identified: genericity, implementation choice • 130 Ecore metaclasses (490 lines of EMFatic) • 2000 utility Java lines • PSM • Main classes: ComponentInstance, Connexion, … • References to the PIM in each instance • 41 Ecore metaclasses (160 lines of EMFatic) • 800 utility Java lines

16. HCLM Transformation • Four step transformation • Parsing HLCM: 400 lines Xtext annotated grammar • Compilation: 4000 lines plain Java • PSM to Backend • Backend Model Dump • Compilation • Choosing implementations (delegated to an external “oracle”) • Concretizing generic elements • Flattening composites • Until convergence: only primitive elements • Final dump • Often single hand made step Backend specific

17. Feedback (1/2) • Modeling tools: huge gain of time • Modeling HLCM • Initial attempt with plain Java: ~1 month work • Second attempt with Ecore: ~2 days work • Direct concrete syntax to model parsing (Xtext) • Ease of comprehension • Most concepts intuitive with UML background • Various level of modeling confusing (model vs. metamodel) • E.g. connectors • Fixed set of interactions: various connectors in the metamodel • 1st class connectors: various connectors in the model • Combination predefined connectors + user defined : ??? • E.g. genericity: manipulation of metamodel elements in the model About same state

18. Feedback (2/2) • Model transformation • Transformation tools not ready 2 years ago (Kermeta , QVT, ATL) • Plain java transformation • API not human friendly • Collection access => whole collection traversal • Omnipresent type casts (constraint not expressible in Ecore) • Error complex to associate with user error • Constraint language evaluation required • Real use • Applications: Parallel method calls, shared memory, Domain decomposition • Backends: L²C , Pure Java, CCM, Charm++, … • Transformation speed: 1500+ components, ≤ 10secs

19. Conclusion • HLCM: a component assembly model for HPC • Hierarchy, Generic Programming, Connectors, Alternative Implementations • Efficiency possible on a wide range of hardware • A model based implementation • Lot of time gained • Some tools not used due to immaturity • Usable in real world • Perspectives / future work • Re-evaluate modeling tools (ATL, OCL, …) • Introduce the time dimension in HLCM • Models@runtime ?