160 likes | 388 Vues
Matthias Vodel Wolfram Hardt vodel@cs.tu-chemnitz.de { hardt }@cs.tu-chemnitz.de. Content. Motivation MPI2Java Concept Performance Analysis SimANet Application Scenarios. APCC 2010 Matthias Vodel , University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de - 2 -.
E N D
Matthias Vodel Wolfram Hardt vodel@cs.tu-chemnitz.de{ hardt }@cs.tu-chemnitz.de Parallel High-Performance Applications with MPI2Java - A Capable Java Interface for MPI 2.0 Libraries -
Content • Motivation • MPI2Java Concept • Performance Analysis • SimANet Application Scenarios APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -2-
High Performance Computing Motivation• Concept • Analysis • SimANet • High complex computing challenges in the field of: • Mathematics • Physics • Chemistry • Engineering • Informatics • Weather forecast Massive parallel computing power State of the art hardware Modern implementation languages APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -3-
MPI Standard Motivation• Concept • Analysis • SimANet • Message Passing Interface • Communication protocol for distributed • computing environments • MPI specifies communication features • Implemented in libraries • Usage by an defined API De facto standard for computer cluster programming MPICH represents a implementation of the MPI specification Performance-optimised libraries for C and Fortran Missing up-to-date implementation for modern languages (object-orientation, template-based design patterns) APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -4-
MPI Support for Java Motivation• Concept • Analysis • SimANet + platform-independent + no C-libraries required + minimal maintenance costs Pure Java Implementations JNI Implementations + usage of related, performance-optimised libraries + less implementation work + significant faster • complete reimplementation of the entire MPI specification • performance problems (higher abstraction level, minimal optimisations) • maintenance to keep the C-libraries • up-to-date • Installation on the target system hardware is more complex (library binaries not platform-independent) APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -5-
MPI Support for Java Motivation• Concept • Analysis • SimANet Pure Java Implementations JNI Implementations APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -6-
Why implementing anew MPI-Java interface?? Motivation• Concept • Analysis • SimANet Since 2003: no further development of existing projects JMPI significant slower than mpiJava mpiJava – bad exception / error handling – missing 64bit support actual cluster computer environment not supported! All Java-MPI libraries are not able to use MPI-2 features APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -7-
MPI2Java - Structure Motivation• Concept • Analysis • SimANet APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -8-
MPI2Java - Interface Motivation• Concept • Analysis • SimANet = in Java implementiert = in C implementiert MPI.java libMPI.c public class MPI { public static String[] Init(...) { ... nativeInit(...); ... } } #include <mpi.h> jobjectArraynativeInit(...) { ... MPI_Init(...); ... } MpiTest.java ... MPI.Init(...); ... MPI.COMM_WORLD.Send(...); ... Comm.java libComm.c public class Comm { public Status Send(...) { ... nativeSend(...); ... } } #include <mpi.h> jobject nativeSend(...) { ... MPI_Send(...); ... } APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -9-
PC Workstation Environment Motivation• Concept • Analysis • SimANet PC workstation - 3Ghz Pentium 4, 2GB RAM, 100Mbit/s fast Ethernet - Scientific Linux 4.4, 2.6.18 Kernel Throughput measurements Pingpong measurements APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -10-
CHiC Environment Motivation• Concept • Analysis • SimANet CHiC - Chemnitz High Performance Computing Cluster - 512 AMD Opteron 64bit cores, 2.6GHz, each 4GB, fiber optics Infiniband network infrastructure - Scientific Linux 4.4, 2.6.18 Kernel Throughput measurements Pingpong measurements APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -11-
Evaluation - SimANet Motivation• Concept • Analysis • SimANet • Simulation Platform for Ambient Networking • Java-based, complex simulation framework for heterogeneous MANETs / WSNs • Supports distributed computing environments • Optimal testbed for Java-MPI parallelisation scenarios APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -12-
Test Scenarios I Motivation• Concept • Analysis • SimANet • SimANet @ CHiC; up to 64 Nodes • Static node topology 100.000 nodes 5000 nodes APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -13-
Test Scenarios II Motivation• Concept • Analysis • SimANet • SimANet @ CHiC; up to 64 Nodes • Dynamic node topology JMPI MPI2Java APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -14-
Thank you … … for your attention. APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -15-
Difference between MPI 1.0 & 2.0 Motivation• Concept • Analysis • SimANet MPI-1 - Point-to-Point operations - Collective operations - Synchronisation - Process groups … MPI-2 - Parallel I/O - Dynamic process management - Remote Memory Access (RMA) During the communication process, each command has to be called & executed in every communication node One communication node is able to initialise a communication process APCC 2010 Matthias Vodel, University of Chemnitz, Germany, vodel@cs.tu-chemnitz.de -16-