html5
1 / 27

D istributed I nteractive C omputing E nvironment

D istributed I nteractive C omputing E nvironment. http:// www.arl.hpc.mil/SciVis/dice. Platforms. Challenge : Stovepiped Applications. HPC Codes. CFD Code 1. CSM Code 1. CCM Code 2. CCM Code 1. Preprocessing. Runtime Analysis. Tools. Postprocessing. Additional Challenges.

lanai
Télécharger la présentation

D istributed I nteractive C omputing E nvironment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Interactive Computing Environment http://www.arl.hpc.mil/SciVis/dice

  2. Platforms Challenge : Stovepiped Applications HPC Codes CFD Code 1 CSM Code 1 CCM Code 2 CCM Code 1 Preprocessing Runtime Analysis Tools Postprocessing

  3. Additional Challenges • Code Setup can be Difficult • Enormous Amounts of Data • Heterogeneous Environment • Security

  4. Distributed Interactive Computing Environment Graphical User Interface and Runtime Visualization Common Distributed Data Model and Format High Performance Computing Resource User’s Workstation

  5. Fully Integrated FMD GUI

  6. Fully Integrated ZnsFlow GUI

  7. Fully Integrated CTH GUI

  8. What is DICE ? A Distributed Computing Environment Composed of Configurable Mega-Components • Heterogeneous DSM * • Data Organization • Graphical User Interface • Visualization User-Ready Computing Environments Customized for Specific Technology Areas

  9. Network Distributed Global Memory Clients VectorPut VectorGet Put Get Barrier Semaphore 0 10M 30M 50M Servers Process Local Address Space Disk File System Shared Memory Physical Data Storage

  10. Light Data Heavy Data • 3D Structured Mesh • 300 x 200 x 300 • Number of Materials • Material Names • Total Residual Mass • X, Y, Z Values • Material Volume Fractions • Cell Temperatures

  11. User Application Scripting and Graphical User Interface Tools Visual Generators Plots Planes Surfaces High Performance Computing Code Visualization Toolkit (vtk) DICE Object Directory Object Oriented Support for Data Model, XML, ... Hierarchical Data Format ( HDF5 ) Data Format : Number Type, Data Space ... OpenGL Network Distributed Global Memory ( NDGM ) with GSSAPI Mesa File I/O Process Control Shared Memory Facility TCP/IP Sockets Operating System Distributed Resources

  12. DICE Data Model and Format Convenience Layer Structured Data Unstructured Data Groups DICE Object Directory Object Oriented Support for Data Model, XML, ... Hierarchical Data Format ( HDF5 ) Data Format : Number Type, Data Space ... FileSystem Serial and Parallel Global Access to Secondary Storage ( Globus - GASS ) Network Distributed Global Memorywith GSSAPI Other ... Distributed Buffer of Bytes

  13. Hierarchical Data Format Version 5 Group Attribute Group Group DataSet Attribute Not Allowed Group DataSet • Host Independent Data Format • Access via API only • Physical Storage thru Drivers • Includes Data Compression • Includes SubSelections Group DataSet DataSet Attribute Attribute Attribute

  14. eXtensible Markup Language • Similar to HTML Format • Human Readable • Pervasive on the WEB • Strict Hierarchy with No Inheritance • Many Parsing Tools Already Exist • < Parent Name=“Top”> • < Child Name=“Left”> • Character Data for Left • < /Child > • <Child Name=“Right” > • Character Data for Right • < /Child > • < /Parent > Parent Child Character Data for Left Child Character Data for Right Parse XML Document Object Model In-Memory Data Structures Serialize

  15. N 5 3 3 5 4 4 3 1 N 4 3 N 3 1 2 2 2 1 2 2 1 1 4 Polyvertex Polyline Triangle Quadrilateral Polygon 6 4 5 5 2 3 3 4 1 3 1 4 1 2 2 Tetrahedron Pyramid Wedge 8 7 5 6 3 j j 4 k 1 2 i i Hexahedron 2DSMesh 3DSMesh DICE Object Directory Base Topologies

  16. Light Data • <Topology Type=“Tetrahedra” NumElements=“3”> • </Topology> • <Geometry> • <Vertices> • <Index Location=“XML” Type=“List”> • 1000 1001 21001 21000 …… • </Index> • <Points Type=“XYZ” Location=“HDF” > • NDGM:File1.h5:/Grid1/Geometry/MyPoints • </Points> • </Vertices> • </Geometry> • <Topology Type=“3DSmesh” • Dimensions=“10 20 30”> • </Topology> • <Geometry> • <Points Type=“XYZ” Location=“HDF” > • NDGM:File1.h5:/Grid1/Geometry/MyPoints • <Region Type=“Slab” Location=“XML” > • 1 2 10 2 3 20 3 4 30 • </Region> • </Points> • </Geometry> 100 x 200 x 300 HPC Code HDF5 . . . . NDGM GASS MPI-I/O

  17. Complexity of Model Defined in Light Data Domain Domain Grid XML Grid Attribute NDGM:Pressure.h5 Attribute NDGM:Temperature.h5 Topology 100 Hexahedra Geometry XYZPoints.h5 HPC Code HDF5 HDF5 File NDGM GASS HPC Code I/O is Natural and Efficient

  18. XML Contains Parameters for Variable Substitution Domain Domain Grid XML Grid • Parameter TimeStep = 0 to 99 • Attribute • NDGM:&TimeStep;/Pressure.h5 Topology 100 Hexahedra Geometry XYZPoints.h5 HDF5 HDF5 for TimeStep 0 HPC Code HDF5 for TimeStep 1 HDF5 for TimeStep 2

  19. Parallel / Distributed Entities Post-Processing Tools Pre-Processing Tools Runtime Tools HPC Codes Convenience Layer eXtensible Markup Language ( XML ) Hierarchical Data Format Verision 5 (HDF5 ) Distributed Data Hub

  20. DICE Data Reader for EnSight Data Reader uses DICE Object Directory API EnSight DICE Object Directory

  21. DICE Visualization using vtk HDF5 HDF5 DICE Object Directory File NDGM GASS Visualization “Subnet” Source Filter Filter Filter Polygons

  22. Adding Performance Friendly Functionality System Programming Language C C++ Fortran77 Fortran90 etc. SWIG Compiled Object Tcl Wrapper } New Scripting Command via Dynamically Loadable Library Perl Wrapper Python Wrapper Java Wrapper

  23. C++ DiceFloat64Array MyData, NewData; DiceInt64Array MyIndex; MyData.SetNumberOfElements( 10000 ); MyData.Generate(0.0, .9999 ); MyIndex.SetNumberOfElements( 100 ); MyIndex.Generate( 0, 99 ); MyIndex *= 2; NewData = MyData[ MyIndex ]; Allocate 10,000 Doubles Set Individual Values Array Operation +, -, *, / Array Indexing Scripting ( Tcl, Perl, etc. ) DiceHDF H5Object H5Object Open NDGM:FileName.h5:/DataSetName DiceMemory MemoryObject MemoryObject SetArray [ DiceFloat32Array ] H5Object Read [ MemoryObject cget -this ] set NewData [ MemoryObject GetArray ] DiceExpr $NewData[ where( $NewData > .5 ) ] = .5 Object Oriented Scripting Interface Open HDF5 File, Query Number Type and Shape Request Automatic Conversion to Float32 MemoryObject has Number Type and Shape Clip Values to .5

  24. DICE Web Interface Under Construction DICE on HPC Resource using Off Screen Rendering SSL Kerberos kinit • Every URL Request is Intercepted to Verify Session Id • DICE Runs as a User Process • HTML FORMS result in Tcl Procedure Invocation • Graphics Updated via Server-Push ( Netscape only ) or Client-Pull • Intended for Limited Interfaces from Remote ( Slow Connection ) or Graphically Challenged Client Machines.

  25. MPI Finite Element MPI Finite Volume Visualization Data Mining Steering Coupling HDF5 NDGM

  26. “Gently Coupling” Codes from Separate Disciplines Distributed Data Hub Finite Volume Structural Mechanics Small Scale Analysis Global Monitor Injection Flow Analysis Runtime Visualization

  27. COTS, GOTS and “Semi-Automatic” Visualization DICE and the HPC Environment X / OpenGL DICE User Interface XML and Http ( SSL + Kerberos ) DICE Server ( User Process ) DICE Object Directory XML, HDF5 I/O One or More Cooperating HPC Codes

More Related