Army High Performance Computing Research Center

1. Army High Performance Computing Research Center Overview of Research Activities Vipin Kumar Director, AHPCRC

2. AHPCRC • High performance computing (HPC) plays a key role in bringing about a transformation from the heavily mechanized, slow to deploy forces, to an objective force. • Mission: • Leading edge HPC research in support of the Army’s science and technology goals for the Objective Force and Army Transformation. • Transfer of technology to the Army via direct interactions with Army scientists and through an Infrastructure Support program. • Education and outreach programs that include the Summer Institute for undergraduate students and a series of international/national workshops and conferences. • Forging of synergistic relationships among partner institutions via strong, collaborative research. Overview of AHPCRC Research Activities

3. AHPCRC Team Clark Atlanta University (CAU) Florida Agricultural and Mechanical University (FAMU) Howard University (HU) Jackson State University (JSU) University of Minnesota (UM) University of North Dakota (UND) NetworkCS (NCS) Overview of AHPCRC Research Activities

4. AHPCRC – Current Research

5. Research Program Overview of AHPCRC Research Activities

6. Objectives: Develop HPC modeling and analysis techniques for low observable technologies to obtain an improved understanding of the entire spectrum of signatures of combat systems in a realistic battlefield environment. Computational modeling of radio frequency (RF) and infrared (IR) signatures, and the effects of coatings on these signatures. Understanding the effect of atmospheric on gathering and detecting acoustic and near infrared (IR) signatures. Portfolio: ELECTROMAGNETIC SIGNATURE MODELING IN THE SYNTHETIC BATTLEFIELD Overview of AHPCRC Research Activities

7. Radar Cross Section (RCS) Modeling of Multilayered Material Structures/Coatings Objectives Determine the radar signatures of military hardware, and the use of various methods to control them, including the shaping of the geometry to reduce electromagnetic signatures, and the use of coatings or shields which are absorptive or which scatter incoming signals to reduce these returns. Methodology The program is focusing on the formulation of a boundary integral approach for the solution of 3-D electromagnetic scattering and on the development of a corresponding code. Overview of AHPCRC Research Activities

8. Extend current generalized response function framework to multiple dimensions Theoretical framework Testing and Validation Develop variationally-based filter Theoretical framework Evaluation Develop software that applies the new filter Initial (preliminary) incorporation of the new filter into Local Analysis and Prediction System (LAPS) Atmospheric Effects on Signature Modeling Overview of AHPCRC Research Activities

9. Objectives: Multi-disciplinary HPC modeling and simulations technology for projectile-armor/anti-armor target interaction (including blast-target interaction) in support of Future Combat Systems (FCS). Computational fluid dynamics and aerothermodynamics for external and internal flows in anti-tank projectiles and missiles. Structural dynamics, contact-impact and penetration. Energetic materials. Portfolio: PROJECTILE TARGET INTERACTION Overview of AHPCRC Research Activities

10. Monte-Carlo based simulation and classical molecular dynamics (MD) approaches to characterize the evolution of surface to volume ratio and properties of energetic nanoparticles. Model includes nanoparticle coagulation and finite coalescence, as well as energy release and non-isothermal effects resulting from coalescence. Surface energy and kinetic rates for caolescence determined from MD calculations. Objective Computational Approach Synthesis and Properties of NanoEnergetic Materials Develop a hierarchical computational approach to characterize the thermophysical properties of nanoparticles and their manufacture in areas of interest to the Army, including: -- Predicting the properties of nanoparticles for application in energetic materials -- Predicting the morphology and architecture of nanoparticles grown from vapor -- Augmenting the experimental program of the Army funded Center for Nanoenergetics Research (CNER) Current Modeling Current Modeling • 20-40 Atoms (quantum mechanics) • 104-105 Atoms (molecular dynamics) • Simple idealized interactions • Limited Spatial and Time scales • Perfect Crystals, 2-Dimensional • Build a model capable of modeling the morphological evolution of aluminum nanoparticles using bulk property data. • Develop a simulation capability for Aluminum nanoparticles using an in-house code or with “Temperature Accelerated Dynamics.” • Conduct detailed comparisons to experimental results whenever feasible. Overview of AHPCRC Research Activities

11. Modeling and Simulation of Nanoparticle Growth inTurbulent Flows • Goal: Develop an analytical and algorithmic framework to facilitate the prediction of nanoparticle formation and growth in turbulent flows. • Zero-dimensional simulations • Laminar flow approximations • Approaches: moment methods, direct simulation, and discrete / sectional methods Instantaneous concentration of 4nm diameter particles Turbulence/transport effects significant downstream Overview of AHPCRC Research Activities

12. Numerical Simulation of Turbulent, High-Speed Flows • Goal: Develop "subgrid-scale" SGS closures for turbulent reacting flows, and to implement these closures in flows representative of those encountered in projectile-target interactions. • Reynolds-averaged Navier-Stokes Simulation • Computationally affordable • Closures model wide range of length scales • Large Eddy Simulation • Capture “large” scale - Model “small” scale • Computationally affordable? Three-dimensional round jet of propane issuing into air. Non-reacting (hydrodynamic and hydrochemical interactions only) Overview of AHPCRC Research Activities

13. Parallel Software Developments • Critical stages • Scalable parallel graph partitioning – Mapping and load balancing • Sub-domain integration – Domain decomposition techniques • Time integration operators • Scalable computations - arbitrary large problems and processors • Congruent amongst sub-domains • Reduced solution times • Numericalscalability – no degradation in convergence of numerical algorithms. • Non-linear solvers and linear solvers • Parallel - ability to deliver speedups in terascale range • Memory utilization • REduced Complexity In Programming Environment – [RECIPE] paradigm • Few lines of code - unified and integrated implementation, increased functionality, arbitrary increase in order of accuracy of time integrators • Code optimization – reduced development efforts SGI Origin bristled fat hypercube interconnection network Cray T3E 3D bi-directional torus interconnection network Overview of AHPCRC Research Activities

14. Objectives: Develop a virtual computing environment for Future Combat Systems which includes a synthetic battlefield and the creation of a multidisciplinary computing environment for virtual HPC design. Portfolio: VIRTUAL COMPUTING ENVIRONMENT FOR FUTURE COMBAT SYSTEMS Visualization Scene Generation Overview of AHPCRC Research Activities

15. High Performance Geographic Information Systems (GIS) for Battlefield Visualization in Future Combat Systems Maps are as important to soldiers as guns Shooters Network Sensor Network HPGIS National Assets, e.g. Maps Example Usage of Geographic Info. Systems (GIS) in Battlefield : • Rescue of pilots after their planes went down (recently in Kosovo) • Precision targeting e.g. avoid civilian casualities (e.g. friendly embassies) • Logistics of Troop movements, avoid friendly fires Overview of AHPCRC Research Activities

16. RPP is used to create physical prototypes of 3D solids from their digital representations, using a “3D printer” attached to a workstation. Model Acquisition • CAD Software • CT Scans • Laser Scanning • 3D Photography Computer-Aided Process Planning • File repair • Model orientation • Slicing • Support creation Model Building via Layered Manufacturing Postprocessing • Remove supports • Improve finish • Inspect model LAN or Internet Rapid Physical Prototyping RPP “prints” the 3D model as a stack of 2D layers, using a technique called Layered Manufacturing. Goal: Create, on demand, physical scale models of enemy terrains and assets to help mission planners and field commanders develop and evaluate different combat strategies. Overview of AHPCRC Research Activities

17. hpc simulations can be launched by VDG clients to “produce” VDG data information grid/sensor grid/shooter grid national assets Internet sources hpc simulations sensor networks VDG API for data producers data can be pushed or pulled into VDG VDG server VDG server VDG server wired or wireless access VDG API for client access battlefield simulation battlefield visualization VDG user The Virtual Data Grid: An Infrastructure to Support Distributed Data-centric Applications VDG is a persistent data network designed to support military applications both in the field and at the Army labs Key Features: Reliable, Efficient, Security, Heterogeneity ET metacomputing portfolio connections may be intermittent Overview of AHPCRC Research Activities

18. Objectives: An understanding of protein interactions with toxins and pathogens at the atomistic level to help counteract chem-bio threats. Modeling of the adsorption, transport, diffusion, and dispersion of chemical and biological agents within, across, or into a variety of media, i.e., the atmosphere, water, soil, clothing, building materials, and vegetation A201 E199 G118 Bound Sarin G119 H440 E327 F290 F288 Millard . (1999) 38 , 7032-9 et al Biochemistry Portfolio: CHEM-BIO DEFENSE AND ENVIRONMENTAL MODELING Overview of AHPCRC Research Activities

19. FAS2 Human A201 AChE E199 G118 Bound Sarin G119 H440 E327 F290 F288 Kryger . Sussman (2000) D56 , 1385-9 et al Acta Cryst Millard . (1999) 38 , 7032-9 et al Biochemistry Current Modeling Approach uses a combination of experimental and theoretical techniques. The above figures are derived from X-ray crystal structures. The structures are then computationally modeled to develop an understanding of the process by which the nerve agents bind to the enzyme. Computational Approach The target of many nerve agents is the enzyme, acetylcholinesterase (AChE). Molecular dynamics tools are used to study the molecular motions that occur inside the structure of the target, when it reacts with chemical “nerve agents,” or is inhibited by FAS2. Objective Develop effective countermeasures to protect personnel from nerve agents such as sarin or the biological snake toxin, FAS2 Finding Reactive Sites in Proteins Nerve Agent Snake Toxin Nerve Agent Snake Toxin Sarin - AChE Complex FAS2- AChE Complex Sarin - AChE Complex FAS2- AChE Complex Overview of AHPCRC Research Activities

20. GIS-Based High Performance Simulation of Dispersion Approach • Geometry Representation • Compatible Geometry Representation for Automatic Mesh Generation • Accurate Flow Solver • Boundary Condition + Initial Condition Overview of AHPCRC Research Activities

21. CFD Modeling of Contaminant Dispersion in Urban Environments • Governing equations: • Incompressible N-S equations • Boussinesq approximation for r • LES model for turbulent dissipation • Numerical method: • Standard predictor-corrector method • Solve Poisson equation for pressure • Use grid masking for solid objects: • Enforce zero flux through surface • Modify equations within object • Solve pressure equation in full domain • Inexpensive, flexible Overview of AHPCRC Research Activities

22. Mechanics of Colloidal Transport in Contaminant Dispersion Problem Statement: • Contaminants adsorb to colloidal particles (e.g. clay fragments in groundwater, aerosol/droplet inclusions). • Colloids are transported greater distances than contaminants dissolved in the fluid or gas phase. Governing Equations: • Flow is resolved on the length scale of typical pore spaces using 3-D Navier-Stokes equations. • Transport is modeled by the microscale convection-diffusion equation. No tunable parameters. Overview of AHPCRC Research Activities

23. Chem-Bio Applications of MM5 Address the added complications of land-surface and land-cover at finer resolutions Improve model adaptation to time varying land-surface conditions through GIS LULC extraction Identify model enhancements required that involve physics of flow of water and transport of heat within variably-saturated, variably-frozen soils Implement MM5 on T3E with a grid spacing of 1-km & evaluate the results of the surface condition and boundary layer packages Evaluation of model reliability Assess methods to incorporate chemical-biological dispersion models implicitly with MM5 (with CAU) Chemical-Biological Applications of Mesoscale Atmospheric Modeling Overview of AHPCRC Research Activities

24. Objectives: Conduct basic research in enabling technologies and computational algorithms in support of the Army's scientific and technology goals of the Objective Force and Army transformation Data mining algorithms for discovery of useful patterns in massive data sets. Software infrastructure for metacomputing systems. Mesh partitioning / domain decomposition. Basic research on the development of new computational algorithms in support of interdisciplinary computational research. Portfolio: ENABLING TECHNOLOGY AND COMPUTATIONAL ALGORITHMS Overview of AHPCRC Research Activities

25. Data Mining for Homeland Defense Objectives: • Information fusion from diverse data sources including intelligence, agencies, law enforcement, profile … • Data mining on this information base to uncover latent models and patterns Overview of AHPCRC Research Activities

26. Data Mining for Network Intrusion Detection Objective:Develop techniques to detect and identify attacks against computers, networks, and the information stored therein. • Misuse Detection - Predictive models • Mining needle in a haystack – models must be able to handle skewed class distribution, i.e., class of interest is much smaller than other classes. • Learning from data streams - intrusions are sequences of events • Anomaly and Outlier Detection • Able to detect novel attacks through outlier detection schemes • Detect deviations from “normal” behavior as anomalies Overview of AHPCRC Research Activities

27. Scheduling and Resource Management in Metacomputing Systems • Metacomputing /Grid technology is a fundamental building block for distributed HPC • interconnect disparate resources, data sources, and computations • virtualization of computing • high performance opportunities • single applications: remote supercomputing, resource aggregation • multiple applications: high throughput • Scheduling is needed to realize this HPC potential Overview of AHPCRC Research Activities

28. Technology Transfer - Collaboration with Army Scientists • AHPCRC has a strong record of joint work with Army scientists and engineers, including numerous scientific papers, workshops, reciprocal short- and long-term visits and software development: • State-of-the-art software developed by AHPCRC researchers has been incorporated into many Army and DoD codes. Such transfers of software are the result of significant interaction with Army scientists. • Many students formerly supported by AHPCRC research programs are now employed as scientists at Army laboratories and various DoD organizations • The Infrastructure Support staff plays an instrumental role in technology transfer, by collaborating in the development of software and applying the latest software developments to specific Army applications. Overview of AHPCRC Research Activities

29. Material Deformations Technology Transfer - Software • Software and tools developed at AHPCRC are being used at the Army and throughout the DoD. • Geometric modeling • Automatic mesh generation • Visualization • Mesh partitioning • Sparse linear systems solvers • Process modeling and simulation • Structural dynamics • Example:METIS and ParMETIS graph partitioning libraries are used world-wide for partitioning unstructured and adaptive graphs and have been incorporated in numerous DoD codes such as CTH/PCTH, Paradyn, ParaAble, ADH, MPMC-NET, ICM-TOXI, and FEMWATER. Overview of AHPCRC Research Activities

30. Summer Institutes are conducted to encourage students, particularly minority and female students, to pursue studies and careers in HPC and defense-critical technology areas: Over 200 students since 1991 Emphasize recruiting from partner schools High success rate Majority of them have entered Master and Ph.D. programs (many at AHPCRC partner institutions) Many placed in internships or have pursued technical careers (ARL, Cray Research, Dupont, Intel Corp, GTE, GM, CEWES, NASA, Dept. of Energy, Lucent Technologies, Dow Chemical, Cigna, Argonne National Laboratory, LLNL, 3M, IBM, TRW, Citibank, Oracle Corp., Lockheed Martin, Konica, Digi International, Diversified Pharmaceuticals) 2 people founded successful companies Summer Institute Overview of AHPCRC Research Activities

31. Outreach - Workshops and Conferences • Provide a forum for the exchange of information among AHPCRC researchers, Army collaborators and other interested DoD researchers: • ARL Intrusion Detection Systems Research WorkshopMarch 19-20, 2002 Higher Education and Applied Technology (HEAT) Center, Aberdeen, Maryland • Computational Electromagnetics (CEM) WorkshopJun 27-28, 2002 Sheraton College Park, Adelphi, Maryland • CFD / CSM for Projectile's Aerodynamics and Propulsion,Aug 13-14, 2002 Clark Atlanta University, Atlanta, GA • Contact Impact Modeling and SimulationSept. 2 (tentative) Minneapolis, Minnesota • Mesoscale Data Integration WorkshopSep 9-10, 2002 University of North Dakota, Grand Forks, ND Overview of AHPCRC Research Activities

32. Outreach - Workshops and Conferences • 10th Conference on Computational Chemistry (JSU, November 2001) Attended by two Nobel Laureates Banquet speaker: Mr. W. Hollis • Second SIAM International Conference on Data Mining(April 11-13, 2002 – Hyatt Regency, Crystal City) • International Conference Series as a Follow-up of the Workshop on Mining Scientific and Engineering Datasets (held at the AHPCRC, 1999,200) Overview of AHPCRC Research Activities

33. Synergistic Relationships Among Partners • Projectile Target Interaction • UM, CAU, FAMU, HU, NCS • Signature Modeling • HU, UND, UM, JSU, NCS • Chem-Bio Defense • CAU, JSU, FAMU, UND, UM, NCS • ET & Computational Algorithms • FAMU, HU, UM • VCE • CAU, UM, NCS Overview of AHPCRC Research Activities

34. Synergistic Relationships Example: Computational Chemistry • Goal is to predict chemical properties of interest. • Sensitivity of energetic materials • Properties of stealth coatings • Conformational and reactive site variations in proteins UM: Geometric modeling and data mining JSU: Computational chemistry and biology FAMU: Energetic materials NCS: Computational chemistry Overview of AHPCRC Research Activities