1 / 24

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation. Ning Liu, Christopher Carothers liun2@cs.rpi.edu. Outline. Backgound Torus model, traffic model BG/L Ross: Massively Parallel Simulator Experiment results Future work. Background.

anahid
Télécharger la présentation

Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Modeling Billion-Node Torus Networks Using Massively Parallel Discrete-Event Simulation Ning Liu, Christopher Carothers liun2@cs.rpi.edu

  2. Outline • Backgound • Torus model, traffic model • BG/L • Ross: Massively Parallel Simulator • Experiment results • Future work

  3. Background • CODES: Enabling Co-Design of Multilayer Exascale Storage Architectures • CODES GOAL: Develop a simulation framework for evaluating exascale storage design challenges. • Hardware Models • Storage Software Models • Storage System Architecture • Exascale I/O Workload Models • Simulation Framework - Integrate models and storage software into simulation framework

  4. Torus Network • Blue Gene and Cray XT supercomputer families adopt a 3-D torus • Upcoming Blue Gene/Q will have a 5-D torus network • Provide low latencies and high bandwidth at a moderate cost to construct.

  5. Torus Traffic and Routing • Using Markovian models • Each node continuously generates packets • Select random destination • Packet size fixed • Dynamic routing VS. static routing • Avoid deadlocks • BGL eager/rendezvous protocols

  6. Discrete Event Model • Logic Process: Node • Events • Packet_generate_event • Packet_send_event • Packet_arrival_event • Packet_process_event

  7. Simulation Testbed: BGL • 32-bit IBM PowerPCs running at only 700 MHz • 1 GB memory per node • 1,024 dual processor “node” per rack • 16-rack, 32,768-processor located at Rensselaer’s Computational Center for Nanotechnology Innovations (CCNI) • Confusion? Simulating BGL torus on top of BGL

  8. ROSS: Parallel Simulator • Serial/Conservative/Optimistic Simulation • Using Jefferson’s Time Warp event scheduling mechanism • Reverse Computation

  9. Validation Using Little’s Law Little's Law: the average number of customers in the store, L, is the effective arrival rate, λ, times the average time that a customer spends in the store

  10. Validation Using Little’s Law

  11. Latency Comparison: BGL vs. Simulation • Using MPI Send()/MPI Recv() • Collected data from 1,024-node torus in a 1x32x32 node configuration

  12. Performance Metrics • The performance study examines the impact of processor/core count on four primary metrics: • (i) committed event-rate, • (ii) percentage of remote events, • (iii) efficiency and • (iv) secondary rollbacks.

  13. Million-Node Torus Scalability • Packet injection rate 10 pkt/ms • peak event-rate of 4.78 G/sec

  14. Efficiency

  15. Remote Event Rate • Random destination selection creates a difficult scenario for parallel event scheduling because each packet randomly selects destination

  16. Secondary Rollback Rate

  17. Billion-Node Torus Scalability • consume 2 TB memory • total number of generated packets is O(1011) • total number of events scheduled is O(1013) • Packet injection rate 200 pkt/ns & 400 pkt/ns • higher rollback probability • larger event population leads to increased queuing overheads

  18. Billion-Node Torus Scalability

  19. Billion-Node Torus Scalability

  20. Future work • Application workload models: Application I/O kernel models, I/O characterization models • I/O aggregator node models • I/O network models: network cards, switches, and topologies • I/O storage node models: storage software • I/O storage software: models and prototype system software • I/O controller models: RAID and enterprise storage devices • Disk models: HDDs and SSDs

  21. Future work • Increase the fidelity of torus network model • Dynamic routing • Virtual channels • Different torus traffic model • Tree network model based on Blue Gene families • MPI_Alltoall(), MPI_Bcast(),MPI_Reduce(); • Complex I/O workload drivers, like PHASTA

  22. Related Work • Heidelberger’s use of the YAWNS protocol to model the Blue Gene/L torus network on a per cycle basis appears to be one of the most accurate models created to date. • Min and Ould Khaoua proposed a torus network model based on circuit switching.

  23. Conslusions • near linear speedup for our torus model • peak event-rate on 32K cores is 4.78 G/sec • demonstrated the ability to model a million-node and billion-node torus network on Blue Gene/L • conducted comparison tests between actual Blue Gene torus network and our model using MPI Send()/MPI Recv()

  24. Thank you for your attention! Questions?

More Related