1 / 23

Multilayer Online Monitoring for Hybrid DSM systems on top of PC clusters with a SMiLE

SMiLE &. Multilayer Online Monitoring for Hybrid DSM systems on top of PC clusters with a SMiLE. Wolfgang Karl, Martin Schulz , and Jörg Trinitis Lehrstuhl für Rechnertechnik und Rechnerorganisation, LRR Technische Universität München, Germany

mabli
Télécharger la présentation

Multilayer Online Monitoring for Hybrid DSM systems on top of PC clusters with a SMiLE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SMiLE & Multilayer Online Monitoring for Hybrid DSM systems on top of PC clusters with a SMiLE Wolfgang Karl, Martin Schulz, and Jörg Trinitis Lehrstuhl für Rechnertechnik und Rechnerorganisation, LRR Technische Universität München, Germany 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Motorola University, Schaumburg, Illinois, USA March 29th, 2000

  2. Motivation • Shared Memory is gaining acceptance • Also on loosely coupled architectures • But ... • Performance problems on loosely coupled systems • Implicit communication  „Black-Box“ behavior • Need for efficient monitoring environment • With minimal impact on the running application • Capable of detecting performance bottlenecks • Extensible to any existing shared memory model 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  3. Overview SMiLEHW-DSM Monitor ExtensibleMonitoring API Multi-layeredMonitoringInfrastructure Global VirtualMemory for Clusters 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  4. Component 1: SCI Virtual Memory SMiLEHW-DSM Monitor ExtensibleMonitoring API Multi-layeredMonitoringInfrastructure Global VirtualMemory for Clusters 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  5. Using Shared Memory on Clusters • Why Shared Memory? • Easier programming paradigm • Close to sequential model, hence easier porting • Challenges on Clusters: • By default no hardware support • Local OS instances • Traditional solutions: • Software DSM systems • Drawbacks: Inefficiencies due to e.g. False Sharing 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  6. User-level communication Read/write to mapped segments NUMA characteristics High End-End Performance Latency: < 3 ms Bandwidth: > 80 MB/s SCI-based NUMA Architecture Export Map SCIphysical addressspace • SCI: Scalable Coherent Interface • State of the art SAN • Link speed of up to 400 MB/s • Export of physical memory • Mapping into virtual memory SCI Ringlet 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  7. Applying SCI‘s HW-DSM • SCI provides a global physical address space • Not directly usable for global virtual memory • Pinned shared segments with own address spaces • Integrate SCI technology with SW-DSM • Page based memory distribution • Map remote pages using SCI • No page migration or replication necessary • Augmented by efficient synchronization • Efficient global virtual memory: SCI-VM 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  8. Flexible and easy-to-useenvironment Running on top of the SCI-VM Shared Memory Applications High-level Prog. Environment (OpenMP, Parallel C++, CORBA) Shmem Programming Model (Threads, TreadMarks, SPMD) SyncMod (Synchronization, Consistency) SCI-VM (Global Virtual Address Space) SCI Hardware (Cluster Environment) • Layered architecture • Global process abstraction • SMP like behavior • Global thread control • Support for variousprogramming models • Transparency for theapplication 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  9. Observing the DSM behavior • Main performance problem: • Bad locality • Overhead due to excessive remote reads • Locality observation difficult • Implicit and fine-grain communication • Data distribution transparently handled by SCI-VM • Need for low-level monitoring system with: • Minimal probe effect • Efficient on-line monitoring capabilities 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  10. Component 2: The SMiLE Monitor SMiLEHW-DSM Monitor SMiLEHW-DSM Monitor ExtensibleMonitoring API Multi-layeredMonitoringInfrastructure Global VirtualMemory for Clusters 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  11. The SMiLE Hardware Monitor SCI out PCI-SCI Bridge Probe SCI in SMiLE Monitor B-Link Interface SCI Network Counter Module PCI local bus DynamicCounterArray Static Counter Array EventFilter PCI Unit 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  12. Features of the SMiLE Monitor • Static monitoring mode • Used on predefined memory areas • Flexible event logic • Spill counter overflows to main memory • Dynamic monitoring mode • Used on whole physical address space • Creation of global access heuristics • Automatic aggregation of neighboring areas • Cache-like swap logic to save hardware resources 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  13. Information delivered • All information acquired is based on SCI packets • Physical addresses • Source/Target IDs • Information can not be used directly • Physical addresses inappropriate • Back-translation to source code level necessary • Need for a monitoring infrastructure • Access to mapping & symbol information • Clean monitoring interface 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  14. Component 3: OMIS SMiLEHW-DSM Monitor ExtensibleMonitoring API ExtensibleMonitoring API Multi-layeredMonitoringInfrastructure Global VirtualMemory for Clusters 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  15. OMIS: Goals and Design • Goal: Flexible monitoring for distributed systems • Specify interface to be used by tools • Decouple tools and monitor system • Increased portability and availability of tools • OMIS Approach • Interface based on Event-Action paradigm • Events: When should something happen? • Action: What should happen? • OMIS provides default set of Events and Actions • Tools define relations between Events and Actions 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  16. OMIS Extensibility • Core functionality • OCM (OMIS Compliant Monitor) core • Access to all node local resources • Communication and Management of Extensions • Adaptation to various environments • OMIS extension modules • Used for features like tracing, checkpointing, ... • Interface for the interoperability of tools • Concurrent deployment of several tools 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  17. Putting it all together Multi-layeredMonitoringInfrastructure Multi-layeredMonitoringInfrastructure SMiLEHW-DSM Monitor ExtensibleMonitoring API Global VirtualMemory for Clusters 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  18. Multi-Layered SMiLE monitoring Tools Prog. Environment Extension High-level Prog. Environment (Specific information) Prog. Model Extension Shmem Programming Model (Specific information) OMIS / OCM Core OMIS SCI DSM Extension SyncMod OMIS/OCM Monitor for DSM systems SCI-VM SMiLE PCI-SCI bridge and monitor Node Local Resources (Statistics of Synchronization mechanisms) (Virt./Phys. Address mappings, Statistics) (Physical addresses, Node IDs, Counters, Histograms) (CPU counters, Cache statistics, OS information) 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  19. Advantages • Comprehensive DSM monitoring • Utilization of information from all components • Structure of execution environment maintained • Generic Shared Memory monitoring • Small Model specific extensions • Flexibility and Extensibility • Profit from existing OMIS environment • Easy implementation • Utilization of existing rich tool base 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  20. Current Status and Future Work • SCI Virtual Memory • Prototype completed • Work on larger infrastructure in progress • SMiLE Hardware Monitor • Prototype by the end of fall • Simulation environment available • OMIS • OMIS definition and OCM core completed • DSM extension in development 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  21. Related Work • Global Memory for clusters • SW-DSM systems like TreadMarks, CVM, ... • SCI-HW based DSM: SciOS, Split-C port • Monitoring for Clusters • SCI deep trace monitor at Trinity College, Dublin • Hardware monitor for SHRIMP adapters • On-Line Monitoring environments • DPCL/dyninst: more low-level than OMIS • Specific solutions like p2d2 (for debugging) 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  22. Conclusions • DSM monitoring crucial for the success of Shared Memory on NUMA architectures • Performance problems due to bad locality • Need for comprehensive infrastructure • SMiLE/OMIS approach • Low-impact low-level monitoring information • Augmented by various data from all layers • Enable back-translation • Access to existing rich tool base 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

  23. For the curious... • SMiLE & OMIS homepages:http://smile.in.tum.de/http://wwwbode.in.tum.de/~omis/ • Email contact:karlw@in.tum.de (Hardware Monitor)schulzm@in.tum.de (Hybrid DSM) trinitis@in.tum.de (OMIS/OCM) 11th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation Martin Schulz, LRR-TUM, 29th March 2000

More Related