1 / 18

Enhancing Performance of T-Applications on Windows Clusters through Dynamic Parallelization

This document outlines the development goals for an advanced Open T-System (OTS) aimed at scientists utilizing Windows-based high-performance computing. The main objectives include achieving Linux-like performance on Windows clusters, adding features such as fault tolerance and visualization, and ensuring seamless operation across heterogeneous clusters. This initiative responds to the needs of the scientific community that prefers Windows over Linux, focusing on ease of use without requiring complex MPI programming. The proposed developments prioritize user accessibility and computational effectiveness.

teague
Télécharger la présentation

Enhancing Performance of T-Applications on Windows Clusters through Dynamic Parallelization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards World-Wide Open T-System PSI RAS, 2006

  2. Long-term Goal • Present a simple in use,automatic dynamic parallelization technology for a scientists, who prefer Windows-based solutions; • Achieve the same performance of the T-applications running on Windows clusters as in a case of the Linux-based clusters; • Advance the Windows version of the Open T-System by adding visualization, fault-tolerance and ability to run T-applications on heterogeneous(Windows+Linux) metaclusters.

  3. Cumputists Wishes • Many scientists prefer to use Windows, not Linux for high-performance computing (by obvious reason) • Many scientists are using C/C++ for high-performance application development • Most scientists are concentrating on the domain-specific problems and do not like like study any kind of MPI-programming technology at all !

  4. Current T-system “community” • PSI RAS (Research & Education) • MSU (Computational Math., Mechanic, Chemistry faculties) • NIICS (satellite image processing) • ChelGU (conformational analysis) • MSIU (education)

  5. Fulfilling Computists Needs • Q: What is a main disadvantage of the Open T-System ? • A: It does not run under Windows (most popular answer at Intel winter school 2005 taking place in Nizhniy Novgorod)

  6. Development priorities 2006 • Run T-applications on Windows hosts & Windows clusters (32bit & 64bit modes) • Run T-applications using variants of communications using multiply pluggable DMPI drivers for SAN, LAN, WAN • Make T-system model more clean for T-application developers by computational process visualization • Run T-applications in fault-tolerant mode • Run T-applications on Itanium & PowerPC

  7. Help for application tuning • Develop a tool for T-program computational process visualization (in particular, to present parallelization dynamic in real time) • Built-in lightweight performance analyzer to automate process of T-program optimization

  8. Achieving power of metaclusters • Fault-tolerant computation mode (using advanced re-computation model) • Run Windows T-application on both Windows and Linux clusters (using the wine emulator) • Dynamic enter/leave of computational nodes/segments

  9. MacroScheduler 2006 advance • Reducing “resource exchange” message count at large clusters • Highly efficient algorithm adopted for non-trivial cluster topologies

  10. Current DMPI features • Implements most popular subset of MPI • mpirun/mpiexec type auto-detection • Dynamic loading for appropriate MPI driver • Ability of static MPI binding • Asynchronous operations support • Large number of supported MPI & PVM implementations

  11. Additional DMPI 2006 features • Direct TCP/IP support (able to operate without MPI library) • Multiply MPI drivers working in parallel to mix different transport systems: SCALI, Myrinet, Infiniband, TCP/IP, TCP/IP over HTTP. • MPI Enter/Leave extensions (in addition to standard mpirun/mpiexec running mode) • MPI_Wait & T-Fun waiting coexistence • Automatic fault detection & recovering • Configuration (topology) information

  12. Cross-platform OpenTS • Unified OpenTS source code tree for ALL supported platforms • Unified DMPI source code tree for ALL supported platforms • Extracting all platform-depended features into separate Platform Abstraction Layer (PAL)

  13. OpenTS for Windows installer • Unified installer for Windows XP/2003/MSCCS editions • Installed OpenTS does not require any commercial package (it is enough to download Visual C++ 2005 Express edition for T-program development freely available from www.microsoft.com site) • Integrated with Visual Studio (T-projects) • Self-testing during installation procedure! • OpenTS SDK is available for T-microkernel developers

  14. Open T-Grid 2006 • Cross-platform: run T-applications on Windows and Linux cluster & hosts at the same time(using WinAPI simulation on Linux) • Fault-tolerance: using new DMPI facilities for immediate recovering and unique functional nature of T-system • User-friendly: run T-visualization as an screensaver or as widget/applet in the taskbar • Safety: digital signature check support for T-executables

  15. Open T-Grid 2006 • Cooperative usage of the Windows and Linux clusters and single hosts • World-Wide T-system-based computational platform • Good (and awaited) platform for scientists from Universities

  16. Official T-System projects (2006)(donation-ordered) • MSCCS (Microsoft RUS ~ $60.000) • T-GRID (RAS ~ $50.000) • T++ & T# (RAS ~ $30.000) • TRIADA (Russia-Belorussia ~ $40.000)

  17. Cost Analysis • Most new OpenTS features are developed for POSIX environment at the current time. • To adopt presented features for Windows we need reasonable financial support at the next year

  18. OpenTS 2006 restrictionsscheduled to be removed • Restricted C++ OOP support: no tfun class methods, no tfun templates; • Intermediate levels of OpenTS API are open, but not documented yet (Active messages, Mobile objects, Supermemory) • Further performance enhancements (intermediate levels template specialization)

More Related