SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Massively Parallel Ensemble Methods Using Work Queue

albina avatar
albina
21 diapositives · 342 Views · Jul 20, 2014

This workshop presentation discusses the revolutionary approach of using massively parallel ensemble methods via Work Queue software to tackle the complexities of protein folding. It highlights the significance of molecular dynamics and the role of misfolded proteins in diseases like Alzheimer's. The presentation shares insights into Folding@Work and the Accelerated Weighted Ensemble (AWE) algorithms. It addresses the challenges of computational efficiency, bottlenecks, and the potential of cloud computing in bioinformatics, while emphasizing the importance of adaptive sampling and collaborative computing.

Massively Parallel Ensemble Methods Using Work Queue - slide 1
01 / 21Massively Parallel Ensemble Methods Using Work Queue
1 / 17

Télécharger la présentation

Massively Parallel Ensemble Methods Using Work Queue
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Massively Parallel Ensemble Methods Using Work Queue Badi’ Abdul-Wahid Department of Computer Science University of Notre Dame CCL Workshop 2012

  2. Overview • Background • Challenges and approaches • Our Work Queue software • Folding@Work (F@W) • Accelerated Weighted Ensemble (AWE) • Discussion and usage of F@W and AWE

  3. Background: Molecular Dynamics • Proteins can be thought of as strings of molecules that adopt 3D conformations through folding. • Many diseases are believed to be caused by protein misfolding • Alzheimer’s: amyloid beta aggregation • BSE (Mad Cow): prion misfolding • Understanding protein folding allows us to better understand disease and develop and test treatments.

  4. Challenges for MD Courtesy of Greg Bowman Bowman, G. R., Voelz, V. A., & Pande, V. S. (2010). Current Opinion in Structural Biology, 21(1), 4–11

  5. One Approach: Specialized Hardware • Anton • Developed by D.E. Shaw Research • 1000x speedup • Expensive and limited access • GPUs • Requires a programming paradigm shift • OpenMM allows MD to run on GPUs • More accessible, but still limited capabilities • Eg: certain features present on CPU MD software are not yet provided for GPU

  6. Another Approach: Massive Sampling • Folding@Home (Vijay Pande at Stanford) • Crowd sources the computation • Donors allow simulations to be run on their computers • 300,000+ active users • CPUs & GPUs • 6800 TFLOPs (x86)

  7. Goal: Sample Rare Events • Pathways through transition regions • Difficult to capture experimentally • High computational complexity • High potential for parallelism

  8. Adaptive Sampling • Traditional methods: Poor sampling of unstable regions • Adapt sampling procedure to enrich these areas UC Davis Chemwiki

  9. Products • Folding@Work • Accelerated Weighted Ensemble • Written with Haoyun Feng • Both are built on top of Work Queue • Use the Python bindings to Work Queue

  10. Folding@Work Data dependencies/organization Simulation lifetime

  11. Accelerated Weighted Ensemble

  12. WW Fip35 • Prep. Simulation • 200+ us • Amber96 • Generalized Born • Timestep of 2fs • Gamma of 1/ps • NvidiaGeForce GTX 480 • OpenMM • MSMBuilder

  13. AWE Resources • Multiple masters • …sharing several pools • …from different sources: • ND GRID: CPUs • Condor: ND + Purdue + U. Wisc. Madison. CPUs • Stanford ICME: GPUs • Amazon EC2: High CPU XL instances • Windows Azure: Large CPU instances

  14. Pool allocation for WQ masters

  15. Execution Times

  16. Bottlenecks: communication • All data is sent through Work Queue • Initialization cost: 34MB • Payloads (to/from): 100KB • Architecture-depend files specified via WQ-exposed $OS and $ARCH variables • 2% of tasks required initialization • < 1s average communication time • < 1% of master execution time wq.specify_input_file("binaries/$OS-$ARCH/mdrun") wq.specify_input_file("binaries/$OS-$ARCH/assign") wq.specify_input_file("binaries/$OS-$ARCH/g_rms")

  17. Bottlenecks: straggling workers • 50% of the time to complete an iteration was due to certain workers taking overlong. • WORK_QUEUE_SCHEDULE_TIME • Fast abort multiplier = 3 • Recently implemented WQ Cancel Task allows task replication to overcome this

  18. Preliminary results Transition network from AWE Sample folding pathways

  19. Conclusions • Work Queue allows Folding@Home style applications to easily be run using Folding@Work • Work Queue allowed the first application of the AWE algorithm to a large biomolecular system • An important aspect was usage of different resources: CPU + GPU on Cloud + Grid + Dedicated • The synchronization barrier in AWE allows straggling workers to have a large impact • Currently testing solution using new WQ features • These preliminary results (AWE) are promising, but further validations needed

  20. Acknowledgements • Prof. Jesús A. Izaguirre (Notre Dame) • Prof. Douglas Thain (Notre Dame) • Prof. Eric Darve (Stanford) • Prof. Vijay S. Pande (Stanford) • Li Yu, Dinesh Rajan, Peter Bui • Haoyun Feng, RJ Nowling, James Sweet • CCL • Izaguirre Group • Pande Group • NSF • NIH

  21. Questions?