SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Fault-Tolerant Parallel Algorithms for Data-Intensive Systems

milo avatar
milo
1 diapositives · 165 Views · Jul 27, 2014

This paper explores fault-tolerant parallel algorithms optimized for data-intensive applications, particularly in light of the decreasing Mean-Time-To-Failure (MTTF) as core counts increase on computing systems. Recognizing that traditional recovery methods may not suffice for future exascale systems, we propose an intelligent replication strategy that minimizes data loss and re-processing by utilizing summarization techniques. Our experimental setup uses C programming with MPI on Opteron processors, demonstrating our approach's effectiveness in handling fail-stop failures while enhancing recovery efficiency.

Fault-Tolerant Parallel Algorithms for Data-Intensive Systems - slide 1
01 / 1Fault-Tolerant Parallel Algorithms for Data-Intensive Systems
1 / 17

Télécharger la présentation

Fault-Tolerant Parallel Algorithms for Data-Intensive Systems
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Fault Tolerant Parallel Data-Intensive Algorithms Mucahid Kutlu, Gagan Agrawal, Oguz Kurt (Ohio State University) Introduction and Motivation ◆The Mean-Time-To-Failure (MTTF) of the sys- tems is decreasing with growing number of cores. ◆For the future exascale systems, it is being argued that check- pointing and recovery time (with current methods) will even exceed the MTTF. ◆Algorithm-based fault-tolerance can be alternative method • Our Approach • - Intelligent Replication • ◆Minimum Data Intersection by dividing data into blocks and distributing them in different processors. • ◆ Passive Replicas • - Summarization • ◆After processing one block, a summary is generated for that block and sent to the master node. • ◆ No need to re-process the blocks that a summary is already sent before the failure. Experimental Setup ◆ Implemented k-means and apriori algorithms in C programming language by using MPI library. ◆ Used 2.5 GHz Opterons processors and 24 GB memory ◆ The number of processors is 8 ◆ In the experiments with Hadoop: ◆Replication factor(R) : 3 ◆Summarization frequency(S) : 4 P1 P2 P3 P4 P5 P6 P7 Experimental Results primary 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Impact of Summary Exchange Frequency in Apriori: Varying Number of Failures Our Goal ◆We focus on only fail-stop failures. ◆We do not use any back up node and continue the process with remaining nodes after the failure. ◆We have two main goals for faster recovery: ◆minimize the data loss, since the lost data needs to be reread from the storage cluster ◆minimize re-processing of the lost data replica 12 13 1 14 2 3 4 5 6 7 8 9 10 11 Recovery Scenario ◆ P1 and P2 fail at the beginning of the iteration. ◆ Master node notifies - P3 to process D2 and D3 - P4 to process D4 ◆ Since all D1 blocks are lost, master node reads D1 from the file system/storage cluster and notifies P4 to process it. Master File System Total Execution Time that Changes with the Number of Failures P1 P2 P3 P4 D1 D2 D3 D4 D5 D6 D7 D8 D6 D7 D1 D8 D2 D3 D4 D5