1 / 16

MapReduce : Simplified Data Processing on Large Clusters

MapReduce : Simplified Data Processing on Large Clusters. By: Jeffrey Dean and Sanjay Ghemawat Presented By: Hind Alhakami. Problem to Solve. Simple queries on huge amount of data . Distribution needed, which Complicate the problem. Solution. Example. Literature.

nodin
Télécharger la présentation

MapReduce : Simplified Data Processing on Large Clusters

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MapReduce: Simplified Data Processing on Large Clusters By: Jeffrey Dean and Sanjay Ghemawat Presented By: Hind Alhakami

  2. Problem to Solve • Simple queries on huge amount of data. • Distribution needed, which Complicate the problem.

  3. Solution

  4. Example Literature • Count number of occurrences of each word in the given literature

  5. Simplified Example • For Simplicity, Consider counting words in one document.

  6. First Step - Map

  7. Second Step - Reduce

  8. Back to the Original Example • {(“the”,3), ...}

  9. Execution Overview

  10. Worker Failure

  11. Master Failure

  12. Experiment • Cluster of approximately 1800 machine. • Two Benchmarks: • Grep • Sort

  13. Performance

  14. Performance Normal execution No backup tasks 200 tasks killed

  15. Optimizations • Combiner function. • Skipping bad records. • Local execution. • Status Information.

  16. Conclusion • MapReduce provides a convenience level of abstraction. • Supports scalability and fault-tolerance. • Preserves network bandwidth.

More Related