Map Reduce: Simplified Data Processing on Large Clusters

1. Map Reduce: Simplified Data Processing on Large Clusters By Jeff Dean & Sanjay Ghemawat Google Inc. OSDI 2004 Presented by : MohitDeopujari

2. Cluster Architecture • Distributed File Systems • Coping with Node Failure • Large-Scale Indexing Topics Covered in Class: Map Reduce

3. A new programming model for processing and generating large data sets • A new abstraction that enables simple computations to be parallelized while hiding away the details • Mapand Reduce are two primitives directly inspired by Functional languages such as LISP • This Functional model enables massive parallelization and fault-tolerance by re-execution Introduction to Map-Reduce: Map Reduce

4. Input consists of key/value pairs, Final output is also key/value pairs • Computation is expressed as two functions : Map and Reduce • Map :- • Written by user • Input is key/value pairs, Output is intermediate key/value pairs • Groups together all intermediate values associated with same intermediate key I and passes onto Reduce • Reduce :- • Also written by user • Accepts intermediate key I and set of values for that key • Merges together these values to form a possibly smaller set of values • Zero or one output value produced per invocation • An iterator is used to feed the intermediate values to the function thus allowing us to handle lists too large to fit in memory The Programming Model: Map Reduce

5. map(String key, String value): // key: document name // value: document contents for each word w in value: EmitIntermediate(w, "1"); • reduce(String key, Iterator values): // key: a word // values: a list of counts intresult = 0; for each v in values: result += ParseInt(v); Emit(AsString(result)); An example to count number of occurrences of each word: Map Reduce

6. An example to count number of occurrences of each word: • the input keys and values are drawn from a different domain than the output keys and values • the intermediate keys and values are from the same domain as the output keys and values. Map Reduce

7. Execution Overview: • MapReduce library in the user program splits the input file into M chunks (16 to 64MB each) and then starts up many copies of the program on a cluster of machines. • A special copy, the Master, assigns each of the other copies (workers) a Map or a Reduce task. • Each worker assigned with the Map task parses the key/value pair from the corresponding chunk of input and passes it to the Map function. Intermediate key/value pairs are produced and buffered in memory. • Periodically, a partitioning function partitions these pairs into R regions and writes to local disk. The locations of these pairs on disk is passed to the master. • When a reduce worker is notified of these locations by a master, the worker sorts the intermediate keys and groups the values by key. Map Reduce

8. Execution Overview: 6. Then, the reduce worker iterates over each unique key and passes on the list of corresponding values to the Reduce function. The output of the Reduce function for that partition is appended to the final output. 7. When all map tasks and reduce tasks have been completed, the master wakes up the user program. At this point, the MapReduce call in the user program returns back to the user code. Map Reduce

9. Fault Tolerance: • Master Failure: • Highly unlikely since there is only a single master • If there is a failure, whole task is aborted and can be restarted • Worker Failure: • If a Map worker fails, it is detected by Master and that node’s map tasks (even if completed) are set to idle and made ready to be rescheduled on other machines • Reduce workers are notified of the new locations from which to pick up data • If a Reduce worker fails, its tasks are set to idle and made ready to be rescheduled on other machines Map Reduce

10. Locality: • Network bandwidth is conserved because of the fact that input data resides on the local disks of the machines that make up the data cluster • The Master knows location of input files. It schedules a map task on a machine that has a copy of the corresponding input data • Thus, on large MapReduce operations, the input data is read locally and bandwidth consumption is minimal Map Reduce

11. Usage at Google: Map Reduce

12. Example uses : Map Reduce

13. Pros : • Simple and powerful interface enables automatic parallelization and distribution of large scale computations • High performance achieved on large clusters of commodity CPUs (cheap hardware) • Fair fault-tolerance and capability to handle failures and still make progress • Locality of input data conserves bandwidth and helps speed up tasks • Wide variety of applications • Was used to completely rewrite the Google Indexing code in turn making it easier to read, manage while improving performance and efficiency Map Reduce

14. Cons : • Debugging is tricky since actual computation occurs in a distributed environment on thousand of machines with work assignments being made dynamically • In case of Map Node failure, the Map tasks need to be started all over again since the intermediate output lies in the failed machine Map Reduce

15. Conclusion : • MapReduce has been widely used at Google for many tasks primarily due to these reasons: • The model is easy to use, even for programmers without experience with parallel and distributed systems, since it hides the details of parallelization, fault-tolerance, locality optimization, and load balancing • Alarge variety of problems are easily expressible as MapReducecomputations • Easy and efficient scalability Map Reduce

16. Conclusion : • Use of a restricted programming model (based on functional model) enables easy parallelization and fault-tolerance through re-execution • Network bandwidth is scarce hence many optimizations need to be built to minimize wastage • redundant execution can be used to reduce the impact of slow machines, and to handle machine failures and data loss Map Reduce