1 / 10

Parallel Applications And Tools For Cloud Computing Environments

Parallel Applications And Tools For Cloud Computing Environments. CloudCom 2010 Indianapolis, Indiana, USA Nov 30 – Dec 3, 2010. Large Scale PageRank with Iterative MapReduce. Shuohuan,Yuduo,Parag,Hui. Outline. m otivation of large scale pagerank o ptimization s trategies

yardley-pitts
Télécharger la présentation

Parallel Applications And Tools For Cloud Computing Environments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Applications And Tools For Cloud Computing Environments CloudCom 2010 Indianapolis, Indiana, USA Nov 30 – Dec 3, 2010

  2. Large Scale PageRank with Iterative MapReduce Shuohuan,Yuduo,Parag,Hui

  3. Outline motivation of large scale pagerank optimization strategies experiments results visualization with PlotViz3

  4. PageRank • Large scale PageRank • Large graph processing become popular • Efficient processing of large scale graph challenges current MapReduce runtimes. • Motivation: common optimization strategies for large scale PageRank • Current status • Twister, Hadoop,DryadLINQ with ClueWeb data set with 50 million pages • MPI PageRank

  5. Optimization Strategies • Cache partitions of web graph in Memory • Twister, Pregel, HaLoop, Surfer, • Static Data (am files) • Partition the web graph • DryadLINQ, (Twister, Hadoop) PageRank • Task granularity should fit the memory and network bandwidth in Cloud infrastructure • Hierarchy messaging in reduce stage • Hadoop, (Twister, DryadLINQ) PageRank • Local merge

  6. Cache Static Data

  7. Partition the WebGraphscalability with various nodes on Madrid

  8. Partition the web graphscalability with various input data size on Tempest

  9. Hierarchy Messaging in Reduce Stage

  10. Visualization with PlotViz31k vertices, red vertex: wikipedia.org

More Related