1 / 15

Adaptive File Transfers for Diverse Environments

Adaptive File Transfers for Diverse Environments. Himabindu Pucha, Purdue University. David G. Andersen, Carnegie Mellon University Michael Kaminsky, Intel Research Pittsburgh Michael Kozuch, Intel Research Pittsburgh. Goal.

footem
Télécharger la présentation

Adaptive File Transfers for Diverse Environments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Adaptive File Transfers for Diverse Environments Himabindu Pucha, Purdue University David G. Andersen,Carnegie Mellon University Michael Kaminsky,Intel Research Pittsburgh Michael Kozuch,Intel Research Pittsburgh

  2. Goal Correctly and efficiently transfer files in wide range of scenarios

  3. Network peers Goal Correctly and efficiently transfer files in wide range of scenarios • Data backup, code update • Software synchronization • Different network speeds Scenario: Data backup, code update Scenario: Software synchronization file in-place Gigabit LAN –DSL links Network Search for similar files Receiver Sender Different disk loads

  4. Problem: Existing Tools Scenario-specific

  5. Challenges • Resources have widely varying performance • Resource performance changes dynamically • Support receivers with different initial state • Do not require resources to be set up in advance

  6. dsync: Design dsync uses all available resources effectively dsync scheduler Network Disk

  7. dsync: Design • Discovers available resources using exposed backpressure information • From disk: “I’m busy writing, don’t read from me.” • From network: “I have lots of incoming packets, don’t spend time doing IO or computation.” • Schedules intelligently across available resources • Disk: use a pre-computed index and/or search entire disk using heuristics • Network:Schedule remaining chunks, least likely to be found on disk

  8. dsync: Preliminary Results Throughput for 1 GB file on a 1 Gbps link dsync defers disk operations when network is faster than disk Bonus: dsync provides best of BitTorrent, rsync, scp …

  9. dsync: Preliminary Results Average download time across 45 receivers, 50% similar file in-place dsync speedup: 5x vs. rsync 2x vs. SET dsync rapidly locates similar files and effectively combines them with peering dsync correctly uses backpressure to defer disk operations when network is faster than disk

  10. BACKUP

  11. dsync: Preliminary Results dsync correctly uses backpressure to defer disk operations when network is faster than disk

  12. Network peers Goal Correctly and efficiently transfer files in wide range of scenarios • Data backup • Code update • Software synchronization • Different network speeds Scenario: Software synchronization Scenario: Data backup Scenario: Code update file in-place Gigabit LAN –DSL links Network Search for similar files Receiver Sender Different disk loads

  13. Problem: Existing Tools Scenario-specific

  14. dsync: Design • Discovers available resources using exposed backpressure information • From disk: “I’m busy writing, don’t read from me.” • From network: “I have lots of incoming packets, don’t spend time doing IO or computation.” • Schedules intelligently across available resources • Disk: use a pre-computed index and/or search entire disk using heuristics • Network:Schedule remaining chunks, least likely to be found on disk

  15. Challenges • Correctly use resources with widely varying performance characteristics • Dynamically adapt to changes in resource performance • Support receivers with different initial state • Do not require resources to be set up in advance

More Related