1. Network Coding for Large Scale Content Distribution Pablo Rodriguez Microsoft Research Christos Gkantsidis Georgia Institute of Technology IEEE INFOCOM 2005 Presented by Ryan

2. Outline • Introduction • Related Works • Model for Cooperative Content Distribution • Performance Evaluation • Conclusion and Future Works

3. Introduction • Large Scale Content Distribution • Typical content distribution solutions • CDN – Content Delivery Network • Placing dedicated equipment around the network • e.g. Akamai • Cooperative content distribution solutions • Self-scalable • Preventing sudden surge of traffic to the source • e.g. BitTorrent

4. Introduction • Network Coding • Allowing intermediate nodes to encode packets • Making optimal use of the available network resources

5. Introduction • An example • Without a global coordinated scheduler • Node B, receiving Packet 1 or 2 from Node A?

6. Introduction • Contributions in the Paper • Proposing a practical system based on network coding • Not require the knowledge of the underlying topology and centralized scheduling • Robust to extreme situations with sudden server and nodes departures • Better performance comparing to source coding and no encoding schemes

7. Related Works • Tree-Based Cooperative Systems • Creating and maintaining shortest-path multicast trees • Bandwidth-limited (by the bottleneck link on the path from the server) • e.g. SplitStream

8. Related Works • Mesh Cooperative Architectures • Improving the download rates by using parallel downloads • Under-utilizing the network resources (the same block traveling over multiple competing paths) • e.g. BitTorrent

9. Related Works • Erasure Codes • Reconstructing the original content of size n from roughly a subset of any n symbols from a large universe of encoded symbols • Network Coding • Based on theoretical calculations (with the detailed knowledge of the topology and a centralized scheduler)

10. The Model • Server • Dividing the file into k blocks • Uploading blocks at random to different clients • Clients (Users) • Collaborating with each other to assemble the blocks and reconstruct the original file • Exchanging information and data with only a small subset of others (neighbors) • Symmetric neighborhood and links

11. The Model • Upon arrival • Contacting a centralized server (like the tracker in BitTorrent) to get a random list of users in the system • Connecting to the returned users to construct the neighborhood

12. The Model • Content Propagation • 1) No Coding • 2) Source Coding • 3) Network Coding

13. The Model • No Coding and Source Coding • Based only on local information for deciding which block to transfer • Random • A random block • Local Rarest • The rarest block in the neighborhood

14. The Model • e.g. BitTorrent system • A combination of the Random and Local Rarest schemes • Random for the first few blocks • Local Rarest afterwards

15. The Model • Network Coding • The node generates and sends a linear combination of all the information available to it

16. The Model • Recovering the original file after receiving k blocks (associated coefficient vectors are linearly independent to each other) • Just solving the system of linear equations

17. The Model • Incentive Mechanisms • Discouraging free-riding • Scheme 1 • Preference to mutual exchanges • Scheme 2 (Tit-for-tat) • Bounding the absolute difference of uploading minus downloading from one to another

18. Performance Evaluation • Round based simulator • Input • Overlay topology • Users’ upload and download capacities • Server’s capacity • Capacity: number of blocks that can be downloaded/uploaded in a single round • Size of file to distribute • Metric • Download finish time

19. Performance Evaluation • Connecting to 4 peers when joining • Max number of neighbors = 6 • Discovering new neighbors when the utilization of the download capacity is below a certain threshold (10%)

20. Performance Evaluation • Homogeneous topologies • 200 users with capacity = 1 • Server’s capacity = 1 • File size = 100 blocks No Coding Source Coding Network Coding

21. Performance Evaluation • Topologies with clusters • Two clusters, 100 users each • Capacity • Within cluster = 8 • Cluster to cluster = 4 • Server • Capacity = 4 • Departing at round 30 • File size = 100 blocks

22. Performance Evaluation No Coding Source Coding Network Coding

23. Performance Evaluation • Heterogeneous capacities • 10 fast users with capacity = 4 • 190 slow users with capacity = 1 • Server’s capacity = 4 • File size = 400 blocks No Coding Source Coding Network Coding

24. Performance Evaluation • Minimum finish time for the fast users = 50 rounds

25. Performance Evaluation • Dynamic Arrivals • 40 empty nodes every 20 rounds • Capacity = 1 • Staying in the system 10 more rounds after finishing • Server’s capacity = 1 • File size = 100 blocks

26. Performance Evaluation

27. Performance Evaluation • Robustness to node departures

28. Performance Evaluation • Leaving after serving 5% extra blocks • Network coding : 100% finish • Source coding : 40% finish • No coding : 10% finish Network Coding Source Coding No Coding

29. Performance Evaluation • Incentive mechanisms • Max difference = 2 (tit-for-tat)

30. Conclusion • A new content distribution system • Not require knowledge of the whole network topology • Easy to schedule content propagation • Good performance in simulations • Download finish time • Robust to server and users departures • Avalanche – a real system implementation using network coding

31. Future Works • Speed of encoding and decoding • Encoding : O(k) • Decoding : inverting a matrix O(k3), reconstructing the file O(k2) • Dominated by reconstruction • Many reads of large blocks from the harddisk • Protection against malicious nodes • Introducing arbitrary blocks • Making the reconstruction of the original file impossible

32. THANK YOU