网络专题选讲

1. 网络专题选讲 华中科技大学 电子与信息工程系 程文青 chengwq@mail.hust.edu.cn 2013年1月

2. 社会网络应用专题选讲 华中科技大学 电子与信息工程系 互联网技术与工程研究中心 黑晓军 Email: heixj@hust.edu.cn Web: http://itec.hust.edu.cn/~heixj 2013.1

3. Outline Introduction Case study Traffic transportNetTube: Exploring Social Networks for Peer-to-Peer Short Video Sharing, 2009 IncentiveP2P Trading in Social Networks: The Value of Staying Connected, 2010 RecommendationCircle-based Recommendation in Online Social Networks, 2012 《网络专题选讲 》

4. Internet Topology

5. Introduction 5 我们生活在一个关系的社会

6. 社会网络应用 6

7. Friend network in Facebook 7

8. Co-authorship network 8

9. Co-authorship in network science 9

10. Ingredient networks 10

11. 911事件——犯罪网络 11

12. Social Networking（General public）

13. Social Networking（General public）

14. Social Networking（Academia）

15. 专著

16. 专著

17. 研究现状 17 • 主要研究机构 • 国外：MIT、Stanford、Maryland、USC、HP、Michigan • 国内：IBM中国研究院、微软亚洲研究院、中科院、中国传媒大学、清华大学、南京大学 • 近年来社会网络成为国内外研究热点 • 美国国家科学基金会（NSF）将社会计算研究领域提供专项资金（2010） • 美国计算机协会ACM，Workshop on Social Network Mining and Analysis(2007~2012) • WWW会议成立“Social Network and Web2.0 Track”论坛(2009) • SIGCOMM，ACM SIGCOMM Workshop on Online Social Networks(2009~2012) • EuroSys, Workshop on Social Network System(2009~2012) • 互联网测量会议（IMC），海量数据仓库国际会议（VLDB），信息与知识管理（CIKM）大量关于社交网络文章 • 全国网络科学论坛（2004～2012），全国复杂网络会议（2005～2011）

18. NetTube:Exploring Social Networks for Peer-to-Peer Short Video Sharing Xu Cheng and Jiangchuan Liu School of Computing ScienceSimon Fraser University British Columbia, Canada October 2009 IEEE INFOCOM, 2009

19. Background (1) • Social Networked Media Sharing – new killer Internet application • Since 2005 • Rich user-generated content (UGC) sharing • Social networks • Among users • Among videos • Changing the popular culture

20. Background (2) • YouTube – a representative • Popular • Market share of around 43% • More than six billion videos viewed in January 2009 • Consumed as much bandwidth as the entire Internet in 2000 • 3rd visits among all Internet sites (after Google and Yahoo) • Fast growing • 20% growth rate per month • 15 hours of new videos are uploaded every minute

21. Motivation (1) Can we make it better ? • The YouTube Crisis – all other sites’ challenge • Severely hindered by client/server architecture • Bandwidth costs • Consumed as much bandwidth as the entire Internet in 2000 • $1 million a day for server bandwidth! • Sold to Google for $1.65 billion in Nov. 2006 • Performance and scalability • “Slow” among the surveyed sites by Alexa.com

22. Motivation (2) • Peer-to-peer (P2P) – alternative to Client/Server • New generation of communication paradigm • Each peer contributes its bandwidth to serve others • Scale well with larger user base • More users, more resources contributed • Success already seen in • BitTorrent, eMule, eDonkey (file sharing) • Video broadcasting • …

23. P2P架构 没有永远在线的服务器 任意主机可以同另一个主机进行通信 节点可以间歇性的连入系统，IP地址可能会变化 peer-peer 23

24. 对等网络流媒体系统 两大设计空间 如何形成重叠网络？ 如何传输内容？ 现有体系结构 树状拓扑 + 推式内容传输 ESM, Yoid, CoopNet, SplitStream, Bullet, Chunkspread … 网状拓扑 + 拉式内容传输 -24- 《网络专题选讲》

25. 网状-拉式对等网络流媒体系统 这类系统非常类似于BitTorrent -25- 《网络专题选讲》

26. 节点软件结构 双缓存 积极下载 vs 保守下载 处理丢失的数据块 缓存控制 -26- 《网络专题选讲》

27. 缓存映像(buffer map) 缓存映像反映了节点缓存所拥有的数据块信息 此映像可以被用来评估用户的播放质量 -27- 《网络专题选讲》

28. 对等网络中的问题 28 内容组织和搜索 内容传输 信誉、激励及安全相关问题

29. CoolStreaming • The first practical large-scale P2P NetTV • Origin of data-driven mesh design • With many follow-ups: PPLive, PPStream, UUSee … X. Zhang, J. Liu, B. Li, and T.-S. P. Yum, CoolStreaming/DONet: A Data-driven Overlay Network for Live Media Streaming, IEEE INFOCOM'05, March 2005. >800 citations J. Liu, S. G. Rao, B. Li, and H. Zhang, Opportunities and Challenges of Peer-to-Peer Internet Video Broadcast, Proceedings of the IEEE, Vol. 96, No. 1, pp. 11-24, January 2008.

30. Data-Driven Mesh • Core operations • Every node periodically exchanges data availability information with a set of partners • Then retrieves unavailable data from one or more partners, or supplies available data to partners • Easy to implement • no need to construct and maintain a complex global structure • Efficient • data forwarding is dynamically determined according to data availability • Robust and resilient • adaptive and quick switching among multi-suppliers

31. Challenges and Opportunities (1) • Challenges – Drastically different statistics • 1.5 year measurement of 5 million videos • http://netsg.cs.sfu.ca/youtubedata/ • Short video clips – stability • 99.6% are less than 700 seconds • “I don’t want to wait for 30 seconds for a two-minute video!” • Searching for sources • High churn rate: join/leave system • Huge number of videos – scalability • Highly skewed • Inefficient for unpopular videos • Very few users watch the same one

32. Challenges and Opportunities (2) • Opportunities – Social networks • No longer independent – videos have related videos • Small-world – strong clustering • Important role

33. NetTube Design (1) • Bi-layer overlay network • Lower-layer – per-video • Download and uploading • Peers stay in previous overlays as sources • Larger and more stable • Upper-layer – social network • Connected by the same peers in different lower-layer overlays • Conceptual relation for searching • Social network brings similar peers closer • Clustering • Efficient searching

34. NetTube Design (2) • Bloom filter based indexing • An efficient approach to keep track of peers’ cached videos • Bloom filter • An m-bit array using k hash functions • Space-efficient • Scalable indexing table • Fast searching • Table size is scalable with the number of videos • Search locally and search in the upper-layer overlay • Social network clustering the similar video

35. NetTube Design (3) Transmission scheduling: From which partner to fetch which data segment ? • Constraints • Data availability • Playback deadline • Heterogeneous partner bandwidth • Rarest-first (BitTorrent’s) doesn’t work !

36. NetTube Design (3) • Variation of Parallel machine scheduling • NP-hard • Conventional Heuristics • Message exchanged • Window-based buffer map (BM): Data availability • Segment request (piggyback by BM) • Less suppliers first • Multi-supplier: Highest bandwidth within deadline first

37. NetTube Design (3) • Short video ? • CODAS: Collaborative Delay-Aware Scheduling

38. NetTube Design (4) • Social network assisted pre-fetching • Most peers finish downloading before playback ends - free time available (about 80 seconds on average) • Using free time to reduce startup delay • Prefix pre-fetching • Avoid wasting bandwidth and space • Enable multiple pre-fetching • Multiple pre-fetching • Accuracy increases greatly • Accuracy increases as watch more videos • Pre-fetching among neighbors • Easy to implement • Social network helps improve efficiency

39. Performance Evaluation (1) • Simulation • Configuration • Based on about 7,000 crawled videos • Scale to more than 10,000 heterogeneous clients • Compare with PA-VoD (MSN Video) • Bandwidth reduction • Save significantly more • More scalable

40. Performance Evaluation (2) • Simulation • Impact of social network • Find more sources: more than 95% within 2 hops • Greatly increase pre-fetching accuracy

41. Performance Evaluation (3) • PlanetLab experiment • Configuration • Maximum 235 PlanetLab nodes • Experiment results • Server bandwidth reduction: more than 40% • Startup delay: average 2.2 s • Playback continuity

42. Summary • Contribution • First social network assisted P2P system for short video sharing • IWQoS’08, INFOCOM’09, IEEE Transactions on Multimedia • Techniques • Bi-layer overlay network • Bloom filter based indexing • Social network assisted pre-fetching • Collaborative delay-aware scheduling • Evaluation results • Greatly reduce server bandwidth • Much lower maintenance cost: $1 million → $60 K • Inherently scalable – P2P • Greatly reduce playback delay • Satisfying startup delay • Continuous playback

43. P2P Trading in Social Networks: The Value of Staying Connected Zhengye Liu, Hao Hu, Yong Liu, Keith Ross, Yao Wang, and Markus Mobius Polytechnic Institute of NYU Dept. of Economics, Harvard Unviversity IEEE INFOCOM, 2010 43

44. Outline 44 Background: P2P Incentive Networked Asynchronous Bilateral Trading (NABT) NABT Efficiency Theory NABT Simulations Conclusions

45. P2P Apps: BitTorrent 45

46. P2P Apps: Skype 46

47. Peer-Assisted Video Streaming 1 6 2 5 3 4 47 • Large scale deployments on Internet • thousands of live/on-demand channels • millions of world-wide users daily • Leading P2P Video Companies • CoolStreaming • PPStream • PPLive • Sopcast • UUSee

48. Major P2P Issues 48 • Traffic localization • P4P • Security • Attacks on • Attacks from • Lack of uniform API • Incentives for peers to contribute resources

49. Partially Successful P2P Incentive + P2P design  Resources Incentives 49 • BitTorrent is popular • 50+ client implementations • Dozen public trackers • 5-10 million users • Why BitTorrent? • First generation P2P applications: Gnutella • 70% of users are free-riders • Second generation P2P applications: BitTorrent

50. The BitTorrent Incentive To get files faster… contribute more bandwidth 50 Implementation of incentive: • The rich play/trade with the rich