1 / 35

Coordinated Management and Peering of Content Delivery Networks

Coordinated Management and Peering of Content Delivery Networks. Gri d Computing and D istributed S ystems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia www.csse.unimelb.edu.au/~apathan. Al-Mukaddim Khan Pathan.

wes
Télécharger la présentation

Coordinated Management and Peering of Content Delivery Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Coordinated Management and Peering of Content Delivery Networks Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, Australiawww.csse.unimelb.edu.au/~apathan Al-Mukaddim Khan Pathan Supervisor: A/Prof. Rajkumar Buyya Masters to PhD Conversion Seminar

  2. Outline • Introduction • Problem statement • Possible solution • Research Goals • Work completed • Research Plan

  3. With the rapid growth of Internet and Web Services are competing each other for finite network and computing resources High availability and responsiveness are keys to business Web sites Large number of users are trying to simultaneously access the same Web site, causing “Flash Crowd” Internet & Web: Challenges

  4. Slow content must traverse multiple backbones and long distances Unreliable delivery may be prevented by congestion or backbone peering problems Not scalable usage limited by bandwidth available at master site Inferior streaming quality packet loss, congestion, and narrow pipes degrade stream quality Problems with the Centralized Web Approach Source: Bruce Maggs, CCGrid 2001 Keynote

  5. Resource Distribution Approach • Content Delivery Networks (CDNs) is the solution to such service degradation • Moving content to the “edge” of the Internet, closer to the end-users

  6. CDN: Abstract View

  7. CDN Commercials and so on…

  8. Outline • Introduction • Problem statement • Possible solution • Research Goals • Work completed • Research Plan

  9. Limitations of Current CDNs • Existing CDNs are proprietary in nature • Each has expensive closed delivery network • No simultaneous usage of the resources of multiple CDNs • CDN expenses have led to consolidation in the market • Commercial CDNs sign Service Level Agreements (SLAs) with their customers • Objective is to provide competitive services satisfying QoS requirements • “Flash Crowd” or “SlashDot Effect” cause adverse business impact • Very often CDNs can not provide QoS to end-users requests • SLA violation to end-up costing the provider

  10. Problem Statement • Enable coordinated and cooperative content delivery between existing CDNs • Maximize performance and timeliness of Web requests of end-users • Maximize the utilization of resources • Reduce cost and labor

  11. Related Work • IETF Content Distribution Internetworking (CDI) model • Federated, multi-provider infrastructure for content delivery workloads • A peering algorithm • CDN brokering • Use of Intelligent Domain Name Server (IDNS) • Content Internetworking Router (CiRouter) • Performance code to run on client side • CoopNet • Client side mechanism • CoDeeN, Coral, Globule, DotSlash • Collaborative content delivery

  12. Outline • Introduction • Problem statement • Possible solution • Research Goals • Work completed • Research Plan

  13. Possible Solution • Ad-hoc or planned peering arrangements between CDNs by leveraging existing infrastructures • Allows a CDN to “scale-out” to meet both flash crowds and anticipated increases in demand • Avoids expense of running a global CDN • Achieves economics of scale, in terms of cost effectiveness and performance for both providers and end-users • Avoids SLA violation • Assists to provide competitive service to catch-up the market

  14. CDN Peering Scenario

  15. Outline • Introduction • Problem statement • Possible solution • Research Goals • Work completed • Research Plan

  16. Key Issues • When to peer? • The circumstances under which peering should be triggered • How to peer? • The strategy taken to form peering between multiple CDNs • Whom to peer with? • The decision making mechanism for choosing CDNs to peer with • How to manage and enforce policies? • Deploying necessary policies and administering them in an effective way

  17. Thesis Aims • Create an architectural framework for coordinated and cooperative CDNs • Apply Virtual Organization concepts for forming peering CDNs • Develop an effective content replication strategy • Use economic models to develop a cooperative pull-based replication technique • Develop an effective load measurement and dissemination technique • Enforce a hierarchical approach to maximize utility of resources • Develop an effective request redirection strategy • Enforce a multiple level request assignment/redirection technique to make it transparent and efficient • Prototype system for peering CDNs • Leverage existing technologies and deploy on real-world testbed

  18. Outline • Introduction • Problem statement • Possible solution • Research Goals • Work completed • Research Plan

  19. Work Completed • An in-depth analysis and survey of the content delivery networks in terms of • Organizational structure, content distribution mechanisms, request redirection techniques, and performance measurement methodologies Publication: A. M. K. Pathan and R. Buyya, “A Taxonomy and Survey of Content Delivery Networks,” Technical Report, GRIDS-TR-2007-4, Grid Computing and Distributed Systems Laboratory, The University of Melbourne, Australia, 12 February, 2007. (Submitted to ACM Computing Surveys) • The necessity of peering among CDNs according to a Virtual Organization (VO)-model • Through identifying significance and evaluating case studies Publication: R. Buyya, A. M. K. Pathan, J. Broberg, and Z. Tari, “A Case for Peering of Content Delivery Networks,” IEEE Distributed Systems Online, 7(10), Los Alamitos, CA, October 2006.

  20. Work Completed (Cont’d) • A comprehensive architectural framework for peering CDNs • With the roles, responsibilities and expected interaction patterns of architectural components • Illustrating performance gain through peering Publication: A. M. K. Pathan, J. Broberg, K. Bubendorfer, K. H. Kim, and R. Buyya, “An Architecture for Virtual Organization (VO)-Based Effective Peering of Content Delivery Networks,” UPGRADE-CN’07, In Proceedings of 16th IEEE International Symposium on High Performance Distributed Computing (HPDC 2007), Monterey, California, USA, 25-29 June, 2007. • An economy-based replication strategy • For on-demand placement of outsourced content Publication: A. M. K. Pathan and R. Buyya, “Economy-based Content Replication for Peering Content Delivery Networks,” TCSC Doctoral Symposium, In Proc. of 7th IEEE International Symposium on Cluster Computing and the Grid (CCGrid’07), Rio De Janeiro, Brazil, 14-17 May, 2007.

  21. VO-Based Peering of CDNs • Initiator of VO-formation • Primary CDN • Participants • Peering CDNs • Explicit members • Primary and any peering CDN(s) • Implicit members • Content providers and end-users • VO types • Short-term on-demand • Long-term

  22. System Architecture

  23. VO Formation (a) Formation of a VO (b) A formed VO

  24. VO-Lifecycle: Flowchart Start Expansion of Shell VO Mi Generates Service Requirements Hotspot Generation Mi Passes Service Requirements to Local PA Shell VO Creation N Policies for Peering Exists? Sufficient Resources Acquired? Y Preexisting Policies Returned Y VO Formation Termination Condition(s) hold? N Short-Term Resource Negotiation Procedure Called Vo terminated

  25. VO Lifecycle 4

  26. Short-Term Resource Negotiation: Flowchart Start Auction Initiation Auction Termination Mi Calculates Payoff Value Bidding Mi’s of Peers Registers With Requesting Mi Auctioneer(s) Collects Bids from Bidders Mi Issues a Call for Bid to Local PA With Auction Policy Winner(s) Set is Returned Subset of Mi’s are Chosen as Distributed Auctioneers Winners are Selected Local PA Passes Call for Bid to Other PAs Bidders Calculate Bidding Function Winners are Paid the Second Lowest Price Each PA Passes Request to the PA

  27. Short-Term Resource Negotiation

  28. Negotiated SLA Components • Description of service requirements • Storage requirements, required rate of transfer (delay threshold), preference, expected duration of receiving service • Administration for VO activities • Specifies the role of the mediator as an authoritative entity • Renegotiation for problem resolution • Illustrates the steps to be undertaken in face of any problem in providing necessary services • Consequences of SLA violation • Range from imposing penalty through reimbursement of part of the revenues lost, to termination of peering relationship, and to disbanding and/or rearranging the VO • SLA bypassing conditions • For situations like the damage of physical resources due to natural disaster, theft etc.

  29. Policy Management to Support SLAs (a) Basic policy framework (b) Policy mapping

  30. Performance Gain Through Peering • Workloads are heavy-tailed in nature characterized by the function, • Service capacity follows Bounded Pareto distribution with p.d.f, , (a) Effectiveness of peering (b) Impact of request-redirection

  31. Three CDNs Peering Scenario • Expected waiting time, • Task variation = 1.5 • Smallest task = 1010.15 • Largest task = 10^7 • Service rate = 1/3000 • Single request stream

  32. Outline • Introduction • Problem statement • Possible solution • Research Goals • Work completed • Research Plan

  33. Research Plan • Evaluate the performance of developed economic model in a realistic simulation environment (Apr. 2007 – Aug. 2007) • Develop an effective load measurement and dissemination technique (Sept. 2007 – Feb. 2008) • Enforce a hierarchical approach to maximize utility of resources • Develop an effective request redirection strategy (Mar. 2008 – Jun. 2008) • Enforce a multiple level request assignment/redirection technique to make it transparent and efficient • Prototype system for peering CDNs (Jul. 2008 – Dec. 2008) • Leverage existing technologies and deploy on real-world testbed • Complete thesis and submission (Jan. 2009 – Mar. 2009)

  34. Thanks for your attention! • More information at: http://www.gridbus.org/cdn/

  35. Coordinated Management and Peering of Content Delivery Networks Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, Australiawww.csse.unimelb.edu.au/~apathan Al-Mukaddim Khan Pathan Supervisor: A/Prof. Rajkumar Buyya Masters to PhD Conversion Seminar

More Related