Internet Service Migration and Placement

1. Internet Service Migration and Placement Part 1 Instructor: Xiaodong Zhang Xiaoning Ding 11/08/2004

2. Outline • Background • OPUS: An Overlay Peer Utility Service • Overview • Architecture • Research issues • Model-based resource provisioning • Overview • Web service model • Model-based resource allocator

3. Outsourcing Services & Utility-based services Utility-based services • The service provider owns the infrastructure • leases the infrastructure to the customers • pay for what you use • Example: Internet data center enabling ASPs to deliver ASP services Outsourcing services • Customer-owned or leased system. • The service provider takes responsibility for managing the customer’s IT and network system – the computing infrastructure – based on customer-defined service level agreements (SLA). • Billed on a monthly or fixed-fee basis.

4. Utility & SLA • Utilities deliver IT resources (CPU, storage, and bandwidth) to hosted application and, ultimately, end users • much as the electric utility transparently delivers power on demand to customers. • Applications agree to Service Level Agreements (SLAs) with the utility

5. Static Provisioning • Dedicate fixed resources per application • Reprovision manually as needed • Overprovision for surges • High variable cost of capacity

6. Load Is Dynamic • ibm.com external site • February 2001 • Daily fluctuations • Workday cycle • Weekends off M T W Th F S S • World Cup soccer site • May-June 1998 • Seasonal fluctuations • Event surges (11x) • ita.ee.lbl.gov Week 6 7 8

7. Adaptive Provisioning offer economies of scale • Network access • Power and cooling • Administration and security • Surge capacity

8. Overlay network and Mobile code • Increasing number of important network services are deploying overlays • CDN, Replicated services, Storage systems... • Dynamically map data and functions onto network resources • Programs and data will adaptively migrate and replicate in response to changing network conditions, client access characteristics,... • Programs dynamically run at optimal network locations • Data dynamically flow to where it is required.

9. Outline • Background • OPUS: An Overlay Peer Utility Service • Overview • Architecture • Research issues • Model-based resource provisioning • Overview • Web service model • Model-based resource allocator

10. OPUS: An Overlay Utility Service Peering Overlay node App demand (per network region) Allocate nodes to services based on current demand

11. OPUS: Overview • targeting utilities consisting of a distributed set of thousands of server sites, each with potentially 1000's of individual machines, cooperating together to fulfill aggregate SLAs • Simultaneously hosts multiple distributed applications • replicated web services • application-layer multicast • content distribution networks. • ...

12. Opus tasks • Resource allocation • Allocate resources among competing applications • Maximize aggregate performance • Based on changing application and network characteristics, SLAs • Replica placement • Closely related to resource allocation • Where to place individual application replicas • Consider dynamically changing client access patterns, network failures, etc.

13. Opus tasks • Overlay topology construction • create overlays that meet application requirements of performance, delay, and reliability • minimize consumed network resources • Request routing • discover the service replica capable of delivering the highest quality of service

14. OPUS: Architecture

15. The service overlay • Each Opus site runs an instance of site manager coordinating resource usage at that site and exchange status summaries with other opus sites. • Interconnects all active nodes and provides overlay services • “Backbone” for coordinated, decentralized resource allocation and resource control

16. The service overlay • Assist the construction and maintenance of application overlay • Dynamic and self-healing • Scalability issue • Hierarchical data dissemination in dicast • Think globally but act locally

17. Adaptive per-application overlay • Each application uses its application overlay to • Route internal application traffic • Disseminate content • Synchronize state information • … • The topology and site allotments are subject to change by resource allocator

18. Security and isolation • Allocating resources to applications at the granularity of individual nodes • Future plan: using virtual machine • Using VLAN to isolate traffic on the wire

19. Research Issues • Overlay topology construction • Resource allocation • Scalable tracking of system characteristics • Reliability QoS guarantees

20. Overlay topology construction • Emphasize scalability • Quantify the benefits of competing structures • Develop scalable distributed constructing algorithms • Initial work • A general overlay topology that enables dynamic tradeoffs between network performance/reliability and cost • Focus on network cost and relative delay penalty (RDP) to characterize overlay topology • Two candidate overlay topologies: K-spanner and LAST.

21. Overlay topology construction

22. Overlay topology construction • Distributed algorithms for building and maintaining the topology • Selectively probing using probabilistic techniques and hierarchy • Using partial, approximate and probabilistic knowledge of network infomation • Having each node gradually migrate to its “proper” location in the overlay.

23. Resource allocation classical economic model • Customers are associated with utility functions specifying the value of the services result from a allotment. (concave functions) • Opus maximizes global value across all applications. • Optimal solution: the marginal value of an additional resource unit is in equilibrium across all customers.

24. Resource allocation App2 Throughput (Value) Gradient 2 App1 Gradient 1 Allocated Resources

25. Resource allocation • Scalability consideration • Adapt from economic resource allocation • Decentralized federation of autonomous local “markets” exchanging information to converge toward a global equilibrium • Celluar structure • Cell: an entire Opus site or a portion of large site • Cells plan their internal allocation locally • Cells operate to trade load or resources

26. Tracking system characteristics • Nodes are partitioned into clusters of size d. • Each cluster elects an agent responsible for disseminating local cluster information • Agents from d adjacent clusters form second-level clusters • All nodes are organized into a tree called dicast tree. Height=logdN

27. Tracking system characteristics Hierarchical data dissemination in dicast

28. Tracking system characteristics • Data travels up the tree, and may be aggregated with data from the nodes • At each level of the tree, an overlay propagates the data among all participating cluster members • Updates are buffered awaiting the arrival of further updates until a threshold is reached, and updates are aggregated • Each node may has • exact information of “nearby” nodes in the same cluster • Aggregate information of remote cluster

29. Reliability QoS Guarantees Address network level failures • Restricted flooding • Redundantly transmit the same data over multiple logical path • Minimizing the overhead • Intermediate nodes re-evaluate the reliability of the remainder of the path, and choose between forwarding redundant data and suppressing duplicate data

30. A 0.98 0.96 0.99 B J D S 0.97 0.97 Reliability QoS Guarantees SAD: 0.96*0.98*0.99=0.931 SAD: 0.97*0.97*0.99=0.931 SA and BD: (1-(1-0.96*0.98)*(1-0.97*0.97)) *0.99=0.987

31. Reliability QoS Guarantees • To match the overlay topology with the failure characteristics of underlying network • Construct overlays with disjoint paths to lower the failure correlation among logical overlay links • Collect statistical information about loss correlation • Use network topology information

32. Outline • Background • OPUS: An Overlay Peer Utility Service • Overview • Architecture • Research issues • Model-based resource provisioning • Overview • Web service model • Model-based resource allocator

33. Overview • Addresses the provisioning problem • Multiple competing services hosted by a shared server cluster (utility) • How much resource does a service need to meet SLA targets • Applications • Static web content • Heavily resource-intensive • Predictable in average per-request resource demands

34. System architecture

35. The use of the model

36. Web service model

37. Web service model

38. Web service model

39. Model-based resource allocator • Periodically invoked by the utility OS executive to adjust the allotments • Focus on memory and storage resources, ignore CPU constraints • Output • an allotment vector for each service • CPU share,Memory and storage allotment [M, φ]

40. Model-based resource allocator • Resource provisioning primitives • Candidate plans initial candidate allotment vectors • LocalAdjust modifies a candidate vector to adapt to local resource constraint or surplus • GroupAdjust modifies a set of candidate vectors to adapt to a resource constrait or surplus

41. Model-based resource allocator Generating Initial Candidates Ρtarget Rp Φ=μs H M (1) |Φ-Φdesired|<ε Rp, Φ, ρtarget Rs (4) λ,H λs (2) Φ=λs / ρtarget

42. References • Utility Computing White Paper: http://www.sun.com/service/utility/FINAL_UC_WP.pdf • Service Utilities: http://issg.cs.duke.edu/utilies.html • D. G. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris, "Resilient Overlay Networks," in 18th ACM Symposium on Operating Systems Principles (SOSP), October 2001, pp. 131-145. • "OPUS: Overlay Utility Service", Rebecca Braynard, Dejan Kostic, Adolfo Rodriguez, Jeff Chase and Amin Vahdat, poster at 18th ACM Symposium on Operating System Principles (SOSP), Banff, Canada, October 2001. ( poster) • R. Braynard, D. Kostic, A. Rodriguez, J. Chase, and A. Vahdat. Opus: an Overlay Peer Utility Service. IEEE OPENARCH 2002. • Ronald P. Doyle, et. al., ``Model-based resource provisioning in a Web service utility", Proceedings of the 4th USENIX Symposium on Internet Technology, 2003.