Autonomous Resource Provisioning for Multi-Service Web Applications

1. Autonomous Resource Provisioning forMulti-Service Web Applications Jiang Dejun,GuillaumePierre,Chi-Hung Chi WWW '10 Proceedings of the 19th international conference on World wide web

2. Agenda • Introduction • Related Work • Autonomous Provisioning • Evaluation • Conclusion • Comments

3. Introduction(1/5) • Major web sites such as Amazon.com and eBay • Are not designed as monolithic 3-tier applications but as a complex group of independent services querying each other • A service is a self-contained application providing elementary functionality • database holding customer information • an application serving search requests

4. Introduction(2/5) • To provide acceptable performance to their customers • providers often impose themselves a Service Level Agreement (SLA) • apply dynamic resource provisioning to respect the SLA target by adding resources when violating the SLA • removing resources when possible without violating the SLA

5. Introduction(3/5) • An essential question in resource provisioning of multiservice web applications • select which service(s) should be (de-)provisioned such that the whole application maintains acceptable performance at minimal cost. • This is a challenge because multi-service applications involve large number of components that have complex relationships with each other

6. Introduction(4/5) • Two possible approaches • models the entire application as a single queuing network • Too complex to capture all services relationships • assigns a fixed SLA to each service separately • May waste resources • Only the front-end service should be given an SLA • each service should be autonomously responsible for its own provisioning • by collaboratively negotiating its performance objectives with other services to maintain the front-end’s response time • “What-if analysis”

7. Introduction(5/5) • The authors show the system which can effectively provision resources to both • traditional multi-tier web application • complex multi-service applications

8. Related Work • Resource provisioning for single-tier [1,4] or multi-tier Web applications [7,10,12,14,15,17] • In [18], model work flow patterns within multiservice applications to predict future workloads for each services component • In [15,16], the works focus on when resources should be provisioned • But allocating new resources becomes much faster now

9. Autonomous Provisioning – System Model(1/5) • A service • a single-tier functional service with an HTTP or SOAP interface hosted in an application server • a single-tier data service with an SQL interface hosted in a database server • Within a multi-service application • Services are commonly organized as a directed acyclic graph • assume that the services of one application are not used simultaneously by other applications

10. Autonomous Provisioning – System Model(2/5)

11. Autonomous Provisioning – System Model(3/5) • assume that some machines are always available to be added to an application • Each resource can be assigned to only one service at a time • Such resource may be a physical machine or a virtualized instance with performance isolation

12. Autonomous Provisioning – System Model(4/5) Global Decision Negotiation Predict

13. Autonomous Provisioning – System Model(5/5) • Step 1: • each service carries out “what-if analysis” to predict its future performance • If the service was assigned an extra machine or removed one • Periodically send result to parent node • Step 2: • Intermediate node selects the maximum performance gain and minimum loss among the children nodes and itself • Finally: • Root node select which service(s) to provision when the SLA is (about to be) violated

14. Autonomous Provisioning – Performance Model(1/3) • use an M/M/n/PS queue to capture the performance of an n-core machine • Expected Response Time: • Rserver : the average response time of the service • n : the number of CPU cores assigned to service • λ: the average request rate • Sserver : the mean service time

15. Autonomous Provisioning – Performance Model(2/3) • A service may also use caches to offload some of the incoming requests from the service itself. • Adding caches potentially improves response time for two reasons • First, cache hits are processed faster than cache misses. • Second, the service itself and all children nodes receive less requests, and can thus process them faster

16. Autonomous Provisioning – Performance Model(3/3) • The performance model calculates the caching impact on the response time as follows • R(m) : the response time of the backend server across m CPU cores • Scache : the cache service time • ρn : the expected cache hit ratio with n nodes

17. Autonomous Provisioning – Model parameterization(1/5) • Most of the model parameters can be measured offline or monitored at runtime. • the request rate can be monitored by the administrative tools of application servers and database servers • The cache service time can be obtained by measuring cache response time offline • But expected cache hit ratio(ρn) and mean service time(Sserver) are harder to measure

18. Autonomous Provisioning – Model parameterization(2/5) • Expected cache hit ratio(ρn) • Using virtual caches • Stores only metadata such as the list of object in caches and their sizes • It receives all requests directed to the service and applies the same operations as a real cache with the same configuration would

19. Autonomous Provisioning – Model parameterization(3/5) • Mean service time(Sserver) • Previous research works measure the service time via profiling under low workload • But authors found that while workload increases, the prediction error rate become higher • To achieve acceptable prediction results, authors apply a classical feedback control loop to adjust the service time at runtime. • The system continuously estimates the service’s response time under the current conditions and compares the error between the predicted response time and the measured one.

20. Autonomous Provisioning – Model parameterization(4/5)

21. Autonomous Provisioning – Model parameterization(5/5) • Define a threshold as a configuration parameter • If the error rate exceeds the threshold, recomputed the service time • S’server : the corrected service time • Rserver : the latest measured response time • n : the number of current CPU cores • λ: the current request rate

22. Autonomous Provisioning – Resource Provisioning of service instances (1/3) • Each service reports performance promises to its parent on behalf of its children and itself: • it reports the best performance gain (loss) possible by adding (removing) a server to (from) a service of the subtree consisting of its children nodes and itself. • Assuming a service i has k immediate children services • Vi,J : the average number of service executions on service J caused by one request from service i

23. Autonomous Provisioning – Resource Provisioning of service instances (2/3)

24. Autonomous Provisioning – Resource Provisioning of service instances (3/3)

25. Autonomous Provisioning – Resource Provisioning of cache instances (1/3) • Provisioning cache instances is harder • it not only changes the performance of the concerned service, but also changes the traffic to its children, which in turn affects their performance • Each service periodically informs its children of the relative workload decrease (increase) it would address to them if it was given one more (one less) cache instance

26. Autonomous Provisioning – Resource Provisioning of cache instances (2/3) • To calculate expected traffic • EIR : expected invocation ratio = expected cache miss rate • Vi,j : the average number of service executions on service j caused by one request from service i • Wi : the request rate of node i • K : the number of predecessors in the graph

27. Autonomous Provisioning – Resource Provisioning of cache instances (3/3)

28. Autonomous Provisioning – Shifting Resource Among Services • In many cases, it can be more efficient to simply reorganize resource assignments within the application without retrieving machines from the resource pool • Vi,j may change due to an update in the application code or a change in user behavior • Shifting resource may be an oscillating behavior • To prevent it, one should define a performance threshold as the criterion for deciding whether to shift

29. Evaluation –Experimental Setup(1/3) • All experiments are performed on the DAS3 cluster at VU University Amsterdam. • The cluster consists of 85 nodes • a dual-CPU/dual-core 2.4GHz AMD Operon DP 280 • 4GB RAM • a 250 GB IDE hard drive • Nodes are connected with a 10Gbps LAN • the network latency between nodes is negligible • set the prediction error threshold for dynamically adjusting the service time to 3%.

30. Evaluation – Experimental Setup(2/3) • Author implement the local performance monitor on application server using the MBeanservletfrom JBoss. • The database server monitoring is based on performance data collected by the admin tool of MySQL. • Author developed the negotiation agent in Java using plain sockets.

31. Evaluation – Experimental Setup(3/3)

32. Evaluation – Model Validation(1/2) • Compare predicted values with the measured response times • using the “XSLT” and “Product” services from Figure 7(c) separately • Set the SLA of each service to a maximum response time of 400ms • Initially assign one server to each

33. Evaluation – Model Validation(2/2)

34. Evaluation – Comparison(1/2) • Comparison with “Analytic” • Figure 7(a) • A well known model • Set SLA of 500ms for the whole application • Analytic does not address multi-service application

35. Evaluation – Comparison(2/2) • Comparison with per-service SLA • Figure 7(b) • Set SLA to 500ms 12 req 16 req 23 req 12 req 25 req 16 req

36. Evaluation – Provisioning under varying load intensity(1/2) • Using figure 7(c) and 7(d) • Set SLA of 500ms • Workload first increases from 2 req/s to 22 req/s, then decrease back to 2 req/s

37. Evaluation – Provisioning under varying load intensity(2/2) 10 req 18 req 16 req 8req

38. Evaluation – Provisioning under varying load distribution(1/2) • Add workload to Service 2 and 3 at the same rate • At time 35, the workload of service 3 decrease while workload of service 2 increase as the same

39. Evaluation – Provisioning under varying load distribution (2/2)

40. Evaluation – Provisioning under varying load locality (1/2) • Define locality as the hit rate for a cache holding 10,000 objects • Increase workload until time 25 when the SLA was violated , and then changing the locality of service3

41. Evaluation – Provisioning under varying load locality(2/2)

42. Conclusion • The paper takes a different stand and demonstrates that provisioning resources for multi-service applications. • Which can be achieved in a decentralized way where each service is autonomously responsible for its own provisioning • Propose to give an SLA only to the front-end service • To author’s best knowledge, no other published resource provisioning algorithm can match or outperform their approach

43. Comments • The paper is using physical machines as resources • In virtual machine, we can dynamic set each VM’s CPU upper bound • It means that maybe we can make the cost less than the paper • The paper is more complex than what we want to do now, but there are something we can refer to • Something like the approach of adjusting the profiling

44. The End • Thanks