An SLA-aware load balancing scheme for cloud datacenters

1. An SLA-aware load balancing scheme for cloud datacenters 指導教授：王國禎 學生：黎中誠 國立交通大學資訊工程系 行動計算與寬頻網路實驗室

2. Outlines • Introduction • Related works • Proposed architecture • Dynamic weighted round-robin scheduling algorithm • Experimental environment • Experimental result • Conclusion • References

3. Introduction • Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility (like the electricity grid) over a network (typically the Internet) [1] The cloud scales: Amazon S3 growth [2]

4. Introduction (Cont.) • All user requests will be connected to the load balancer and then forward to appropriate server • The function of load balancing is to distribute the excess local workload evenly to the whole cloud federation, aims to realize a high ratio of user satisfaction and facility utilization in the cloud [3] • Improper allocation rules might cause the inefficiency of the system

5. Introduction (Cont.) • The following figure illustrates a general load balancer architecture

6. Introduction (Cont.) • In this paper we focus on two issues • load balancing control • centralized or decentralized • load balancing algorithm • static or dynamic

7. Related work (Cont.) Three-level centralized load balancer architecture [4] Centralized load balancer architecture means there is a manager which will receive every incoming request or be responsible for communication When system grows to exceed the capacity of manager, it will increase management cost

8. Related work (Cont.) Structure of Decentralized Load Balancer [5]

9. Related work (Cont.) Structure of Meta-Scheduler [5]

10. Related work (Cont.) Qualitative comparison of different load balancing architectures

11. Proposed Architecture We propose a new architecture to support dynamic load balancing in cloud data center There are two levels in our load balancer design: global balancer and local balancer Each global balancer connects with one local balancer that forms a virtual zone

12. Proposed Architecture (Cont.) • Local balancer • Monitoring local VM’s status which are in the same virtual zone • Choosing proper VM to handle request by scheduling algorithm • Global balancer • Global balancers connected to each other via P2P connection • When global balancer responsible area overload, it will redirect requests to other virtual zone

13. Proposed Architecture (Cont.) • Our proposed system architecture is shown as follows:

14. Proposed Architecture (Cont.) • The relationship between the modules is shown as follows:

15. Proposed Architecture (Cont.) • Request Handler • This module receives user requests and forward to the Request Scheduler module • Request Scheduler • This module assigns user requests from Request Handler to correspond VM based on the weight from Weight Adjustment module • Load Monitor • It will monitor current four load metrics (CPU, MEM, NET, Disk I/O utilization) of each VM which belong to this local balancer

16. Proposed Architecture (Cont.) • History Storage • When Load Monitor module collects load information, it will be stored in this module. These data can support Load Prediction module to predict the load at next time slot • Load Prediction • This module uses load history which is stored in History Storage module. This load prediction information will send to Weight Adjustment module

17. Proposed Architecture (Cont.) • SLA Requirement • This module lists the SLA requirements which are required by services. These SLA parameters can be used to provide Weight Adjustment module adjusting the weight of each • Weight Adjustment • This module will adjust the weight of each virtual machine which belong this local balancer according to load information from Load Monitor and prediction information from Load Prediction

18. Proposed Architecture (Cont.) • VM pool • There are some suspended VMs in VM pool. When current VM can’t handle incoming request, Request Handler will wake up some VMs from here to support providing services

19. Dynamic weighted round-robin scheduling algorithm We set up an experimental environment as shown below We use following formula to determine the weight of each machine

20. Dynamic weighted round-robin scheduling algorithm The critical resource is different when cloud datacenter provides varies services Load Monitor module collects four load metrics. These four load metrics are utilizations of CPU, memory, network bandwidth, and disk I/O Capacity index

21. Dynamic weighted round-robin scheduling algorithm Artificial Neural Network (ANN) has the availability of optimization and prediction We use delta learning rule in our neural network design (see Fig. 8) In our ANN design, we consider avoiding violation of the response time which is provisions in SLA.

22. Dynamic weighted round-robin scheduling algorithm The following figure shows schematic representation of the artificial neural network model

23. Dynamic weighted round-robin scheduling algorithm We train the weights via delta learning rule

24. Dynamic weighted round-robin scheduling algorithm • Transfer function: (Log-sigmoid) • : pre-reaction rate • :response time stipulated in the SLA • : actual response time • : learning rate • : input

25. Dynamic weighted round-robin scheduling algorithm Capacity index Neural index Weight

26. Dynamic weighted round-robin scheduling algorithm

27. Experimental environment (Cont.) We build a small test bed for our experiment, the load balancer as show in Fig. 8 These experimental results are based on real traffic rather than simulation Our experimental scenario is that there are three VMs with different capability which running apache web servers in a virtual zone We use the modified apache benchmark (ab) to collect experimental data

28. Experimental environment (Cont.) Our experimental environment and parameters is shown in the following table

29. Experimental environment (Cont.) Configuration capability of each VM  is shown in the following table

30. Experimental result We compare four weight adjustment schemes and list the results

31. Experimental result (Cont.) We compare with or without the VM pool module based on our proposed algorithm

32. Conclusions We propose architecture based on decentralized load balancer which is different from general centralized balancer Combination of system performance monitoring and neural network to adapt to any service Our proposed algorithm can achieve minimum response time in the limited resources This load balancing algorithm can avoid SLA violations via VM pool module

33. Conclusions (Cont.) The experimental results support that our proposed algorithm has faster response time than the other scheduling algorithms Therefor the proposed algorithm can handle more requests per second Since our scheduling algorithm is simple and efficient, it is well-suited for cloud computing environments to service a large number of requests with short response time

34. References [1] V. Nae, A. Iosup, and R. Prodan, "A Scheduling Strategy on Load Balancing of Virtual Machine Resources in Cloud Computing Environment“, in Parallel and Distributed Systems, IEEE Transactions on , 2010, pp. 380 - 395. [2] R. Suselbeck, G. Schiele, and C. Becker, "Towards a Load Balancing in a Three-level Cloud Computing Network," in Network and Systems Support for Games (NetGames), 2009, pp. 1 - 2. [3] Shu-Ching Wang, Kuo-Qin Yan, Wen-Pin Liao, and Shun-Sheng Wang, "A Load Balancing Mechanism Based on Ant Colony and Complex Network Theory in Open Cloud Computing Federation," in IEEE ICCSIT, 2010, pp. 108 - 113. [4] RajkumarRajavel, "A Comparative Study into Distributed Load Balancing Algorithms for Cloud Computing," in IEEE INCOCCI, Erode, 2010, pp. 419 - 424.