Efficient Resource Provisioning in Compute Clouds via VM Multiplexing

1. Efficient Resource Provisioning in Compute Clouds via VM Multiplexing Xiaoqiao Meng, Canturk Isci, Jeffrey Kephart, Li Zhang, Eric Bouillet, Dimitrios Pendarakis IBM T.J. Watson Research Center

2. Resource provisioning in cloud • Cloud creates an illusion of “infinite” pool of resources • Cloud manages resources via creating and consolidating Virtual Machines • Cloud resource provisioning: decisionon VM size and VM placement • Efficiency of resource provisioningmeasured by overall resource utilization Virtual machine pools Servers Networkequipments Storage

3. Peak-load as VM size Chance for capacity violation < 1% VM sizing • Static: fixed size upon VM creation • Easy to manage • Lack of elasticity in resource reuse • Dynamic: size adapts to demand • High resource utilization • Avoiding over and under-provisioning is challenging CPU utilization for a VM instance in 4 months

4. Traditional VM placement VM size Physical host capacity VM 1 VIRTUALIZATION VM 2 VM 3 VM 4 • Placement is done separately from sizing • VM size is fixed regardless of where it is placed • Target: minimizerequired physical servers or reduce energy cost • Formulated as a multi-dimensional vector packing problem

5. Outline • VM multiplexing • Related work • Design • Performance constraint • VM selection • Joint-VM sizing • Applications • Summary

6. s3 s2 s1 We expect s3 < s1 + s2. Benefit of multiplexing ! VM workload multiplexing • Multiplex VMs’ workload on same physical server • Aggregate multiple workload. Estimate total capacity need based on aggregated workload • Performance level of each VM be preserved Separate VM sizing VM multiplexing

7. Example for VM multiplexing Traditional method: Provision VMs separately. Total capacity need= 1.04 cpu Alternative method: Consolidate and provision three VMs as a whole Total capacity need= 0.67 cpu Gain from statistical multiplexing!

8. Potential Improvement by VM multiplexing • Test on servers in a Global Hosting Services • 1325 physical hosts, 15000 VMs • Based on 3-month CPU/memory utilization data Average capacity saving per physical server = 39%

9. Related work • Commercial capacity planning tools consider each VM’s need in isolation • IBM WebSphere CloudBurst, VMware capacity planner, Novell PlateSpin Recon, Lanamark Suite • Some prior work consider inter-VM interaction and compatibility to improve resource provisioning • Hotcloud’09: co-place VMs not contending for same resource type • OSDI’08, Hotcloud’09, VEE’09: co-place VMs with memory sharing • MASCOTS’09. Eurosys’09: statistical multiplexing VM to save power • NOMS’08: VM consolidation by formulating an optimization problem

10. Design of enabling VM multiplexing • Describe VM performance constraint • Application performance as a function of VM capacity • Construct super-VM by multiplexing VMs • Find VM combinations with complementary workload • Joint-VM sizing • Determining total capacity needs for super-VM 2 cpu 3 cpu Sizing super-VM 1 8 7 5 1 8 7 5 1 2 3 4 6 3 2 4 6 3 2 4 5 6 7 8 Describe VM performance constraint 1 cpu 1.5 cpu Construct super-VM by multiplexing VMs • VM Consolidation • Place super-VM on host • Standard VM placement techniques plugged in

11. VM performance constraint Workload Size c Performance constraint on VM size Number of intervals with size violation < β Total interval number time T • Basis for computing VM size • Parameters β, T adapt to various application types • Small T and β for time-sensitive, critical applications • Large T and β for time-elastic, long-running background jobs • T=0, β=0 corresponds to peakload-based sizing

12. Given n VMs with individual , Let If super-VM size satisfies , each VM must satisfy individual Performance constraint with VM multiplexing Choose the most stringent performance constraint Important feature : Total capacity need for super-VM without decreasing individual VM performance easily derive Knowing individual VM’s performance constraint

13. Construct super-VM • Key for multiplexing: complementary workload patterns • Approximately measured by correlation coefficient • Solution • Greedy search • Recursively find VM pairs with most strong negative correlation • Clustering • Pairwise distance measured by correlation coefficient Complementary workload! Strong negative correlation Not-so-complementary workload! Strong positive correlation

14. Greedy search • Three-step process • Find the VM pair (i,j) giving the highest correlation coefficient. Output (i,j) as a candidate for joint provisioning • Remove VM i and j • Repeat Step 1 and 2 until all input VMs are removed Correlation coefficient matrix

15. 3. Determine total size by super-VM performance constraint Super-VM sizing 1. Aggregation 2. Workload forecasting

16. Workload forecasting • Decouple individual VM’s workload into fluctuating and regular components. Forecasting limited to aggregated fluctuating workload • Standard timeseries forecasting methods are applied. Choose the one with smallest forecasting error based on historical data

17. Modeling forecast error • Modeling forecasting error to compensate for workload variability • Applied to any forecasting method • With explicit error model • Compute error statistical distribution by training error model with historical data • Without explicit error model • Use Kernel-Density-Estimation (KDE) to estimate error statistical distribution by historical data

18. Application to VM consolidation • Tested on performance data from four cloud hosts • Use First-fit-decreasing for VM consolidation • Performance gain • With VM multiplexing, 28%-62% less physical hosts required compared to no-multiplexing

19. Consolidation with various perf. constraint • Assume each VM has same performance constraint • Observations: • Maximum extra capacity saving is 45% for T=0, β=0 • Correspond to peakload-based sizing • Saving shrinks when β increases • Workload dynamics make less different in terms of VM capacity violation • With forecasting, VM consolidation slightly more aggressive • Lead to higher actual β than expected on certain physical hosts • Overall, 88%-96% of hosts have actual β than expected

20. Application to providing VM resource guarantees • Apply VM multiplexing to providing resource guarantees • Define “joint reservations” instead of individual VM reservations • Enforce joint reservations with the “resource pool” abstraction:

21. Application to providing VM resource guarantees • 16%-75% of more VMs admitted by enabling joint-reservations. Gain subject to performance constraint • Higher gain for more stringent performance constraint

22. Summary • Multiplex VMs with complementary workload patterns to save more overall resource • Three design components • Performance constraint • VM selection • Joint-VM size estimation • Enormous capacity saving in applications

23. Thank you! Q & A

24. Modeling forecast error • Modeling forecasting error Workload Size c time