An Efficient Threshold-Based Power Management Mechanism for Heterogeneous Soft Real-Time Clusters

1. An Efficient Threshold-Based Power ManagementMechanism for HeterogeneousSoft Real-Time Clusters Leping Wang, Ying Lu University of Nebraska-Lincoln, USA September 4, 2014

2. Outline • Motivation • Related Work • Problem Statement • Threshold-based approach • Evaluation • Conclusion

3. Motivation • Why power management (PM) for heterogeneous clusters • The power-related costs dominate the total cost of ownership of a cluster system • Most PM mechanisms are applicable to homogenous systems • Heterogeneous clusters are already widespread

4. Motivation • Opportunities for PM in heterogeneous clusters • Turn off or hibernate idle servers • Dynamically scale operating frequency/voltage (DVS) for underutilized servers • Distribute more requests to power-efficient servers

5. New Challenges • Decide not only how many but also which cluster servers should be used to process current requests, when necessary • Identifying the optimal load distribution for a heterogeneous cluster is a non-trivial task

6. Related Work • PM in homogeneous systems • [Bianchini et al. 2004], [Bohrer et al. 2002], [Chase et al. 2001], [Chen et al. 2005], [Elnozahy et al. 2002], [Rajamani et al. 2003] • PM in heterogeneous systems • [Heath et al. PPoPP2005], [Rusu et al. RTAS2006]

7. Related Work • Current PM approaches for heterogeneous clusters • Search-based algorithms • Extensive performance measurements • Long optimization process •  high customization costs upon new installations, server failures, cluster upgrades or other changes

8. Goal and Components • Goal • Near-optimal power consumption • QoS (average response time guarantee) • Efficient algorithm • Three components • Vary-on/off • DVS with feedback control • Optimal workload distribution

9. System Model 1.CPU-bounded server clusters (e.g. web server cluster) 2.One front-end server 3.N heterogeneous back- end servers

10. Optimization Problem Cast the PM to an optimization problem • Objective: Minimize the total cluster power consumption J • QoS constraints: • Decisions on • Which servers should be used to process the current workload cluster , i.e., decide xi: 0 or 1 • How should the workload clusterbe distributed to active back-end servers, i.e., decide λi such that • According to i, back-end server set its CPU frequency fi

11. Power and Capacity Models : The ith server’s throughput : The ith server’s performance coefficient • Power Model • Capacity Model : Total power consumption : The ith server’s on/off state : The ith server’s constant power consumption : The ith server’s operating frequency : The ith server’s dynamic power consumption

12. Optimization Problem • According to the M/M/1 queuing model and our server capacity model, we have • To make , we know

13. Optimization Problem • The optimization problem is formed as follows • Minimize: Subject to:

14. Optimization Problem • No analytical method to get the closed-form solution on i and xi • Time complexity of search-based algorithm • Basic idea of our efficient PM • Use a heuristic method to decouple decisions on xiand i, then solve them separately to obtain near-optimal solutions.

15. Threshold-Based Approach • An efficient PM heuristic • Efficient offline analysis: • Divides the possible workload range into N sub-ranges • For each sub-range, the PM decisions are derived offline • Online execution: Periodically, • Front-end server: workload clusteris predicted and depending on the range cluster falls into, the corresponding PM decisions will be followed • Back-end server: applies DVS mechanism to decide fi

16. Offline Analysis • Order the heterogeneous back-end servers, i.e., generates a sequence, called ordered server list • Produce server activation thresholds 1, 2, … N such that if cluster  (k-1, k], it is optimal to turn on the first k servers of the ordered server list • Optimal workload distribution problem is solved for the N scenarios where cluster  (k-1, k], k=1, 2, …, N (time complexity: (N))

17. Offline Analysis • When cluster  (k-1, k], the first k servers of the ordered server list are turned on and the optimization problem becomes • Minimize: Subject to: Solution: the optimal workload distribution i

18. Algorithm • Our method, denoted as TP-CP-OP • Server Ordered ListOrder all back-end servers according to their Typical Power (TP) efficiencies • Server Activation Thresholds Consider both server Capacity constraints and Power efficiencies (CP) • Optimal Workload Distribution (OP)

19. Dynamic Voltage Scaling Feedforward M/M/1 Based Controller i fi errori Feedback PI Controller fi ith Back-end Server + + - Ri

20. Evaluation • A small cluster with 4 back-end servers • Continuous operating frequency ranged in (0, fi_max] • Discrete operating frequency levels in [fi_min , fi_max] • A large cluster with 128 back-end servers in 8 different types

21. Evaluation • Synthetic workload and Real Workload • Desired average response time is set at 1s • Evaluation metrics: average response time and power consumption • Each simulation lasts 3000s • Power management in every 30s

22. Evaluation • Baseline algorithms • Threshold-based approaches: AA−AA−CA, SP−CA−CA, EE-RT-HSC • Optimal power management solution OPT-SOLN obtained by a search-based algorithm

23. Evaluation • Average Response Time

24. Evaluation • Power Consumption

25. Conclusion • A efficient power management algorithm for heterogeneous server clusters • Mathematical models based • Minimum performance profiling • Workload threshold based • Low algorithm time complexity • Balance overhead and optimal solution • Fewer number of server on/off changes • Near-optimal power consumption

26. TechnicalReport • L. Wang and Y. Lu. Efficient power management of heterogeneous soft real-time clusters. Technical Report TR-UNL-CSE-2008-0004, University of Nebraska-Lincoln, 2008

27. Questions or Comments? ? Thanks! Leping Wang, Ying Lu

28. Evaluation Effect of Feedback Control

29. Evaluation Effect of Feedback Control