Optimal Sleeping in Datacenters

1. Optimal Sleeping in Datacenters Dana Butnariu Princeton University EDGE Lab June – September 2011 Joint work with Professor Mung Chiang, IoannisKamitsos, Sangtae Ha and JasikaBawa

2. Background • User/Client = Person using an internet service • Request = Network service needed by the client • Server/Core = Physical device which handles client requests • Data center = Multiple servers grouped together in one location • Delay/Latency = Time elapsed between issuing a request and receiving a reply • Job = Form in which the client request is perceived at data center level • Controller = Physical device which controls the behavior of servers inside a data center • Thread = Processing unit available inside servers, responsible for handling client requests

3. Problems and solutions • Problem 1 => Servers in data centers are either underutilized or idle for long periods of time. • Effect => A lot of energy is wasted by these servers. • Simple solution => Put servers to sleep when they are not used. • Problem 2 => Putting servers to sleep too often can lead to high switching costs. • Problem 3 => Sleeping and waking up servers takes time. • Effect 1 => If an optimal algorithm is not used more energy may be consumed by switching server state. • Effect 2 => User requests will be handled with high delay. • Better solution => Use math and optimization theory to come up with an optimal server sleeping pattern.

4. Optimality • Want optimality in terms of the tradeoff between energy consumption and request handling delays. • Mathematical model designed by members of the EDGE Lab: • PCPU: Power consumption due to active core. Qi: Queue occupancy at state i. • Ech: Switching cost. Wi: Optimal policy at previous state.

5. Theory and Practice • Theoretical model offers good results in terms of the tradeoff between energy savings and delay. • Problem 1=> Theory is just a prediction, not a fact. Will the theoretical model results be confirmed by experimental results? • Problem 2 => Experiments in large data centers are hard to carry out, supervise and control. • Simple solution => Build a small scale replica of a data centers inside the EDGE Lab. • Use laptops to simulate servers and a central controller. • Use power meters to measure laptop energy consumption. • Use a multithreaded model for the central controller to simulate datacenter operations and interactions.

6. Experimental Setup

7. Experimental Setup • Controller: • Generates jobs. • Enqueues and dequeues jobs. • Implements the optimal sleeping policy. • Turns on and puts to sleep servers. • Dispatches jobs to servers according to the sleeping policy. • Servers: • Handle job requests represented by video clips. • Have a video encoding component and a video display. component for each job handled. • Are turned on and go to sleep based on the central controller input.

8. Experimental Results – Energy savings vs load analysis • When no idling is assumed, energy savings fall within 1-7% of theoretically predicted ones for all traffic loads. • Energy savings naturally fall with idlingas idling assumes that servers are not put to sleep, but instead left on when no job is running.

9. Experimental Results – Comparison with Base Policies • Compared with baseline policies, the sleeping policy performs better in terms of energy savings. • Baseline policy 1 => Number of on servers = Number of jobs in queue. Baseline policy 2 => Number of on servers = Number given by a certain job threshold.

10. Experimental Results – Multiple server performance • Each server consumes the same power and processes jobs at the same speed. • Moving from 1 to 8 servers, average delay decreases 4 times, but energy consumption increases only 1.6-1.7 times, giving a good energy consumption-delay tradeoff.

11. Experimental Results – Job splitting across servers • Used c servers to simulate the performance of one single server which is c-times faster and consumes c-times more energy in order to handle a job. • Diminishing returns from increasing number of servers => 4 servers sufficient to get most of the resource benefits.

12. Conclusions • New solution : • to lower the increasing energy usage in datacenters. • which provides an optimal tradeoff between energy savings and delay in handling client requests. • for which both theory and experimental results show that a balance between energy and delay is easily achievable. • Experimental results: • confirm theoretical predictions. • show that energy savings are substantial for low loads (70-80%) and moderate for high loads (30-40%). • prove that the optimal server sleeping policy behaves better in terms of energy savings than common baseline policies. • show that even as delay increases by a moderate amount, energy consumption remains almost constant.

13. Acknowledgements • Professor Mung Chiang, ELE Department. • EDGE Lab members IoannisKamistos and Sangtae Ha. • Undergraduate ELE major JasikaBawa. • All other members of the EDGE Lab. Questions?