1. Intelligent Placement of Datacenters for Internet Services ÍñigoGoiri, Kien Le, JordiGuitart, Jordi Torres, and Ricardo Bianchini

2. Motivation • Internet services require thousands of servers • Use multiple “mirror” datacenters • High availability and fault tolerance • Low response time • Spend millions building and operating datacenters • Consume enormous amounts of brown energy

3. Datacenter construction costs • Each datacenter costs >$100M to construct • The smaller datacenters are rated at ~25MW • Examples: • Microsoft DCs in Virginia & Chicago: $500M each

4. Energy costs and carbon emissions Sources: [Qureshi’09], EPA

5. Intelligent Placement of Datacenters Goal: Manage the monetary and environmental costs • Define framework • Model costs and datacenter characteristics • Define optimization problem • Create solution approaches • Collect cost and location-related data • Create placement tool

6. Outline • Motivation • Placing datacenters • Evaluation • Conclusion

7. Selecting datacenter locations • Model datacenter placement • Network latencies • Availability

8. Selecting datacenter locations • Model datacenter placement • Network latencies • Availability • CAPEX costs • Distance to electricity and networking infrastructure • Land and construction (maximum PUE) • Power delivery, cooling, backup equipment • Servers and networking equipment

9. Selecting datacenter locations • Model datacenter placement • Network latencies • Availability • CAPEX costs • Distance to electricity and networking infrastructure • Land and construction (maximum PUE) • Power delivery, cooling, backup equipment • Servers and networking equipment • OPEX costs • Maintenance and administration • Electricity and water prices (average PUE)

10. Selecting datacenter locations • Model datacenter placement • Network latencies • Availability • CAPEX costs • Distance to electricity and networking infrastructure • Land and construction (maximum PUE) • Power delivery, cooling, backup equipment • Servers and networking equipment • OPEX costs • Maintenance and administration • Electricity and water prices (average PUE) • Incentives (taxes)

11. Selecting datacenter locations • Model datacenter placement • Network latencies • Availability • CAPEX costs • Distance to electricity and networking infrastructure • Land and construction (maximum PUE) • Power delivery, cooling, backup equipment • Servers and networking equipment • OPEX costs • Maintenance and administration • Electricity and water prices (average PUE) • Incentives (taxes)

12. Formulating the problem • Goal • Minimize CAPEX and OPEX • Constraints • Response times < MAX LATENCY for all users • Min consistency delay between 2 DCs < MAX DELAY • Min system availability > MIN AVAILABILITY • Output • Number of servers at each location • Minimum cost

13. Solving the (non-linear) problem • Linear Programming • Does not support non-linear costs • Brute force • Too slow • Simple heuristics • May not produce accurate results efficiently

14. Our approach for solving the problem • Evaluate each potential solution • Quickly via Linear Programming (LP) • Consider neighboring configurations • Simulated annealing (SA) • Cost optimization process • Combine SA and LP SA LP LP Current solution Near neighbor

15. Our approach for solving the problem SA LP LP $10.3M/month $13.8M/month SA SA LP LP $9.2M/month $10.7M/month

16. Summary of our approach • Generate a grid of tentative locations • Collect data about each location • Define datacenter characteristics • Instantiate optimization problem • Solve optimization problem

17. Tool demo • We built a tool that • Embodies the problem • Input data for the US • Multiple solution approaches Short video at: http://www.darklab.rutgers.edu/DCL/dcl.html

18. Outline • Motivation • Placing datacenters • Evaluation • Conclusion

19. Comparing locations for60k-server DC

20. Interesting questions • How much does… … lower latency cost? … higher availability cost? … faster consistency cost? … a green DC network cost? … a chiller-less DC network cost?

21. Cost of 60k-servergreen DC network Green DC network costs $100k/month more, except when latency <70ms

22. Cost of a 60k-serverchiller-less DC network Chiller-less DC network is cheaper but it cannot achieve low latencies

23. Conclusions • First scientific work on smart datacenter placement • Proposed framework and optimization problem • Proposed solution approach • Characterized many locations across the US • Built a tool to automate the process • Answered many interesting questions • Results show that smart placement can save millions • Work enables smaller companies to reap the benefits

24. Intelligent Placement of Datacenters for Internet Services ÍñigoGoiri, Kien Le, JordiGuitart, Jordi Torres, and Ricardo Bianchini