1 / 61

Introduction to Cloud Computing

Introduction to Cloud Computing. Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology The Netherlands.

yosef
Télécharger la présentation

Introduction to Cloud Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction toCloud Computing Alexandru Iosup Parallel and Distributed Systems GroupDelft University of TechnologyThe Netherlands Our team: Undergrad Tim Hegeman, Stefan Hugtenburg, Jesse Donkevliet … Grad Siqi Shen, Guo Yong, Nezih Yigitbasi Staff Henk Sips, Dick Epema, Alexandru Iosup, Otto Visser Collaborators Ion Stoica and the Mesos team (UC Berkeley), Thomas Fahringer, Radu Prodan, Vlad Nae (U. Innsbruck), Nicolae Tapus, Mihaela Balint, Vlad Posea, Alexandru Olteanu (UPB), Derrick Kondo (INRIA), Assaf Schuster, Mark Silberstein, Orna Ben-Yehuda (Technion), Kefeng Deng (NUDT, CN), ... SPEC RG Cloud Meeting

  2. What is Cloud Computing?3. A Useful IT Service “Use only when you want! Pay only for what you use!” Q: What do you use? Q: Why not this level? Q: Why not this level?

  3. Agenda • What is Cloud Computing? • IaaS Clouds, the Core Idea • The IaaS Owner Perspective • The IaaS User Perspective • Reality Check • Conclusion

  4. IaaS Cloud Computing VENI – @larGe: Massivizing Online Games using Cloud Computing

  5. Joe Has an Idea ($$$) MusicWave (Source: A. Antoniou, MSc Defense, TU Delft, 2012. Original idea: A. Iosup, 2011.)

  6. 10% … Solution #1 Buy or Rent • Big up-front commitment • Load variability: NOT supported (Source: A. Antoniou, MSc Defense, TU Delft, 2012. Original idea: A. Iosup, 2011.)

  7. Solution #2 • NO big up-front commitment • Load variability: supported Deploy on IaaS Cloud Q: So are we just shifting the problem to somebody else, that is, the IaaS cloud owner? (Source: A. Antoniou, MSc Defense, TU Delft, 2012. Original idea: V. Nae, 2008.)

  8. Inside an IaaS Cloud Data Center (Source: A. Antoniou, MSc Defense, TU Delft, 2012. Original idea: A. Iosup, 2011.)

  9. User C User B MusicWave Time and Cost Sharing Among Users (Source: A. Antoniou, MSc Defense, TU Delft, 2012.)

  10. Main Characteristics of IaaS Clouds • On-Demand Pay-per-Use • Elasticity (cloud concept of Scalability) • Resource Pooling • Fully automated IT services • Quality of Service

  11. Agenda • What is Cloud Computing? • IaaS Clouds, the Core Idea • The IaaS Owner Perspective:How to Deploy a Cloud? • The IaaS User Perspective • Reality Check • Conclusion

  12. IaaS Cloud Deployment Models Private On-premises Public Off-premises Hybrid (Source: A. Antoniou, MSc Defense, TU Delft, 2012. Original idea: Mell and Grance, NIST Spec.Pub. 800-145, Sep 2011.)

  13. Resource Sharing Models Grids IaaS Clouds Space-Sharing Time-Sharing Q: Which one is better? MusicWave MusicWave MusicWave OtherApp OtherApp OtherApp Host OS Host OS

  14. Virtualization Applications Applications Guest OS Guest OS Q: What is the problem? Virtual Resources Virtual Resources Q: What to do now? MusicWave OtherApp OtherApp VM Instance VM Instance Virtualization Host OS

  15. Virtualization and The Full IaaS Stack Applications Applications Applications Guest OS Guest OS Guest OS Virtual Resources Virtual Resources Virtual Resources VM Instance VM Instance VM Instance Virtual Machine Manager Virtual Machine Manager Virtual Infrastructure Manager Physical Infrastructure

  16. The Virtual Machine Lifecycle Q: Is this fair? (Source: A. Antoniou, MSc Defense, TU Delft, 2012.)

  17. Use Case: Amazon Elastic Compute Cloud (EC2) • Prominent IaaS provider • Datacenters all over the world • Many VM instance types • Per-hour charging

  18. Agenda • What is Cloud Computing? • IaaS Clouds, the Core Idea • The IaaS Owner Perspective • The IaaS User Perspective:How to Use Clouds? How to Choose Clouds? • Reality Check • Conclusion

  19. Workload MusicWave OtherApp OtherApp OtherApp Load = 4 OtherApp MusicWave Time RunTime= 6

  20. Sources: CNN, Zynga. Source: InsideSocialGames.com Use Case: Workloads of Zynga (Massively Social Gaming) Selling in-game virtual goods: “Zynga made est. $270M in 2009 from.”http://techcrunch.com/2010/05/03/zynga-revenue/ “Zynga made more than $600M in 2010 from selling in-game virtual goods.”S. Greengard, CACM, Apr 2011

  21. Use Case: Workloads of Zynga (Massively Social Gaming) • Load can grow very quickly Load

  22. Provisioning and Allocation of Resources Provisioning Allocation Load Time

  23. Provisioning and Allocation of Resources Q: What is the interplay between provisioning and allocation? Provisioning Allocation Load Time

  24. Provisioning and Allocation Policies Q:How many policies exist? Q: How to select a policy? Provisioning Allocation When? From where? When? Where? How many? etc. Load Which type? etc. Time (Source: A. Antoniou, MSc Defense, TU Delft, 2012.)

  25. Use Case:Two Provisioning Policies, Compared Startup OnDemand Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure-as-a-Service Clouds, (submitted). PDS Tech.Rep.2011-009

  26. Use Case:Two Provisioning Policies, Compared Metrics for comparison • Job Slowdown (JSD ): Ratio of actual runtime in the cloud and the runtime in a dedicated non-virtualized environment • Charged Cost (Cc) • Utility (U) Q: Charged cost vs Total RunTime? Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure-as-a-Service Clouds, (submitted). PDS Tech.Rep.2011-009

  27. Use Case:Two Provisioning Policies, Compared Workloads Increasing Bursty Uniform Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure-as-a-Service Clouds, (submitted). PDS Tech.Rep.2011-009

  28. Use Case:Two Provisioning Policies, Compared Environments Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure-as-a-Service Clouds, (submitted). PDS Tech.Rep.2011-009

  29. Use Case:Many Provisioning Policies, Compared Job Slowdown (JSD) Q: Why is OnDemand worse than Startup? A:waiting for machines to boot

  30. Use Case:Many Provisioning Policies, Compared Charged Cost (Cc) Q: Why is OnDemand worse than Startup? A:VM thrashing Q: Why no OnDemand on Amazon EC2?

  31. Use Case:Many Provisioning Policies, Compared Utility (U)

  32. Agenda • What is Cloud Computing? • IaaS Clouds, the Core Idea • The IaaS Owner Perspective • The IaaS User Perspective • Reality Check: Who Uses Public Commercial Clouds? • Conclusion

  33. The Real IaaS Cloud “The path to abundance” On-demand capacity Cheap for short-term tasks Great for web apps (EIP, web crawl, DB ops, I/O) VS Tropical Cyclone Nargis (NASA, ISSS, 04/29/08) http://www.flickr.com/photos/dimitrisotiropoulos/4204766418/ • “The killer cyclone” • Not so great performance for scientific applications (compute- or data-intensive) September 16, 2014 35

  34. (Source: http://www.cca08.org/files/slides/w_vogel.pdf)

  35. Zynga zCloud: Hybrid Self-Hosted/EC2 • After Zynga had large scale • More efficient self-hosted servers • Run at high utilization • Use EC2 for unexpected demand (Sources: http://seekingalpha.com/article/609141-how-amazon-s-aws-can-attract-ugly-economics and http://www.undertheradarblog.com/blog/3-reasons-zynga-is-moving-to-a-private-cloud/)

  36. Other Cloud Customers • 218 virtual CPUs • 9TB/2TB block/S3 storage • 6.5TB/2TB I/O per month (Source: http://markbuhagiar.com/technical/businessinthecloud/)

  37. Agenda • What is Cloud Computing? • IaaS Clouds, the Core Idea • The IaaS Owner Perspective • The IaaS User Perspective • Reality Check • Conclusion

  38. Conclusion Take-Home Message • Cloud Computing = IaaS + PaaS + SaaS • Core idea = lease vs self-own • On-Demand, Pay-per-Use, Elastic, Pooled, Automated, QoS • The Owner Perspective • Time-Sharing • Virtualization • The User Perspective • Variable workloads • Provisioning and Allocation policies • Reality Check: 100s of users http://www.flickr.com/photos/dimitrisotiropoulos/4204766418/

  39. Thank you for your attention! Questions? Suggestions? Observations? More Info: Alexandru IosupA.Iosup@tudelft.nlhttp://www.pds.ewi.tudelft.nl/~iosup/ (or google “iosup”)Parallel and Distributed Systems GroupDelft University of Technology • http://www.st.ewi.tudelft.nl/~iosup/research.html • http://www.st.ewi.tudelft.nl/~iosup/research_cloud.html • http://www.pds.ewi.tudelft.nl/ Do not hesitate to contact me…

  40. … And Now For Something Different • Ongoing projects in Cloud Computing at TUD

  41. Cloudification: PaaS for MSGs (Platform Challenge) Build Social Gaming platform that uses (mostly) cloud resources • Close to players • No upfront costs, no maintenance • Compute platforms: multi-cores, GPUs, clusters, all-in-one! • Performance guarantees • Hybrid deployment model • Code for various compute platforms—platform profiling • Load prediction miscalculation costs real money • What are the services? • Vendor lock-in? • My data

  42. Data, Data, DataUnderstanding the Behavior of Players in Online Social Games 871 460 32 198 127 73 1075 66 51 1004 240 596 (w/ E.Dias, A. Olteanu, F. Kuipers) Iosup et al. An Analysis of Implicit Social Networks in Multiplayer Online Games.IEEE Internet Computing

  43. Proposed hosting model: dynamic • Using data centers for dynamic resource allocation Massive join Massive leave Massive join • Main advantages: • Significantly lower over-provisioning • Efficient coverage of the world is possible Nae, Iosup, Prodan. Dynamic Resource Provisioning in Massively Multiplayer Online Games. IEEE TPDS 2011

  44. Resource Provisioning and AllocationStaticvs.DynamicProvisioning 250% Over-provisioning = WASTE (%) 25% Nae, Iosup, Prodan. Dynamic Resource Provisioning in Massively Multiplayer Online Games. IEEE TPDS 2011 [Source: Nae, Iosup, and Prodan, ACM SC 2008]

  45. Why Portfolio Scheduling? • Old scheduling aspects • Workloads evolve over time and exhibit periods of distinct characteristics • No one-size-fits-all policy: hundreds exist, each good for specific conditions • Data centers increasingly popular (also not new) • Constant deployment since mid-1990s • Users moving their computation to IaaS-cloud data centers • Consolidation efforts in mid- and large-scale companies • New scheduling aspects • New workloads • New data center architectures • New cost models • Developing a scheduling policy is risky and ephemeral • Selecting a scheduling policy for your data center is difficult • Combining the strengths of multiple scheduling policies is …

  46. What is Portfolio Scheduling? In a Nutshell, for Data Centers • Create a set of scheduling policies • Resource provisioning and allocation policies, in this work • Online selection of the active policy, at important moments • Periodic selection, in this work • Same principle for other changes: pricing model, system, … Deng, Song, Ren, Iosup: Exploring portfolio scheduling for long-term execution of scientific workloads in IaaS clouds. SC 2013: 55

  47. Performance Evaluation 1) Effect of Portfolio Scheduling (1) • Portfolio scheduling is 8%, 11%, 45%, and 30% better than the best constituent policy A portfolio scheduler can be better than any of its constituent policies Q: What can prevent a portfolio scheduler from being better than any of its constituent policies? Deng, Song, Ren, Iosup: Exploring portfolio scheduling for long-term execution of scientific workloads in IaaS clouds. SC 2013: 55

  48. Performance Evaluation 1) Effect of Portfolio Scheduling (2) • Job selection policies such as UNICEF and LXF that favor short jobs have the best performance Q: How well do you think a single (provisioning, job selection, VM selection) policy would perform? Will it be dominant? (Rhetorical) Deng, Song, Ren, Iosup: Exploring portfolio scheduling for long-term execution of scientific workloads in IaaS clouds. SC 2013: 55

More Related