SeIUCCR Summer School September 2011

1. SeIUCCR Summer School September 2011 Agenda Day 1 Monday 12th Sept 2011 11:00am – 11:15am Introduction & welcome Claire Devereux, NGS/SeIUCCR, STFC Rutherford Appleton Laboratory (RAL) 11:15am – 11:45pm Introduction to the NGS, David Wallom, NGS Technical Director, University of Oxford 11:45am – 12:15pm Introduction to the SSI, Steve Crouch, SSI Software Guru, University of Southampton 12:30pm – 1:30pm Lunch 1:30pm – 2:15pm Introduction to cloud computing David Wallom, NGS Technical Director, University of Oxford 2:15pm – 3pm Eduservcloud Matt Johnson, Head of Research and Development, Eduserv 3pm – 3:30pm Coffee break 3:30pm – 4:30pm Making images & data storage in the Cloud Richard Tarrant, University of Reading 4:30pm – 5:30pm Introduction to Linux Andrew Richards, Associate Director - Operations and Services, Oxford e-Research Centre

2. An Introduction to Cloud Dr David Wallom, Associate Director - Innovation (Oxford e-Research Centre) Technical Director (UK NGS) VP-Community (OGF) Thanks to NIST Clouds Introduction

3. Outline • What is Cloud…? • Using Cloud (technically) • Using cloud (non-technical) • Nationally available resources

4. What is cloud?

5. A Working Definition of Cloud Computing • Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. • This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models. WallomsDef: If a user speaks to a person to get access to resources, its virtualisation, if the user gets access through a computational interface it’s a Cloud! Courtesy of NIST

6. 5 Essential Cloud Characteristics • On-demand self-service • High performance network access (not necessarily JANet quality though) • Resource pooling Location independence • Rapid elasticity/service scalability • Measured service/usage is accounted for Courtesy of NIST

7. Service Models of Cloud Computing: SaaS, PaaS, IaaS • SaaS: Software as a Service –> Google Apps, Force.com, Facebook, Microsoft Office Live; use deployed SaaS provider

8. Microsoft Azure Services Azure™ Services Platform Source: Microsoft Presentation, A Lap Around Windows Azure, Manuvir Das

9. Service Models of Cloud Computing: SaaS, PaaS, IaaS • SaaS: Software as a Service –> Google Apps, Force.com, Facebook, Microsoft Office Live; • PaaS: Platform as a Service–>Google App Engine, Azure Platform, Oracle Fusion; use deployed Application package PaaS provider

10. Microsoft Azure Azure™ Services Platform .NET PHP Python Ruby … Web Standards + Industry Standards Visual Studio and Eclipse

11. Service Models of Cloud Computing: SaaS, PaaS, IaaS • SaaS: Software as a Service –> Google Apps, Force.com, Facebook, Microsoft Office Live; • PaaS: Platform as a Service–>Google App Engine, Azure Platform; • IaaS: Infrastructure as a Service–>Amazon Web Services, NGS Cloud, Eduserv use instantiated OS image IaaS provider

12. Amazon AWS

13. 4 Deployment Models • Private cloud • enterprise owned or leased, e.g operated by your institutional Information Services • Community cloud • shared infrastructure for specific community, e.g. provided only to UK Universities, e.g. Eduserv (Swindon) • Public cloud • Sold to the public, mega-scale infrastructure, e.g. Amazon • Hybrid cloud • composition of two or more clouds, e.g. what it says on the tin! Courtesy of NIST

14. Common Cloud Characteristics • Cloud computing often leverages: • Massive scale (one research projects scaling) • Homogeneity • Virtualization • Resilient computing • Low cost software • Geographic distribution • Service orientation • Advanced security technologies Courtesy of NIST

15. The NIST Cloud Definition Framework Deployment Models Hybrid Clouds Service Models Community Cloud Public Cloud Private Cloud Essential Characteristics Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS) Massive Scale Resilient Computing On Demand Self-Service Homogeneity Geographic Distribution Common Characteristics High PerfNetwork Access Rapid Elasticity Virtualization Service Orientation Resource Pooling Measured Service Low Cost Software Advanced Security Based upon original chart created by Alex Dowbor - http://ornot.wordpress.com

16. Usage Models of Cloud

17. Private/Public Multiple Clouds • NGS cloud • Amazon cloud • Eduserv cloud • Users • Globally distributed; • different resources/cost; • different applications; • non standardised: different AAA and UI. • EGI cloud

18. Mediated Private/Public Multiple Clouds • Amazon cloud • NGS cloud • Management • Interface • Users • Automation; • load balancing; • costs reduction; • usability. • Eduserv cloud • EGI cloud

19. Hybrid Multiple Clouds • NGS cloud • Institutional cloud • Amazon cloud • Users • Eduserv cloud • Federation of Local and Global resources • Elasticity managed by local cloud not user • different resources/cost; • different applications; • non standardised: different AAA but single UI through private provider • EGI cloud

20. Migration Paths for Cloud Adoption • Use public clouds • Develop private clouds • Build a private cloud • Procure an outsourced private cloud • Migrate data centers to be private clouds (fully virtualized) • Build or procure community clouds • Organization wide SaaS • PaaS and IaaS • Disaster recovery for private clouds • Use hybrid-cloud technology • Workload portability between clouds

21. Using an IaaS • Users retains (full) control on: • operating system: • create, modify or use existing OS images; • VM instantiation and management (start, stop, #VMs); • networking: • elastic IP, virtual firewalls, isolation (security groups); • data: • create and manage EBS devices; • snapshotting. Great flexibility vs. extra effort

22. Cloud Infrastructure for Research Centralisation VsFederation Centralisation: one large, dedicated datacentre that serves the national HEI demand Federation: heterogeneous set of local infrastructures are coordinated nationally in order to satisfy the HEI demand Criteria for evaluation Funding Scalability Flexibility Maintenance Support Accountability Obsolescence Competitiveness Security

23. Client Tools Command Line Interface HybridFox RightScale Gems RightAws

24. Cloud Computing Security

25. Security is the Major Issue

26. Analyzing Cloud Security • Some key issues: • trust, multi-tenancy, encryption, compliance • Cloud security is a tractable problem • There are both advantages and challenges

27. General Security Advantages • Shifting public data to a external cloud reduces the exposure of the internal sensitive data • Cloud homogeneity makes security auditing/testing simpler • Clouds enable automated security management • Redundancy / Disaster Recovery

28. Cloud Security Advantages • Data Fragmentation and Dispersal • Dedicated Security Team • Greater Investment in Security Infrastructure • Fault Tolerance and Reliability • Greater Resiliency • Hypervisor Protection Against Network Attacks • Possible Reduction of C&A Activities (Access to Pre-Accredited Clouds) • Simplification of Compliance Analysis • Data Held by Unbiased Party (cloud vendor assertion) • Low-Cost Disaster Recovery and Data Storage Solutions • On-Demand Security Controls • Real-Time Detection of System Tampering • Rapid Re-Constitution of Services • Advanced HoneynetCapabilities

29. General Security Challenges • Trusting someone else'ssecurity model • Customer inability to respond to audit findings • Limitations in obtaining support for investigations • Indirect administrator accountability • Proprietary implementations can’t be examined • Loss of physical control

30. Cloud Security Challenges • Data dispersal and international privacy laws • EU Data Protection Directive and U.S. Safe Harbor program • Exposure of data to foreign government and data subpoenas • Data retention issues • Need for isolation management • Multi-tenancy • Logging challenges • Data ownership issues • Quality of service guarantees • Dependence on secure hypervisors • Attraction to hackers (high value target) • Security of virtual OSs in the cloud • Possibility for massive outages • Encryption needs for cloud computing • Encrypting access to the cloud resource control interface • Encrypting administrative access to OS instances • Encrypting access to applications • Encrypting application data at rest • Public cloud vs internal cloud security • Lack of public SaaS version control

31. Cloud Resources Available

32. UK NGS Cloud Activities • NGS Agile Deployment Environments EPSRC funded, 2 years, JISC 1 Year • Staff: • David Wallom (OeRC, Oxford); • David Fergusson (NeSC, Edinburgh); • Steve Thorn (NeSC, Edinburgh); • Matteo Turilli (OeRC, Oxford). • Goals: • EC2 compatible, open source solution; • development of a dedicated pool of images; • collecting data about feasibility, costs, stability; • identify use cases and gather further requirements.

33. NGS Cloud Prototypes Oxford IV 3 x 4 Xeon 6 core; 48GB ram. 2 x 1 Xeon 2 core; 32GB ram. • Ubuntu 10.10; • Ubuntu Enterprise Cloud; • 2+2 bounded public NICs on CC; • 12TB ECB, 12TB Walrus on SED disks; • TPM on every motherboard.

34. NGS Cloud Usage 2010/2011 • 106 registered users: uptake has been very fast and constant throughout the whole testing period; • 26 institutions: 23 HEI both universities and colleges, 3 companies; • 30 projects; • 10 research areas. Teaching Physics Ecology Geography Life sciences Medicine Social Science Mathematics Engineering Cloud R&D

35. Exemplar Case Studies • Evolutionary Genomics: “analysis and Information management of Next Generation Sequencing (NGS) of Genomic data poses many challenges in terms of time and size. We are exploring the translation of high quality NGS scientific analysis pipelines to make best use of Cloud infrastructure”; • Geospatial Science: “geospatial data is a mix of raster and vector data. As rasterizing is CPU-hungry process, and all maps displayed on the screen of the final user are rasters, it is more efficient to do the process on the server side. I am investigating how this process can be dispersed across many, if not unlimited instances in a cloud”; • Agent-based modelling of crime: “at the moment I have a tomcat server that hosts some web services used to run social simulation model, it needs access to the file system to run fortran scripts, create files etc. There are loads of problems with running our own server at uni and I think a virtual machine that I could have control over would be much better”.

36. How to get an account • Website • Innovation • Cloud@NGS • Provide the following to the NGS Support Centre: • a paragraph long description of how you intend to use our Cloud Prototype. • one or more fixed IP addresses from which you will want to access the Cloud. • Register at the following addresses (please register at both to gain access to the full infrastructure) https://cloud.oerc.ox.ac.uk:8443 • Access the portal and download your credentials zip-file. • Use a client to access the cloud resources.

37. Other Institutions • Looking to create UK federation of private cloud resources, starting with; • Edinburgh (previous NGS cloud pilot site) • Reading • Imperial • STFC RAL

38. Over to Eduserv