1 / 26

Virtual Appliances and Education using Clouds

Virtual Appliances and Education using Clouds. Dr. Renato Figueiredo ACIS Lab - University of Florida. Background. Traditional ways of delivering hands-on training and education in parallel/distributed computing have non-trivial dependences on the environment

simone
Télécharger la présentation

Virtual Appliances and Education using Clouds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virtual Appliances and Education using Clouds Dr. Renato Figueiredo ACIS Lab - University of Florida

  2. Background • Traditional ways of delivering hands-on training and education in parallel/distributed computing have non-trivial dependences on the environment • Difficult to replicate same environment on different resources (e.g. HPC clusters, desktops) • Difficult to cope with changes in the environment (e.g. software upgrades) • Virtualization technologies • Remove key software dependences • Allow packaging and replication of hands-on, executable educational environments • Can be deployed, managed with cloud technologies

  3. Overview • Virtual appliances • Virtual networking • Grid appliances and FutureGrid • Support for classes in FutureGrid • Demo

  4. Guiding principles • Fidelity: activities should use full-fledged, executable software: education/training modules • Learn using the proper tools • Reproducibility: Creators of content should be able to install, configure, and test their modules once, and be assured of the same functional behavior regardless of where the module is deployed • Incentive to invest effort in developing, testing and documenting new modules

  5. Guiding principles • Deployability: Students and users should be able to deploy modules in a simple manner, and in a variety of resources • Reduce barriers to entry; avoid dependences upon a particular infrastructure • Interactive use • Community-oriented: Modules should be simple to share, discover, reuse, and expand • Create conditions for “viral” growth

  6. Role of clouds and portal • Executable modules – virtual appliances • Deployable on FutureGrid resources • Deployable on other cloud platforms, as well as virtualized desktops • Community sharing – Web 2.0 portal, appliance image repositories • An aggregation hub for executable modules and documentation

  7. What is an appliance? • Hardware/software appliances • TV receiver + computer + hard disk + Linux + user interface • Computer + network interfaces + FreeBSD + user interface

  8. Virtual appliance example • Linux + Apache + MySQL + PHP A web server Another Web server LAMP image instantiate Virtualization Layer copy Repeat…

  9. Educational appliances • A flexible, extensible platform for hands-on, lab-oriented education on FutureGrid • Support clustering of resources • Virtual machines + virtual networking to create sandboxed modules • Virtual “Grid” appliances: self-contained, pre-packaged execution environments • Group VPNs: simple management of virtual clusters by students and educators

  10. Virtual Networking • A single appliance encapsulates software and configuration • Cluster/Grid/Cloud computing • Middleware expects a collection of machines, typically on a LAN (Local Area Network) • Appliances need to communicate and coordinate with each other • Each worker needs an IP address, uses TCP/IP sockets

  11. Virtual cluster appliances • Virtual appliance + virtual network Virtual network Hadoop + Virtual Network Another Hadoop worker A Hadoop worker instantiate Virtual machine copy Repeat…

  12. Grid appliance in a nutshell • Plug-and-play clusters with a pre-configured software environment • Linux + (Hadoop, Condor, MPI, …) • Scripts for zero-configuration • Hands-on examples, bootstrap infrastructure, and zero-configuration software – you’re off to a quick start

  13. Virtual Network - GroupVPN • Setting up and managing typical VPNs can be daunting • VPN server(s), key distribution, NAT traversal • GroupVPN makes it simple for users to create and manage virtual cluster VPNs • Key insights: • Web 2.0 interface: create/manage user groups • All the complexity of setting up and managing VPN links is automated

  14. Deploying virtual clusters • Same image, different VPNs Group VPN Hadoop + Virtual Network Another Hadoop worker A Hadoop worker instantiate Virtual machine copy GroupVPN Credentials Repeat… (from Web site) Virtual IP - DHCP 10.10.1.1 Virtual IP - DHCP 10.10.1.2

  15. Support for classes on FutureGrid • Classes are setup and managed using the FutureGrid portal • Project proposal: can be a class, workshop, short course, tutorial • Needs to be approved by FutureGrid project to become active • Users can be added to a project • Users create accounts using the portal • Project leaders can authorize them to gain access to resources • Students can then interactively use FG resources (e.g. to start VMs)

  16. Use of FutureGrid in classes • Cloud computing/distributed systems classes • U.of Florida, U. Central Florida, U. of Puerto Rico, Univ. of Piemonte Orientale (Italy), Univ. of Mostar (Croatia) • Distributed scientific computing • Louisiana State University • Tutorials, workshops: • Big Data for Science summer school • A cloudy view on computing • SC’11 tutorial – Clouds for science • Science Cloud Summer School (this month)

  17. Cloud computing classes • Massimo Canonico, U. Piemonte Orientale • Difficulties to overcome: • Hardware issues: find enough free physical machines able to host virtual machines • Software issues: time to install/configure as many as possible different cloud platforms • University was not able to provide me the necessary hardware and software support • Students started to play with FutureGrid • After attending few lessons, they were able to start/stop virtual instance with several Cloud Computing platforms

  18. Cloud computing classes • Students used Eucalyptus, OpenStack and Nimbus • Half were not computer scientists. • As FutureGrid freely shares their physical machines and their cloud platforms, decided to freely share all materials of my class. • Hands-out, configuration files and link to useful documentation are available • https://portal.futuregrid.org/contrib/cloud-computing-class

  19. Cloud computing classes • Graduate-level “Cloud computing for Data-Intensive Sciences” (Judy Qiu, Fall 2010) • Virtualization technologies and tools • Infrastructure as a service • Parallel programming (MPI, Hadoop) • FutureGrid provided a set of software options that made it possible for students to work on different projects along the system stack.

  20. Term Projects Dryad/DryadLINQ #1 Matrix Multiplication (Swapnil,Amit,Pradnay) #2PhyloD (Ratul,Adrija,Chengming) Higher Level Languages Iterative MapReduce #3 LDA (Changsi, Yang) #4MemCache (Saliya, Yiming ,Jerome) #5 Avro (Yuduo, Yuan, patanachai) #6PageRank (Shuo-Huan,Parag) Cloud Platform Cloud Infrastructure #7 Nimbus, Eucalyptus (Stephen, Sonali, Shakeela) Cloud Infrastructure Cloud Storage #8 Cloud Storage Survey (Xiaoming, Nixiaogang) Hypervisor/Virtualization Virtualization #9 Hypervisor Performance Analysis Project (James , Andrew) (Slide courtesy of Judy Qiu)

  21. Big Data for Science Johns Hopkins Iowa State Notre Dame Penn State University of Florida Michigan State San Diego Supercomputer Center Univ.Illinois at Chicago Washington University University of Minnesota University of Texas at El Paso University of California at Los Angeles IBM Almaden Research Center 300+ Students (200 on sites from 10 institutes; 100 online) IU MapReduce and UF Virtual Appliance technologies are supported by FutureGrid. July 26-30, 2010 NCSA Summer School Workshop http://salsahpc.indiana.edu/tutorial Indiana University University of Arkansas (Slide courtesy of Judy Qiu)

  22. Demonstration • Deploying virtual appliance node on FutureGrid (Nimbus @ Alamo) • Connecting to virtual machine • Virtual networking • Running sample job

  23. Demonstration • Pre-instantiated VM to save us time: • cloud-client.sh --conf alamo.conf --run --name grid-appliance-2.05.03.gz --hours 24 • Connect to VM • ssh root@VMip • Check virtual network interface • ifconfig • Ping other VMs in the virtual cluster • Submit Condor job

  24. Uploading and sharing images • APIs available to upload images, customize, save, and share images • Community education pages are available • FutureGrid Web portal allows users to publish their own content • Tutorials, presentations on Web portal; VMs on image repositories

  25. Where to go from here? • Tutorials on FutureGrid and Grid appliance Web sites for various middleware stacks • Condor, MPI, Hadoop • A community resource for educational virtual appliances • Success hinges on users effectively getting involved • If you are happy with the system, let others know! • Contribute with your own content – virtual appliance images, tutorials, etc

  26. Questions? • More information: • BOF: tomorrow 4:45pm, Burnham (8th floor) • http://www.futuregrid.org • http://grid-appliance.org • This document was developed with support from the National Science Foundation (NSF) under Grant No. 0910812 to Indiana University for "FutureGrid: An Experimental, High-Performance Grid Test-bed." Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF

More Related