1 / 29

Nereus Desktop Cloud

Nereus Desktop Cloud. Massively Parallel Computing. Dr Rhys Newman Chief Architect, IGD Group Oxford University. iVEC Seminar, Dec 7 th 2009. Cloud Computing Hype. “Grid Computing” coined in 1999, and became “Cloud Computing” in 2007 Both overhyped….

vlad
Télécharger la présentation

Nereus Desktop Cloud

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nereus Desktop Cloud Massively Parallel Computing Dr Rhys Newman Chief Architect, IGD Group Oxford University iVEC Seminar, Dec 7th 2009

  2. Cloud Computing Hype “Grid Computing” coined in 1999, and became “Cloud Computing” in 2007 Both overhyped…. Grid technology entered Gartner’s “Trough” in 2007, now Cloud computing at the peak of expectations….

  3. What is a Global Desktop Cloud? There are over 1 billion PCs in the world, mostly on the desktop Most are idle most of the time (90%+) At least 30% (and growing) have DSL connections to the internet Desktop cloud technology is the software which enables secure, standardised access to this vast amount of under-used computing power

  4. Desktop Clouds are Green IT PC Manufacture is 50% of its lifetime CO2 footprint Use the cloud? Cooling is a 30-100% overhead Datacentres have a large CO2 footprint Just exports the environmental impact Desktop Cloud Technology! Uses existing computers No extra cooling or housing costs Fewer servers in datacentres  less CO2 Plenty of idle time even when being “used” Making productive use of desktop idle time compensatesfor the environmental impact of a PC

  5. Global Desktop Cloud Potential BOINC is largest example with 550,000 computers Runs over 30 world-leading (famous) research projects Releases more computational power than the worlds fastest supercomputer. But… has only captured 0.05% of the global potential! Barriers are still too high… GB Ethernet Linux 1Billion desktop PCs = 200 x (Top 500 supercomputers combined)  $100 Million worth of CPU time wasted per day! “Vanilla” computing power ever more in demand… Analysis from Top500.org (Nov 2008)

  6. Technical Barriers? We all know “Moore’s Law”…. Gilder's Law: "bandwidth grows at least three times faster than computer power."

  7. Some Academic Projects….

  8. Human Barriers! Security PC owners are rightly scared of executing unvetted code Convenience Installing software can be difficult Hardware / OS variations make support difficult Extra software  extra maintenance Usability Develop/support code for a wide range of HW/SW Scalability Can the system grow to millions of clients?

  9. Desktop Cloud Middleware Checklist – What do you need? Security foreign code must be sandboxed to prevent illegal access Reliability failures must be managed silently or reported usefully Homogeneity developers should not need to manage the issues of heterogeneous software/hardware Ease of Use developers should not need a PhD in computing to use it! Privacy host should not be able to deduce or know what the “job” is doing in the background Scalability all of the above must scale to the millions….

  10. NereusV – Massively Parallel Computing NereusV - 5th Version of a 5-year research project at Oxford University Open source (GPL v2)  free for commercial and academic use Extensively tested and now deployed around the world Licensed commercially too, with growing interest amongst large SI vendors

  11. NereusV – Pure Java Desktop Cloud Security Uses the Java Applet Sandbox which has run unvetted code in browsers for over a decade Reliability Uses standard HTTP and the industry standard JVM supported by all HW and SW vendors Homogeneity Java is inherently cross platform – the commonest software platform on the planet! Ease of Use If you can write an Applet, you can use NereusV Privacy All code buried inside the JVM. No local storage so no “left over” files to look into ScalabilityNereusV network built as a forest of trees, one of the few structures known to scale globally (eg: email, DNS, Google)

  12. Web Servers NereusV Network Architecture User’s Server Nereus Server 1 User’s PC Nereus Server 3 Nereus Server 2 Nereus Applets (1000-10000 per server)

  13. Medical Image Processing • Changes in the retina indicates progress of eye disease • Diabetes sufferers particularly susceptible • Take images every few months and look for changes, but… • The retina will be photographed in different positions, by different cameras and operators • Some changes are just dust on the camera lens • Examining images is skilled work $$$ • Use computing to “brute force” through pixels and detect changes • 1000s CPU hours needed

  14. Nereus Demo #1 Start up an Application server Push code out to Nereus Clients Collect the Results!

  15. Nereus Cloud Internationally (Thanks to Aidan Hotan at Curtin) www.galafrey.org / gdg-nereuscpu3.physics.ox.ac.uk www-physics.ox.ac.uk nereus.ox.ac.uk nereus.cira.curtin.edu.au www.nereuscloud.net

  16. Demo #2 – Financial Modelling • Look for patterns in financial time series • Predicting outcomes better than 50/50 makes money • Hedge Funds pay a lot of smart people a lot money to do this • Can we do better with dumber algorithms but with loads of computing power?

  17. Medicine and Astrophysics • NereusV for Molecular modelling • Atomic level simulation of protein agglomeration in Alzheimer's research • Results published in Physical Review E* • Astrophysics sky surveys • Not quite SETI@Home - looking for pulsars rather than ET! • Like SETI@Home there’s a lot of data which can be processed in parallel * C.F. Lee, J. Loken, L. Jean and D.J. Vaux (2009) Elongation dynamics of amyloid fibrils: a rugged energy landscape picture. Physical Review E 80, 041906. E-print: arXiv:0904.2505.

  18. NereusV + JPCRunning Native Code JPC Launched - JavaOne 2007 JPC Improved - JavaOne 2008 • NereusV only runs Java code… …and then only within the Applet sandbox • What about legacy code, most often x86 Linux? • Use JPC: a pure Java x86 PC emulator….

  19. JPC – Pure Java x86 PC Emulator PC OS PC emulator Java VirtualMachine PC Operating System PC hardware

  20. JPC:Removes Hardware/OS Dependence Super-secure Sandbox for untrusted code PC OS PC emulator Java VirtualMachine Java VirtualMachine Java VirtualMachine Java VirtualMachine STB Operating System PC Operating System Mobile phone OS PC Operating System PC hardware Set top box Mobile phone

  21. JPC – Java x86 Hardware Emulator CD-ROM Drive Interrupt Controller Network PCI ISA Bridge PCI Host Bridge PS/2 Interface Keyboard DMA Controller Interval Timer PCI Bus Mouse Processor VGA Graphics VGA BIOS Real Time Clock Floppy Drive MMU IDE Interface Memory BIOS Floppy Controller Hard Disk Drive

  22. JPC - Current Status • JPC Peaks at 20% Native Speed • Dynamic compilation • No need for signed applet • Hardware level snapshots • Virtual PC is “Instant On” wherever downloaded • Instant hardware-level pause/resume • Boots DOS and modern Linux distros

  23. OS Name Desktop Prompt Not Booting DSL DELI Feather Open BSD TTY Ubuntu 8.10 Kernel Panic, SMP issue? Puppy Kernel Panic, fatal exception in interrupt Knoppix Kernel Panic, fatal exception in interrupt JPC - OS Support • Currently testing with 23 Linux distributions including:

  24. Multi-GB Disk Images? Disk Image Data (read only) Webserver • Store disk images on a server • Data loaded over the network on demand • Writes are cached locally • Better load time and memory footprint • Use HTTP 1.1 range header to request parts of the Disk image • Remote HD images can be served by any standard webserver e.g. Amazon…. • The guest OS is unaware of this, it thinks it’s got a normal HDD!

  25. JPC Demo Visit http://www-jpc.physics.ox.ac.uk/ or just Google “JPC” • “Instant On” Linux inside the Java Applet sandbox! • Remote HDD • Snapshot taken at the prompt/desktop

  26. Global MPC Infrastructure Nereus Server 1 PC Checkpoint Store Broker MyPC #1 MyPC #2 Nereus Server 2 MyPC #3 MyPC #4 Nereus Server 3

  27. NereusV and JPC • Collect idle computing resources together with NereusV • Deploy “hibernated” virtual machines in vast numbers • Thousands of computers doing your processing… • On demand • Without approval processes / vetting procedures • Very cheaply • Environmentally friendly

  28. So many things to do…. http://www-nereus.physics.ox.ac.uk

  29. Participation Request • Can you help build the NereusV network for research? • NereusV is the most cost effective source of CPU power • Much research to be done – no $$$ for supercomputers! • Benefits to all – results published • Do you have a problem NereusV could solve? http://www-nereus.physics.ox.ac.uk

More Related