1 / 6

Enabling parameter sweep jobs via scaling cluster-based computing services in the cloud

Explore the advantages of scaling cluster-based computing services in the cloud for enabling parameter sweep jobs. This paper discusses load balancing of application instances, on-demand CPU and network bandwidth, and adaptive system storage capacity. Key benefits include cost reductions due to efficient management of consolidated remote clusters and easy access to massive internet bandwidth. With providers like Amazon, Google, and Microsoft leading the market, we detail practical steps for deploying applications using EC2, enhancing efficiency and reducing local hardware dependency.

bjorn
Télécharger la présentation

Enabling parameter sweep jobs via scaling cluster-based computing services in the cloud

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Enabling parameter sweep jobs via scaling cluster-based computing services in the cloud Nadya Williams nadya@sdsc.edu Phil Papadopoulos philip.papadopoulos@gmail.com Wilfred Li wilfredli01@gmail.com

  2. Cloud Anatomy • Application instances load balancing • CPU and network bandwidth on demand • System storage capacity goes up/down • Costs Reduction: • local hardware via • more efficient handling of consolidated • remote clusters • telecommunication: printing, files transfer Different tasks use single or multiple instances • From 1Byte to a few Tbyte Computing cloud • Large collection of CPU + Memory + Storage • Access to massive internet bandwidth • Physically located not at customer site • Easy access to computing power and bandwidth • Available to anyone and relatively inexpensive

  3. Cloud providers July 15, 2010 04:46PM Network World Report http://www.btclogic.com/pov/rankings.cfm Contenders: AT&T Aylus Networks Rackspace Heavyweights: Microsoft Google Red Hat Salesforce Symantec Vmware Citrix EMC Oracle Level 3 Cisco Champions: Amazon IBM

  4. Scaling the cluster Amazon Cloud Storage Elastic Compute Cloud (EC2) copy AMI & boot Condor pool Application services EC2 condor execute machines

  5. Step by step Step #1: start development server # rocks start host vm devel-server-0-0-2 # ssh devel-server-0-0-2 add application: binary, libs, DB, etc # shutdown –h now # export EC2_PRIVATE_KEY=/root/.ec2/pk.pem # export EC2_CERT=/root/.ec2/cert.pem # rocks create ec2 bundle devel-server-0-0-2 imagename=devel-server-0-0-2-pragma # rocks upload ec2 bundle devel-server-0-0-2 pragma19imagename=devel-server-0-0-2-pragma # ec2-register pragma19/devel-server-0-0-2-pragma.manifest.xml # ec2-run-instances -n 5 -t m1.large ami-5840aa31 -ksi2010 -gsi2010 –d "condor:rocce.ucsd.edu:40000:40050” # ec2-describe-instances Step #2: add to development server Step #3: create AMI Step #4: start AMI

  6. Thank you! Questions?

More Related