Goal: Provide an extensible execution management system for large-scale distributed systems, including PlanetLab and the

1. Distributed Application Management Using PLuSHJeannie Albrecht, Christopher Tuttle, Alex C. Snoeren, and Amin VahdatUC San Diego CSE{jalbrecht, ctuttle, snoeren, vahdat} @ cs.ucsd.edu http://ramp.ucsd.edu/projects/plush Goal: Provide an extensible execution management system for large-scale distributed systems, including PlanetLab and the Grid Motivation Experiment Description • Problem: How do we deploy, manage, and maintain distributed applications that simultaneously run on hundreds of heterogeneous physical machines around the world? • Existing approaches for finding resources and managing applications are cumbersome, manual, and error-prone; tools exist to address some issues, but utility is limited by lack of integration • PLuSHprovides a unified environment to support the distributed application design and deployment life cycle on the Grid, PlanetLab, and local site clusters <plush> <project name="demo_proj"> <software name=“demo_soft" type="tar"> <package name=“demo.tar" type="web"> <path>http://ramp.ucsd.edu/plush/demo/demo.tar</path> </package> </software> <configuration name="demo_conf"> <component name=“demo_group"> <software name=“demo_soft"/> <rspec> <num_hosts>100</num_hosts> </rspec> <execution> <process name=“demo"> <path>./demo.exe</path> <cmdline><arg>--runtime=300</arg></cmdline> </process> </execution> </component> <resources>   <resource type="planetlab" group="ucsd_3" />   </resources> </configuration> <experiment name="demo"> <execution> <configuration name="demo_conf"/> </execution> </experiment> </project> </plush> Define a name for the project Describe software packages The configurationcontains componentand executiondefinitions A component is a description of a group of resources rspecis an abstract description of the resources desired The resources tag defines which set of resources to use Architecture The experimentselects previously defined configurations for execution • PLuSH consists of an application controllerthat communicates with clientprocesses running on each of the available resources • An abstract system description specifies the file transfer method, software to install, program execution, and process monitoring details Pluggable Framework Application Description Resource Discovery Resource Acquisition Service Deployment ApplicationControl Resource Monitor ContentDistribution Storage Authentication Access Control • Sample distributed application lifecycle, including the 5 main life cycle stages (blue boxes) and their key underlying components • One goal of PLuSH is to allow users to plug in customized, environment-specific components for each phase in the life cycle • For example, a Grid user may choose to query MDS aggregators for resource discovery, submit a job to aGRAM-enabled scheduler like PBS for resource acquisition, and use GridFTP for service deployment The PLuSH Controller • Takes abstract system description and pool of resources as input • Uses a resource matcher to select and allocate a subset of resources that meets the user criteria • Installs a set of user-defined software packages and project files • Configures and starts processes, monitoring the running application • After execution is complete, cleanup actions are executed at the clients Research Contributions & Current Status • Research Contributions • A high-level specification language for distributed computations that captures the requirements of a broad range of applications • Extensibility from the ground up to support a range of mechanisms for resource discovery, resource acquisition, security, and authentication • Current Status • PLuSH supports application management on PlanetLab and in site clusters • Integration with the vGrADSis currently in progress