irma-cooley
Uploaded by
25 SLIDES
407 VUES
250LIKES

Hands-On Grid Building Workshop with Globus and CondorG Implementation

DESCRIPTION

Join this interactive hands-on session led by Ashok Adiga and Victor Bolet, focusing on building and configuring a grid environment using Globus and CondorG. Participants will learn to set up essential components such as Pacman, Ganglia, and MyProxy, and dive into job submission through the SURAGrid portal. We will cover critical networking setup, security configurations, and the importance of synchronized time for grid operations. This workshop is ideal for those looking to deepen their understanding of grid technology and its practical applications.

1 / 25

Télécharger la présentation

Hands-On Grid Building Workshop with Globus and CondorG Implementation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Technology: The Rough Guide Grid Building Hands-On Session Ashok Adiga, Texas Advanced Computing Center Victor Bolet, Georgia State University

  2. Grid Lab Session • Overview: • Laptop Environment • Pacman Install • Globus Install and Config • Configure GSI • Certificates • Command Line Job Submission • GridFTP

  3. Grid Lab Session • Overview: (cont) • Ganglia Install • CondorG • MyProxy • Using a Grid Portal: SURAGrid • Portal Job Submission • Wrap Up

  4. Grid Lab Session • Components • GridPort – CondorG • SimpleCA – Globus • MyProxy – Ganglia • Hardware • Portal Server – MPI Cluster • CA Server – Laptops (Grid Nodes) • Ganglia Server

  5. Laptop Environment • Red Hat Enterprise 2.6.11 • Bash shell • Root Account: login: root password: Unx4me. • User Account: login: griduser password: griduser • Log in as griduser • Right click on desktop and select “Open Terminal” (you will need 2 terminals) • Login as root in one of the terminal windows • In one of the terminals type: su – • Enter the root password • Files for workshop are located in /root/workshop

  6. Network Setup • Grids are picky about networks • Host names • For a host cert, the fully qualified host name must be in there • When using services, host names are provided to call back to send status information • Firewalls • GridFTP, status messages can be blocked • On your laptop: • Applications menu -> System Settings -> Network • In the DNS tab, set the Hostname to be • labXX.tacc.utexas.edu (replace XX with your number) • File -> Save and then File -> Quit

  7. Time Setup • Grid security requires that the times on the hosts match up well • Grid-proxy-init • On a host that thinks it is 11am • You get a proxy valid from 11am today • Globusrun to a host • To a host that thinks that it is 10am • Does not allow you to run - your proxy isn’t valid • Check the time on your laptop • If it isn’t correct: • Use the date command to correct it • date [MMDDhhmm.ss]

  8. Pacman Install • In the root window, change into the vdt directory: • cd /usr/local/vdt • Fetch the pacman tarball: • wget http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-latest.tar.gz • Expand the tarball: • tar zxvf pacman-latest.tar.gz • Install Pacman: • cd pacman-3.15/ • source setup.sh

  9. Globus Install and Config • In the root window, change to Globus install directory • cd /usr/local/vdt • Fetch Globus package using Pacman: (for the classroom) • pacman -get http://129.114.2.195/pacman/vdt_136_cache:Globus • To fetch the Globus package outside the classroom use: (note: this is usually a 200-300mb download) • pacman -get http://www.cs.wisc.edu/vdt/vdt_136_cache:Globus • Installation Prompts: • Answer ‘n’ to the following: Where would you like to install CA files? Choices: r (root) - install into /etc/grid-security/certificates (existing CA files will be preserved) l (local) - install into $VDT_LOCATION/globus/share/certificates n (no) - do not install • Answer ‘y’ to all other prompts

  10. Configure GSI • In the root window,change to the install directory: • cd /usr/local/vdt • Configure Globus GSI: • source setup.sh • mkdir /etc/grid-security • gpt-build /root/workshop/globus_simple_ca_bffae137_setup-0.17.tar.gz gcc32dbg • gpt-postinstall (NOTE: Ignore Warning Messages) • /usr/local/vdt/globus/setup/globus_simple_ca_bffae137_setup/setup-gsi -default • Copy Certificates: • cd /etc/grid-security/certificates • tar xvf /root/workshop/certs.tar • Link the Globus Files • /root/workshop/gsi_certlinks.sh • This script does the following: ln -s /etc/grid-security/certificates /usr/local/vdt/globus/TRUSTED_CA ln -s /etc/grid-security/certificates /usr/local/vdt/globus/etc/certificates ln -s /etc/grid-security/globus-host-ssl.conf /usr/local/vdt/globus/etc/globus-host-ssl.conf ln -s /etc/grid-security/globus-user-ssl.conf /usr/local/vdt/globus/etc/globus-user-ssl.conf ln -s /etc/grid-security/grid-security.conf /usr/local/vdt/globus/etc/grid-security.conf

  11. Obtaining Host & User certificates • simpleCA used as Certificate Authority in this hands-on session • Part of Globus Toolkit distribution • Server side code pre-installed for hands-on • Client utilities were created and packaged during simpleCA server install • globus_simple_ca_bffae137_setup-0.17.tar.gz • For detailed documentation see: http://www.globus.org/toolkit/docs/4.0/admin/docbook/ch07.html • installing and configuring simpleCA server • signing user & host certificates

  12. Certificates • Globus Certificates are stored in 2 places: • Host / Server Certificates: • /etc/grid-security/certificates • User Certificates (private key): • ~/.globus

  13. Requesting Certificates • Generate Host Certificate Request (in root window): • source /usr/local/vdt/setup.sh • grid-cert-request –host `hostname` • (answer yes to the warning about fully qualified names) • cp /usr/local/vdt/globus/etc/hostkey.pem /etc/grid-security/ • Generate User Certificate Request (in griduser window): • source /usr/local/vdt/setup.sh • grid-cert-request • Enter your name • set a passphrase (Need to remember this!!!) • Certificates are ready to be Signed: • Inform the Instructor your certificates are ready

  14. Grid Security Files (local certs) • Verify policy files and “.0” files (in griduser window): • ls /etc/grid-security/certificates • Generate a proxy certificate: • grid-proxy-init • enter pass phrase used to generate certificate • Verify the certificate details: • grid-cert-info • grid-cert-info –subject

  15. Command Line Job Submission • Verify your DN is in the current grid-mapfile on bandera.tacc.utexas.edu • Test the certificate (GRAM Authentication): • globusrun –a -r bandera.tacc.utexas.edu • Try a simple job: • globus-job-run bandera.tacc.utexas.edu /bin/hostname

  16. GridFTP Create a test file to copy • echo “1 2 3 4 5” > /tmp/<hostname>numbers.txt • Copy the file to bandera using GridFTP • globus-url-copy file:///tmp/<hostname>numbers.txt \ gsiftp://bandera.tacc.utexas.edu/tmp/<hostname>numbers.txt • See your file on Bandera (instructor to display)

  17. Ganglia Monitoring Tool • Collector Machine: lab01 (preinstalled) • URL: http://129.114.2.195/ganglia • Collector machine runs gmetad service • Grid nodes report to lab01 through local gmond service • Server install documentation/download: • http://ganglia.sourceforge.net • Version 3.0.1

  18. Ganglia Install • Install gmond Service: (in root window) • rpm –i /root/workshop/ganglia-gmond-3.0.1-1.i386.rpm • edit /etc/gmond.conf • Under cluster: • name=<hostname> • owner=“Your Name” • Under udp_send_channel: • comment out (#) the mcast_join statement • add a "host = labXX.staff.utexas.edu" statement • Under udp_recv channel: • comment out (#) the mcast_join statement • comment out (#) the bind statement • Restart gmond • service gmond restart • Go to URL and see your node: • http://129.114.2.195/ganglia

  19. Condor Install, Config, Startup • Installs & configures all of Condor • We’ll just be using the Condor-G and DAGMan parts • In the root window, change to Globus install directory • cd /usr/local/vdt • Fetch Condor package using Pacman: (for the classroom) • pacman -get http://129.114.2.195/pacman/vdt_136_cache:Condor • To fetch the Globus package outside the classroom use: (note: this is a 100mb download) • pacman -get http://www.cs.wisc.edu/vdt/vdt_136_cache:Condor • Installation Prompts: • Would you like Condor to run automatically? • Yes • Update root environment so that you can use Condor • source /usr/local/vdt/setup.sh • Start the Condor Master (which starts all other Condor daemons) • cd /usr/local/vdt/condor/sbin/ • ./condor_master • You should see the condor daemons running as the condor user • ps -eaf | grep condor

  20. Using Condor-G • Working in the griduser window • Update the environment • source /usr/local/vdt/setup.sh • Look at the condor commands in /usr/local/vdt/condor/bin • condor_q to look at queued jobs • condor_submit to submit a condor job • condor_rm to remove a submitted job

  21. Simple Condor Submission • Create a simple condor submit script called, say, globus-ls.sub containing: Universe = globus globusscheduler = bandera.tacc.utexas.edu Executable = /bin/ls Arguments = /usr/bin Transfer_Executable = false Output = ls.out Error = ls.err queue • Then do a condor_submit globus-ls.sub • You can do a condor_q to watch your job run • Can also try to submit jobs to other labXX systems, if you are in the grid-mapfile there

  22. MyProxy • Install MyProxy client (In root window): • cd /usr/local/vdt • pacman -get http://www.cs.wisc.edu/vdt/vdt_136_cache:MyProxy • MyProxy host name: • pecos.tacc.utexas.edu • for MyProxy server installation instructions, see: http://www.globus.org/toolkit/docs/4.0/security/myproxy/admin-index.html • Load MyProxy Certificate: • myproxy-init -s pecos.tacc.utexas.edu -l <portal_username> * portal_username is the same as your wireless account • enter User Certificate passphrase • enter MyProxy password • Need to Remember this Password!!

  23. Using a Grid Portal: SURAGrid • Based on GridPort • Quick install instructions: • http://gridport.net/main/quickstart-download.html • Log into the User Portal: • https://gridportal.sura.org • Username: (same as wireless account name – ie lab01) • Password: (same as username) • Retrieve MyProxy Certificate: • Click on “Proxy Management” tab • Click “Get New Proxy” • Enter username and password used in myproxy-init • Verify subject of MyProxy certificate

  24. SURAGrid Portal (cont.) • File Management tab: • Select Host: bandera.tacc.utexas.edu • Manage local files on bandera • Job Submission tab: • Host: bandera.tacc.utexas.edu • Port: 2119 (default) • Executable: /bin/hostname • Standard Output: /home/griduser/hostname.out • Use File Management tab to see if file is there

  25. Wrap Up Questions?

More Related