SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Using the Grid

tomas avatar
tomas
10 diapositives · 224 Views · Aug 22, 2014

This guide provides step-by-step instructions for getting started with grid computing, focusing on accessing the LHC (Large Hadron Collider) resources. It details how to obtain a digital certificate, join a Virtual Organisation (VO), and access a local User Interface Machine (UI) for job submission. Key concepts like job descriptions using JDL (Job Description Language), input/output sandboxes, and the process for submitting jobs via the grid are explained. Detailed examples help users prepare and submit their jobs successfully.

#using the grid #using the grid ppt #powerpoint #presentation
Using the Grid - slide 1
01 / 10Using the Grid
1 / 17

Télécharger la présentation

Using the Grid
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Using the Grid Steve LloydQueen Mary, University of London GridPP13 Durham July 2005

  2. Getting Started 1. Get a digital certificate Authentication – who you are http://ca.grid-support.ac.uk/ 2. Join a Virtual Organisation (VO) For LHC join LCG and choose a VO Authorisation – what you are allowed to do http://lcg-registrar.cern.ch/ 3. Get access to a local User Interface Machine (UI) and copy your files and certificate there Lxplus: > source /afs/cern.ch/project/gd/LCG-share/2.4.0/sl3/etc/profile.d/grid_env.sh GridPP13 Durham July 2005

  3. Overview Resource Broker Input Sandbox Output Sandbox Script you want to run Output files (Ntuples, Logs...) Storage Element Storage Element Other files (Job Options, Source...) Compute Element Grid UI Machine Job Description Proxy Certificate Job Output Sandbox Input Data Output Data GridPP13 Durham July 2005

  4. Job Preparation Prepare a file of Job Description Language (JDL): #############Hello World################# Executable = "/bin/echo"; Arguments = "Hello Steve welcome to the Grid "; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; ######################################### Simplest Case? Input files Script to run Job Options My C++ Code Realistic Case: ############# athena.jdl ################# Executable = "athena.sh"; StdOutput = "athena.out"; StdError = "athena.err"; InputSandbox = {"athena.sh", "MyJobOptions.py", "MyAlg.cxx", "MyAlg.h", "MyAlg_entries.cxx", "MyAlg_load.cxx", "login_requirements", "requirements", "Makefile"}; OutputSandbox = {"athena.out","athena.err", "ntuple.root", "histo.root", "CLIDDBout.txt"}; Requirements = Member("VO-atlas-release-10.0.1", other.GlueHostApplicationSoftwareRunTimeEnvironment); ################################################ Output Files Choose ATLAS Software Version GridPP13 Durham July 2005

  5. Job Preparation A Job Script (ATLAS Specific): #!/bin/bash # Script to run Athena Analysis on the Grid # Steve Lloyd source $VO_ATLAS_SW_DIR/software/10.0.1/setup.sh source $SITEROOT/dist/10.0.1/Control/AthenaRunTime/*/cmt/setup.sh # Set up the correct directory structure for my files (?) cmt create UserAnalysis UserAnalysis-00-02-02 # Fails to create these for some reason mkdir UserAnalysis/UserAnalysis-00-02-02/UserAnalysis mkdir UserAnalysis/UserAnalysis-00-02-02/src/components mv AnalysisSkeleton.cxx UserAnalysis/UserAnalysis-00-02-02/src mv AnalysisSkeleton.h UserAnalysis/UserAnalysis-00-02-02/UserAnalysis mv UserAnalysis_load.cxx UserAnalysis/UserAnalysis-00-02-02/src/components mv UserAnalysis_entries.cxx UserAnalysis/UserAnalysis-00-02-02/src/components mv requirements Makefile UserAnalysis/UserAnalysis-00-02-02/cmt # Build my code export CMTPATH=`pwd`:${CMTPATH} cd UserAnalysis/UserAnalysis-00-02-02/cmt cmt config source setup.sh gmake # Back to base cd ../../.. # Botch required to find PDGTABLE.MeV: #cp $SITEROOT/dist/10.0.1/InstallArea/share/PDGTABLE.MeV . # Run the job: athena.py AnalysisSkeleton_jobOptions.py GridPP13 Durham July 2005

  6. Job Submission Make a copy of your certificate to send out (~ once a day): [lloyd@lcgui ~/atlas]$ grid-proxy-init Your identity: /C=UK/O=eScience/OU=QueenMaryLondon/L=Physics/CN=steve lloyd Enter GRID pass phrase for this identity: Creating proxy .............................. Done Your proxy is valid until: Thu Mar 17 03:25:06 2005 [lloyd@lcgui ~/atlas]$ Submit the Job: VO File to hold job IDs JDL [lloyd@lcgui ~/atlas]$ edg-job-submit --vo atlas -o jobIDfile athena.jdl Selected Virtual Organisation name (from --vo option): atlas Connecting to host lxn1188.cern.ch, port 7772 Logging to host lxn1188.cern.ch, port 9002 ================================ edg-job-submit Success ==================================== The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ The edg_jobId has been saved in the following file: /home/lloyd/atlas/jobIDfile ============================================================================================ [lloyd@lcgui ~/atlas]$ GridPP13 Durham July 2005

  7. [lloyd@lcgui ~/atlas]$ edg-job-status -i jobIDfile------------------------------------------------------------------1 : https://lxn1188.cern.ch:9000/tKlZHxqEhuroJUhuhEBtSA2 : https://lxn1188.cern.ch:9000/IJhkSObaAN5XDKBHPQLQyA3 : https://lxn1188.cern.ch:9000/BMEOq90zqALvkriHdVeN7A4 : https://lxn1188.cern.ch:9000/l6wist7SMq6jVePwQjHofg5 : https://lxn1188.cern.ch:9000/wHl9Yl_puz9hZDMe1OYRyQ6 : https://lxn1188.cern.ch:9000/PciXGNuAu7vZfcuWiGS3zQ7 : https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQa : allq : quit------------------------------------------------------------------Choose one or more edg_jobId(s) in the list - [1-7]all:7*************************************************************BOOKKEEPING INFORMATION:Status info for the Job : https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQCurrent Status: Done (Success)Exit code: 0Status Reason: Job terminated successfullyDestination: lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-shortreached on: Wed Mar 16 17:45:41 2005*************************************************************[lloyd@lcgui ~/atlas]$ Job Status Find out its status: Ran at: Valencia RAL CERN Taiwan Taiwan GridPP13 Durham July 2005

  8. Job Retrieval Retrieve the Output: [lloyd@lcgui ~/atlas]$ edg-job-get-output -dir . -i jobIDfile Retrieving files from host: lxn1188.cern.ch ( for https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ have been successfully retrieved and stored in the directory: /home/lloyd/atlas/lloyd_0uDjtwbBbj8DTRetxYxoqQ ********************************************************************************* [lloyd@lcgui ~/atlas]$ ls -lt /home/lloyd/atlas/lloyd_0uDjtwbBbj8DTRetxYxoqQ total 11024 -rw-r--r-- 1 lloyd hep 224 Mar 17 10:47 CLIDDBout.txt -rw-r--r-- 1 lloyd hep 69536 Mar 17 10:47 ntuple.root -rw-r--r-- 1 lloyd hep 5372 Mar 17 10:47 athena.err -rw-r--r-- 1 lloyd hep 11185282 Mar 17 10:47 athena.out GridPP13 Durham July 2005

  9. Reading Data Either specify in the JDL (then resource broker will try and find a CE 'close' to the SE): Logical file names ############# athena.jdl ################# ... InputData = {"lfn:rome.004201.recov10.ZeeJimmy._00018.AOD.pool.root", "lfn:rome.004201.recov10.ZeeJimmy._00018.AOD.pool.root"}; DataAccessProtocol = {"gsiftp"}; ... ################################################ "rome" stands for ATLAS Rome Workshop Production not the location of the file! Or pull the files from inside the job script: #!/bin/bash # Get the data lcg-cp -v --vo atlas lfn:rome.004201.recov10.ZeeJimmy._00018.AOD.pool.root file://`pwd`/Zee_18.pool.root lcg-cp -v --vo atlas lfn:rome.004201.recov10.ZeeJimmy._00019.AOD.pool.root file://`pwd`/Zee_19.pool.root ... GridPP13 Durham July 2005

  10. Conclusions • The Grid is here – it works! • Currently difficult to install and maintain the middleware and the experiment’s software • It is VERY straightforward to use if all you have is a binary executable • It is straightforward to use if someone has installed your experiment's software somewhere • There are huge resources available: On 29 June 5789 out of 14079 CPUs free (UK: 1548 out of 2966) Use it! GridPP13 Durham July 2005