1 / 13

Optimizing High Energy Physics Applications with NorduGrid: A Case Study

This document discusses the application of NorduGrid for executing high-energy physics tasks, specifically focusing on the ATLAS Data Challenge exercises. The stability of NorduGrid enables the efficient distribution of computational tasks, facilitating the execution of legacy code adapted for grid computing. It includes prerequisites for users, such as valid credentials and user interface installation, alongside examples of job submission scripts. The report highlights the challenges and methodologies of running simulations on a grid, showcasing the potential and need for future development in data and job management in grid environments.

danno
Télécharger la présentation

Optimizing High Energy Physics Applications with NorduGrid: A Case Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Application examples Oxana Smirnova (Lund, EPF)3rdNorduGridWorkshop, May23,2002

  2. NorduGrid & applications • The level of the NorduGrid stability allows to execute real tasks • Most of the applications so far are in the high energy physics domain • Apart of the simple curiosity and testing, there is a bare necessity to run ATLAS Data Challenge exercises • Requires a lot of computing power • Intensions are to run it in a distributed (Grid) manner a.s.a.p. Oxana Smirnova

  3. Pre-requisites • Valid NorduGrid credentials • A machine with the User Interface installed • Includes minimal Globus client installation • A user may want to make some personal adjustments: • .ngrc # Sample .ngrc file# Comments starts with #NGDEBUG=1NGDOWNLOAD=/tmp • .ngiislistldap://grid.nbi.dk:2135/O=Grid/Mds-Vo-name=NorduGridldap://grid.quark.lu.se:2135/O=Grid/Mds-Vo-name=NorduGrid Oxana Smirnova

  4. (* user-specified job name *) (jobName=Bsubc) (* standard output file *) (stdout="myoutput.dat") (* flag whether to merge stdout and stderr *) (join="yes") (* request e-mail notification on status change *) (notify="e chafik@quark.lu.se") & (* an executable *) (executable=bc_run) (* files to be staged in before the execution *) (inputFiles= (run.dat "") ) (* files to be staged out after the execution *) (outputFiles= (bc.hbook "") ) Double heavy hadron production xRSL specifications: Oxana Smirnova

  5. Double heavy hadron production ngsub –f bsubc.xrsl Oxana Smirnova

  6. Re-used JETSET/PYTHIA generation • Problem: • A legacy code (FORTRAN) uses specific input cards • Each job needs a new random seed • “Gridification” is still trivial: • Generate xRSL files on-fly • Submit for execution a shell-script, wrapped around the actual executable Oxana Smirnova

  7. #!/bin/sh # Submits several run.sh jobs with different random seed if [ $# -eq 0 ]; then echo "" echo Usage: $0 njobs nevents echo "" exit 127 fi MAXCOUNT=$1 MAXEVT=$2 echo "Submitting $MAXCOUNT jobs, $MAXEVT events each..." FLOOR=10000 count=1 while [ "$count" -le $MAXCOUNT ] do number=0 while [ "$number" -le $FLOOR ] do number=$RANDOM done let "inseed = $number*100" jname="flong"$count outname=$jname".out" echo "Job" $jname "started with random seed" $inseed let "count += 1" cat << EOXRSL > ffun.xrsl & (executable=run.sh) (arguments=$inseed $MAXEVT) (executables=ffungen) (inputFiles=(ffungen "")) (outputFiles=(ffun.hbook "")) (jobName=$jname) (stdout=$outname) (join=yes) (maxCpuTime=100) (ftpThreads=6) (middleware="NorduGrid-0.1.6") EOXRSL ngsub -f ffun.xrsl sleep 10 done Re-used JETSET/PYTHIA generation Job submission script ffun.sh Oxana Smirnova

  8. #!/bin/sh cat << EOF > ffun.inp $2 2 0 8 0.04 6 2 0 1 $1 EOF time ./ffungen Re-used JETSET/PYTHIA generation Executable run.sh Job submission: ffun.sh 40 50000 Oxana Smirnova

  9. Re-used JETSET/PYTHIA generation Oxana Smirnova

  10. ATLAS DC1 • Problem: • Executed binary, libraries etc do not belong to a user, but are a part of the ATLAS runtime environment • Input files are residing on [remote] storages • However, as soon as the runtime environment is set up, the task becomes even easier on the Grid, comparing to a traditional method (see the Demo later) • Again, the most convenient method may be to produce xRSL and steering files on-fly Oxana Smirnova

  11. ATLAS DC1 xRSL example &(executable=“\/$ATLAS_ROOT/bin/atlsim") (arguments="-w 0 -b dc1.000016.simu.0000.nordugrid.kumac partition=0001 nskip=0 ntrig=2") (stdout=out.txt)(stderr=err.txt) (outputFiles= ("out.txt" "")("err.txt" "") ("dc1.000016.simu.0001.nordugrid.zebra" "")("dc1.000016.simu.0001.nordugrid.his" "")) (inputFiles= ("atlsim.makefile" "http://www.nbi.dk/~waananen/atlas/atlsim.makefile") ("atlas.kumac" "http://www.nbi.dk/~waananen/atlas/atlas.kumac") ("atlsim.logon.kumac" "http://www.nbi.dk/~waananen/atlas/atlsim.login.kumac") (" dc1.000016.simu.0000.nordugrid.kumac " " dc1.000016.simu.0000.nordugrid.kumac ") ("gen0016_1.root" "rc://@grid.uio.no:389/lc=ATLAS,rc=NorduGrid,dc=nordugrid,dc=org/gen0016_1.root") ) (runTimeEnvironment="ATLAS-3.0.1") (jobname="dc1.000016.simu.0001.nordugrid") (notify="waananen@nbi.dk") Oxana Smirnova

  12. ATLAS DC1 Oxana Smirnova

  13. Summary • NorduGrid is ready to run “traditional” applications in a Grid environment • There is still plenty of room for development: e.g., data management, production job management etc Oxana Smirnova

More Related