MILC Code Basics

1. MILC Code Basics Carleton DeTar First presented at Edinburgh EPCC HackLatt 2008 Updated 2013 HackLatt 2008

2. MILC Code Capabilities • Molecular dynamics evolution • Staggered fermion actions (Asqtad, Fat7, HISQ, etc) • Clover fermion action • Pure gauge • Schroedinger functional • Hadron spectroscopy • Staggered mesons and baryons • Clover mesons and baryons • Mixed staggered/clover mesons • Static/light spectroscopy • Quarkonium spectroscopy (S and P-wave) • Current matrix elements • Leptonic decay (fpi, fB, fD) • Semileptonic decay (heavy-light) • Miscellaneous • Topological charge • Dirac matrix eigenvectors and eigenvalues • Nonperturbative renormalization of currents HackLatt 2008

3. Two views of the code. Powerful and capable. HackLatt 2008

4. Or a path to confusion and misery. Mathias Gruenewald: Temptation of St Anthony (1515) HackLatt 2008

5. Supported File Formats • Gauge configuration file formats • MILC, SciDAC (ILDG), NERSC, Fermilab • Dirac propagator file formats • USQCD, Fermilab • Staggered propagator file formats • USQCD, MILC, Fermilab HackLatt 2008

6. Supported SciDAC C-Coded Packages • QIO (I/O) • QMP (Message passing) • QLA (linear algebra – single processor) • QDP/C (linear algebra – data parallel) • QOPQDP (“Level 3” optimized) • (More in the next session) http://usqcd.jlab.org/usqcd-software/ HackLatt 2008

7. Precision • Global single or double precision • Mixed precision in some applications Portability • Any scalar machine • Any MPP machine with MPI • GPU, Intel Xeon Phi (somewhat) HackLatt 2008

8. MILC Code Organization • Application directories • With compilation targets • Library directory • Linear algebra routines • Shared procedures (“generic”) directories • Shared across applications HackLatt 2008

9. MILC Code Organization • Application directories: examples • cd ks_imp_dyn (application) • make su3_rmd (target) (Asqtad R algorithm) • make su3_spectrum (another target) (staggered spectroscopy) • cd ks_imp_rhmc (application) • make su3_rhmc (Asqtad RHMC algorithm) • make su3_rhmc_hisq (target) (HISQ algorithm) • cd clover_invert2 (application) • make su3_clov (clover spectroscopy, etc.) HackLatt 2008

10. MILC Code Organization • Shared procedures directories: examples • generic (common to all applications) • generic_ks (common to staggered fermion applications) • generic_wilson (common to clover and Wilson fermion apps) HackLatt 2008

11. Building the MILC Code • Download sourcehttp://www.physics.utah.edu/~detar/milc_qcd.html • Unpack • Configure • Build • Check HackLatt 2008

12. Building the MILC Code • Unpack • tar –xvzf milc_qcd-7.7.10-a7.tar.gz HackLatt 2008

13. Building the MILC Code • Configure (crude old fashioned!) • Copy default Makefile to application directory • cd ks_imp_rhmc • cp ../Makefile . • Edit (example in next slide) • Makefile • ../libraries/Make_vanilla • ../include/config.h HackLatt 2008

14. Editing the Makefile -- examples #---------------------------------------------------------------------- # 2. Architecture # Compiling for a parallel machine? Uncomment this line. #MPP = true #---------------------------------------------------------------------- # 3. Precision # 1 = single precision; 2 = double PRECISION = 1 #---------------------------------------------------------------------- # 4. Compiler # Choices include mpicc cc gcc pgcc g++ ifeq ($(strip ${MPP}),true) CC = /usr/local/mvapich/bin/mpicc else CC = gcc endif HackLatt 2008

15. Optimization Possibilities • Build plain MILC version • Build with SciDAC optimized QOP support • Requires installing the SciDAC packages. (See second tutorial and code home page). • Build with GPU QUDA support • Requires installing QUDA and QUDA-MILC packages. (See the code home page) • Build with OpenMP directives on loops • Very rudimentary and spotty. May need to edit some code by hand to insert OMP macros. HackLatt 2008

16. Otimization choices controlled in the Makefile -- examples #---------------------------------------------------------------------- # 5. Compiler optimization level OPT = -O3 #---------------------------------------------------------------------- # 6. Other compiler optimizations (depending on compiler) OCFLAGS = # Compiling with OpenMP? OMP = true #---------------------------------------------------------------------- # 10. SciDAC package options WANTQOP = true # turns on all optimized QOPQDP modules #---------------------------------------------------------------------- # 14. GPU/QUDA Options WANTQUDA = true # turns on possible QUDA selections WANT_FN_CG_GPU = true # turns QUDA CG WANT_FL_GPU = true # QUDA link fattening WANT_FF_GPU = true # QUDA fermion force term WANT_GF_GPU = true # QUDA gauge force ifeq ($(strip ${MPP}),true) CC = /usr/local/mvapich/bin/mpicc else CC = gcc endif HackLatt 2008

17. Building and Checking the Code • Build (for example) • make su3_rmd • Check single precision su3_rmd • make check “PROJS=su3_rmd” “PRECLIST=1” • Check all targets in this directory • make check HackLatt 2008

18. Running the code • su3_rhmc < inputfile > outputfile • su3_rhmc inputfile > outputfile • su3_rhmc inputfile outputfile HackLatt 2008

19. Sample parameter input (su3_rmd) prompt 0 nflavors1 2 nflavors2 1 nx 16 ny 16 nz 16 nt 64 iseed 5682304 warms 0 trajecs 2 traj_between_meas 1 beta 6.76 mass1 0.05 mass2 0.5 u0 0.8441 microcanonical_time_step 0.02 steps_per_trajectory 4 max_cg_iterations 300 max_cg_restarts 5 error_per_site .000005 error_for_propagator .000002 npbp_reps 1 prec_pbp 1 reload_serial ../../binary_samples/lat.sample.l4444 save_serial_scidac lat.test.scidac lattice dimensions molecular dynamics measurements HackLatt 2008

20. Summary • The MILC code is versatile and portable • I have given a brief overview of the code structure • I have touched on the procedures for building and running the code HackLatt 2008

21. Tutorial 1 Goals • Run precompiled code • Modify the input parameters • Build a different target • Modify the Makefile and build • Download and unpack the code (if time) HackLatt 2008