SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

CMIP5 Download Tutorial

ellard avatar
ellard
20 diapositives · 362 Views · Jul 29, 2014

CMIP5 Download Tutorial. Jennifer M. Adams 12 January 2012 /data/cmip5/extras/CMIP5_Tutorial.pptx. /data/cmip5/docs/data_reference_syntax.pdf Eleven keywords u niquely describe a data set:

CMIP5 Download Tutorial - slide 1
01 / 20CMIP5 Download Tutorial
1 / 17

Télécharger la présentation

CMIP5 Download Tutorial
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. CMIP5 Download Tutorial Jennifer M. Adams12 January 2012 /data/cmip5/extras/CMIP5_Tutorial.pptx

  2. /data/cmip5/docs/data_reference_syntax.pdf Eleven keywords uniquely describe a data set: Project. Institute. Model. Experiment. Frequency. Realm. MIP Table. Product. Variable. Ensemble. Version. Vocabulary Lesson

  3. Project (Activity): cmip5Institute: The institute responsible for model results, e.g. MOHC,NOAA-GFDL, NCAR, MPI-M, NASA-GMAO.…Model: The name of the model used,e.g. HadCM3, GFDL-ESM2M, CCSM4, MPI-ESM-LR….Experiment: The name of the experiment family and type,e.g. amip, amip4xCO2, decadal1960, piControl, historical….

  4. Frequency: The interval between time samples. Options are:yr, mon, day, 6hr, 3hr, subhr, monClim, fxRealm: The high level modeling component. Options are: atmos, ocean, land, landIce, seaIce, aerosol, atmosChem, ocnBgchem MIP Table: A spreadsheet* entry for realm and variable components,e.g. Amon, Omon, Lmon, LImon, day, …. * /shared/cmip5/docs/standard_output.xlsx

  5. Variable: A short name that identifies a physical quantity, e.g. pr, ps, psl, tas, tauu, tauvuas, vas….Product: A designation of CMIP5 files. Options are: output, output1, output2, unsolicitedEnsemble: A name that distinguishes among closely related simulations and includes 3 numbers: realization (rN), initialization method (iM), and perturbed physics (pL).e.g. r1i1p1, r7i1p1, r1i3p1, r1i1p2.… Version: Uniquely identifies a particular version of the data set, e.g. v20110923, v20111208, v1, v2.…

  6. /project/cmip5/jma/desired.txtThis list, based on user requests, is managed by Tim and Jennifer. /project/cmip5/jma/acquired.txt This list, based on contents of /data/cmip5/data, is auto-updated. Each Downloader will be assigned an item from the desired list and must grab all available models and ensemble members for the given Experiment/Realm/Frequency/MIP Table. Desired & Acquired Data Sets

  7. For Downloading: http://pcmdi3.llnl.gov/esgcet/home.htm http://cmip-gw.badc.rl.ac.uk/home.htmhttp://ipcc-ar5.dkrz.de/home.htm For Searching: http://esg.prototype.ucar.edu/home.htm Always Use Firefox! ESGF Gateways

  8. Gateway Login

  9. OpenID: https://pcmdi3.llnl.gov/esgcet/myopenid/jennifer Username: jennifer Password: sdf,WER.5 Authentication

  10. Gateway Search

  11. Gateway Search

  12. Dataset URL Gateway Search Results Dataset ID

  13. Select Files

  14. Download Wget Script

  15. mv wget-download.sh wget.cmip5.output1.MOHC.HadCM3.decadal2000.mon.atmos.Amon.r1i2p1.v20110708.sh Rename Wget Script IT IS VERY IMPORTANT THAT YOU DO THIS CORRECTLY! Because … Otherwise there is no way to tell what dataset wget-download.sh is configured to grab I need the keywords in the new script name to enable several automation scripts Some of the metadata (esp. version number) can’t be captured any other way

  16. Create a top level working directory (you will need my help with this):mkdir /shared/working/cmip5/jma Make several subdirectories under your top level directory, one for each wget: cd /shared/working/cmip5/jma mkdir a01 a02 a03 a04 a05 a06 a07 a08 a09 a10 a11 a12 a13 a14 a15 a16 sftp wget.renamed.sh from laptop to /shared/working/cmip5/jma/a01/ Set Up Work Environment

  17. mkdir $HOME/.esgcd $HOME/.esgcp /data/cmip5/extras/MyProxyLogon-ESG.jar . cp /homes/jma/.esg/update ../update Certificates

  18. Login to a server: cpu1-cpu6 cd /shared/working/cmip5/jma/a01 /project/cmip5/jma/dorun.sh & This script filters out unwanted files, notes wget format, runs edited wget script, captures output in log file, and monitors script’s progress. Run it in the background. /project/cmip5/jma/ckrun.sh When dorun.sh is no longer working, this script evaluates the success of wgets, checks if all desired files are here, returns status of download. Launch the Wget

  19. If ckrun.sh determines that the download is complete, it will create a file called ‘done’. Take a moment to celebrate, then move on to next data set. • If ckrun.sh determines that download is incomplete, there was an error. • Look for a file in working subdir named ‘formatA’ or ‘formatB’ or ‘formatC’. If format=A: • Go back to the Gateway and get a new wget script • Copy new wget.renamed.sh into working subdirectory • Rerun dorun.sh • If format=B or C: • You may need to update your certificates • You do not need a new wget script; just rerun dorun.sh Repeat As Necessary

  20. Watch out for zombies, a.k.a. processes that are hung and can’t be killed • Use dataset URLs to circumvent the search • Start by downloading only these high-priority models first: Pitfalls & Shortcuts