1 / 55

Command-Line Data Reduction for XMM: Part I and II Overview and Example Scripts

This guide explores command-line analysis and scripting for XMM data reduction. It provides an overview of the command-line environment, reasons to use scripts, and practical examples such as SASmakecalev and SASprepareXMM. The document discusses setting up the user environment, running scripts for imaging and spectral analysis, and emphasizes the benefits of scripting, such as consistency and efficiency in data processing. You'll find essential parameters, help commands, and tips for executing complex commands effectively.

margie
Télécharger la présentation

Command-Line Data Reduction for XMM: Part I and II Overview and Example Scripts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. XMM data reduction: part IISAS command-line analysis and scripting Andy Read

  2. Overview • Command-line examples/help • Why use command-line and scripts? • Example scripts : SASmakecalev & SASprepareXMM • Imaging scripts & spectral scripts • Scripts within scripts • Final remarks XMM data reduction: part II

  3. Command line examples Setting up the user environment (from Tim’s talk) • To run on XROA system: > sas-setup-6.5.0 initialises software > setenv SAS_ODF (path_to_ODF_directory) e.g. /data/71/tro/xmm_data/0106860101/odf/ > setenv SAS_CCFPATH /data/rb2/sas_ccf > cifbuild fullpath=yes >& cifbuild.log builds ccf.cif– calibration index file > setenv SAS_CCF (path_to_ccf.cif_file) > odfingest odfdir=$SAS_ODF outdir=$SAS_ODF >& odfingest.log builds ***SUM.SAS file in ODF directory – ODF summary file necessary for reprocessing XMM data reduction: part II

  4. Command line help - Simple help – parameters and default values XMM data reduction: part II

  5. - More - Parameters, parameter types, default values and ‘one-line’ help XMM data reduction: part II

  6. - Full help – Links to SAS web pages - Cut and paste link into browser XMM data reduction: part II

  7. Why use scripts? • Scripts are just a list of command-line calls/commands run sequentially • There are many powerful command line calls • Can run intensive analysis on many datasets, sources etc. • Can be sure of doing exactly the same thing every time, e.g. running exactly the same procedures on many sources/datasets • No need for interactive use • - can run overnight (no need to be there) • - can run as cron jobs (no need to be even logged in) • Often need to redo entire analysis, e.g. • - New calibration files • - New ODF files • - Mistake, change in e.g. source positions, detection threshold etc. Useful scripting cookbook at http://starlink.rl.ac.uk/star/docs/sc4.htx/sc4.html XMM data reduction: part II

  8. Example script : SASmakecalev XMM data reduction: part II

  9. Executable t-shell script Comment lines Runs at low priority Number of arguments On-screen help – appears whenever incorrect number of arguments is given (e.g. zero) If correct number of arguments is given (7) then input arguments (1-7) converted to parameters used in script, and we move into the main body of the script XMM data reduction: part II

  10. Check that ODF exists (otherwise exit) Get SAS version (for directory naming) Create (if necessary) output directory, and (if allowed) enter XMM data reduction: part II

  11. Check ODF Set SAS_ODF environment variable Check for ***SUM.SAS file (Check for ***SUM.ASC file) Create ccf.cif – calibration index file Set SAS_CCF environment variable Create ***SUM.SAS file in ODF directory – ODF summary file necessary for reprocessing Output all SAS environment variables to screen – useful check XMM data reduction: part II

  12. Run emproc (if required) Run emchain (if required) Run epproc (if required) Run epchain (if required) List created final calibrated event lists and exit XMM data reduction: part II

  13. Run script XMM data reduction: part II

  14. Example script : SASprepareXMM XMM data reduction: part II

  15. Soft Proton flaring in an observation of a Galaxy Group XMM data reduction: part II

  16. Executable t-shell script Comment lines Runs at low priority Number of arguments On-screen help – appears whenever incorrect number of arguments is given (e.g. none) XMM data reduction: part II

  17. If correct number of arguments is given (9) then input arguments (1-9) converted to parameters used in script If requested : Check ODF Check for ***SUM.SAS file Create ***SUM.SAS file in ODF Set SAS_ODF environment variable XMM data reduction: part II

  18. If requested : Create ccf.cif – calibration index file Set SAS_CCF environment variable Output all SAS environment variables to screen – useful check XMM data reduction: part II

  19. Convert patcode parameter into an expression for evselect XMM data reduction: part II

  20. MOS1 example (comment) XMM data reduction: part II

  21. If $m1 does not equal 0, OK – continue to next line If $m1 equals 0, ignore this whole section XMM data reduction: part II

  22. If $m1 equals F, then we attempt to find the file Else, the file name is given by the parameters XMM data reduction: part II

  23. Remove a temporary file if it already exists XMM data reduction: part II

  24. Long list all the MOS1 event files in the directory to temporary file XMM data reduction: part II

  25. Count the number of entries (MOS1 event files) in the temporary file XMM data reduction: part II

  26. If only one MOS1 event file, use this XMM data reduction: part II

  27. If more than one MOS1 event file, try to find the largest XMM data reduction: part II

  28. Initiate loop through ‘i’ (i=0) XMM data reduction: part II

  29. loop through ‘i’ until ‘i’ equals number of event files, then end XMM data reduction: part II

  30. Increment ‘i’ by +1 (i=1) XMM data reduction: part II

  31. Get size of ‘i’th file : Type the file, extract the top (head) ‘i’ lines, then extract the bottom (tail) line – file size is the 5th string in long listing XMM data reduction: part II

  32. If size of ‘i’th file is larger than largest recorded, note ‘i’ (‘ibig’) and size as largest recorded XMM data reduction: part II

  33. Loop round to next ‘i’ XMM data reduction: part II

  34. Get name of largest file (i.e. file corresponding to ‘ibig’) : Type the file, extract the top (head) ‘ibig’ lines, then extract the bottom (tail) line – file name is the 9th string in long listing Set $m1 to this filename XMM data reduction: part II

  35. Create (evselect) lightcurve : High-energy, single events, ‘good’ flags, 100s bins XMM data reduction: part II

  36. Output help to screen: What is being done, what the user should do XMM data reduction: part II

  37. Plot (dsplot) the light curve to the screen for user to examine XMM data reduction: part II

  38. Request upper and lower threshold values from user XMM data reduction: part II

  39. Use threshold values to create (tabgtigen) good times GTI file XMM data reduction: part II

  40. Create (evselect) intermediate event file : filter on low-E threshold, pattern, flags, etc. XMM data reduction: part II

  41. Create (evselect) final event file : filter intermediate file for good times using GTI file XMM data reduction: part II

  42. Same for MOS2 XMM data reduction: part II

  43. Same for pn XMM data reduction: part II

  44. XMM data reduction: part II

  45. Enter MOS1 thresholds Same for MOS2 Same for pn New filtered event files (r1***EV***) smaller than originals XMM data reduction: part II

  46. Already seen xmmselect GUI - clean data, and produce science products Only handles one dataset at a time Essentially a handy GUI ‘wrapper’ to the SAS-task ‘evselect’ evselect is the command-line workhorse of SAS analysis tasks Many scripts are command-line ‘wrappers’ of evselect and other SAS-tasks XMM data reduction: part II

  47. Script to create images - Essentially an evselect wrapper, plus other tasks (e.g. asmooth) XMM data reduction: part II

  48. e.g. 300-2000eV sky (X/Y) 2′′ images (raw and smoothed) [MOS-sdtq, pn-sd] MOS1 MOS2 pn MOS1 smoothed MOS2 smoothed pn smoothed XMM data reduction: part II

  49. Script to create spectra + ARF, RMF and do grouping etc. Again, evselect wrapper, plus arfgen, rmfgen, grppha etc… SAS-task equivalent especget now much improved XMM data reduction: part II

  50. createspectrum to create BG spectrum Note: can set parameters (here, co-ordinates from Tim’s region files) on the command line Run script (using parameters) XMM data reduction: part II

More Related