1 / 28

Flagging: When Good Data Go Bad

Flagging: When Good Data Go Bad. Scott Schnee & Amy Kimball Nov 9, 2011 Interferometry Discussion Group. Initial Flagging. Shadowing Pointing Errors Reported Unreported Observing Log Other obvious problems. Initial Flagging. Shadowing Issue at low elevations Issue for compact arrays.

aziza
Télécharger la présentation

Flagging: When Good Data Go Bad

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Flagging: When Good Data Go Bad Scott Schnee & Amy Kimball Nov 9, 2011 Interferometry Discussion Group

  2. Initial Flagging • Shadowing • Pointing Errors • Reported • Unreported • Observing Log • Other obvious problems

  3. Initial Flagging • Shadowing • Issue at low elevations • Issue for compact arrays In CASA for an ALMA data set: flagdata(vis=‘vis.ms’, mode=‘shadow’, diameter=12.0) In MIRIAD for a CARMA data set: csflag(vis=‘vis.ms’, carma=true)

  4. Initial Flagging • Pointing Errors • Reported (e.g. wind) • Unreported (e.g. pointing model) In MIRIAD for a CARMA data set: uvflag(vis=‘vis.ms’, select=“pointing(5,100000)”, flagval=flag) uvflag(vis=‘vis.ms’, select=“el(85,90)”, flagval=flag)

  5. Initial Flagging • Observing Log Many observatories will note problems that affect the system for part or all of a track. This could be weather-related or hardware-related, and often requires that some data be flagged.

  6. Initial Flagging • Other obvious problems Source Sys Temps (K) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 URANUS 235 216 1443 327 369 290 150 182 195 165 158 188 184 980 204 3C84 183 163 1477 257 308 225 103 130 136 110 107 131 126 880 120 3C111 187 167 1106 263 314 231 100 133 141 113 110 135 129 892 125 L1451MM 183 165 1675 261 314 224 105 132 138 118 107 134 128 883 131 L1451MM 184 164 1445 259 310 226 103 131 137 110 108 133 127 887 128 0336+323 185 164 1327 258 311 226 107 132 139 112 9820 133 127 887 122 3C111 185 165 1231 262 312 230 102 133 139 113 9449 134 129 884 135 3C111 184 164 1320 258 311 225 103 131 136 114 9979 133 126 890 131 L1451MM 181 162 1412 258 309 225 107 130 135 112 9928 131 126 876 121 L1451MM 182 163 1318 257 308 226 106 130 136 110 105 131 126 892 118 0336+323 182 163 1406 257 309 225 103 130 135 112 105 131 125 878 121 3C111 183 163 1506 259 309 226 106 132 136 115 106 132 127 895 114

  7. Initial Flagging • Other obvious problems Tsys plots from NGC 3256 ALMA CASA Guide

  8. What to look for? • Plots of amplitude and phase vs time and channel • Iterate over • Antenna • Spectral window • Source • Make plots of bandpass and gain calibrators first • Easy to find bad data of a bright point source • Hard to find bad data of a faint extended source

  9. What to look for? • Smoothly varying phases and amplitudes can be calibrated • Discontinuities can not be calibrated • Features in the calibrators that may not be in the target data can cause problems

  10. Amplitude vs Time From TW Hydra ALMA Guide Color: Polarization One spectral window (spw) plotted

  11. Locating the Bad Data in plotms Draw a box around the suspected bad data.

  12. Locating the Bad Data in plotms Click locate and CASA will send information about the data to the logger.

  13. Locating the Bad Data in plotms Bad data can be flagged by pressing this button or using the flagdatatast at the CASA prompt.

  14. Locating the Bad Data in plotms Flagger’s remorse can be corrected by unflagging good data

  15. Amplitude vs FrequencyBirdies From TW Hydra CASA Guide Brown and Green show phase calibrators Orange shows TW Hya

  16. Amplitude vs FrequencyEdge Channels

  17. Amplitude vs FrequencyEdge Channels Data that should be flagged

  18. Amplitude vs FrequencySpectral Lines in Bandpass Calibrator From TW Hydra Band 7 Guide Spectral line in Titan

  19. Phase vs TimePhase Jumps From Antennae ALMA CASA Guide First batch of data Second batch of data

  20. Possible Flagging Technique • Flag obviously bad data • Calibrate the data • Flag newly found bad data • Re-calibrate • Iterate (3, 4) or declare victory

  21. After Calibration, Look Again From NGC 3256 ALMA CASA guide Amplitude vs Time, after calibration

  22. Sage Advice From Rick Perley to a much younger Scott Schnee: “When in doubt, throw it out.”

  23. Online flags • (from Steve Myers and Josh Marvil) • Tbuff (before August 2011) • Quacking sometimes necessary

  24. Known RFI at science.nrao.edu • http://www.gb.nrao.edu/IPG/rfiarchivepage.html • https://science.nrao.edu/facilities/evla/observing/RFI/index

  25. A flagging/calibration recipe • EXAMINE bandpass/flux calibrator(s) • FLAG bandpass/flux calibrators • APPLY bandpass/flux calibration to itself • APPLY bandpass/flux cal to phase cal sources • EXAMINE phase cal sources • FLAG phase cal sources • APPLY phase calibration to itself • APPLY bandpass/flux/phase cal to targets • EXAMINE targets • FLAG targets Iterate Iterate Repeat as necessary

  26. Flagversion control in CASA • Beware of applycal!

  27. Recognizing low-frequency RFI • Average all times: RFI visible on the shortest baselines One EVLA user’s recommendation: flag these frequencies on all baselines at all times

  28. Individual timeranges can be bad

More Related