Beating Noise Observational Techniques

1. Beating NoiseObservational Techniques ASTR 3010 Lecture 11 Textbook

2. From previous classes • We learned that in the astronomical data pre-processing, bias subtraction, dark subtraction, flat-fielding, bad pixel correction, cosmic ray hits removal are important. • There are several observation techniques that can take care of the above correction nearly automatically.

3. Subtraction of two images taken with a small offset • image1 – image2 • bad pixels, bias, and dark are removed! • But, cosmic rays remain. • Need to apply flat-fielding - =

4. ?Dither Observation? • Dither Observation Series of images taken with telescope offsets. box5 box4

5. Dithering Observation Detector Array Observation No 1 dead pixels hot pixels cosmic rays

6. Dithering Observation Observation No 2 dead pixels hot pixels cosmic rays

7. Dithering Observation Observation No 3 dead pixels hot pixels cosmic rays

8. Dithering Observation Observation No 4 dead pixels hot pixels cosmic rays

9. Dithering Observation Observation No 4 dead pixels hot pixels cosmic rays

10. Dithering Observation • Five observations obs1 … obs5 flux3 True Flux flux1 flux5 flux2 flux4

11. Dithering Observation • Five observations This will take care of bias, dark, cosmic ray hits, bad pixels! obs1 … obs5 Ordered list flux4 < flux5 < flux2 < flux1 < flux3 median value

12. Dozens of Dithered images with small offsets • SuperResolution

13. With the P&A telescope Not Yet!

14. P&A Dithering by Hand

15. Real example : Keck NIRC2 AO observation of HIP88945

16. Shift and Add look at those hot pixels!!

17. # demonstrating the Shift and Add concept. # # import libraries from ds9 import * from numpy import * import pyfits im1=pyfits.getdata('n0002.fits',ignore_missing_end=True) im2=pyfits.getdata('n0003.fits',ignore_missing_end=True) im3=pyfits.getdata('n0004.fits',ignore_missing_end=True) im4=pyfits.getdata('n0005.fits',ignore_missing_end=True) im5=pyfits.getdata('n0006.fits',ignore_missing_end=True) # create a big enough array to cover any shifted images... BIG = zeros((1201,1201)) # Then, we will find the positions of the star in each image. # Shift the image in the BIG array such that the star appears # at the center of the array (601,601) posx_im1=276 posy_im1=279 BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im1 posx_im1=397 posy_im1=162 BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im2 posx_im1=155 posy_im1=403 BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im3 posx_im1=397 posy_im1=401 BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im4 posx_im1=154 posy_im1=161 BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im5 d=ds9() d.set_np2arr( transpose(BIG) )

18. Shift and Median is better!! Python HW2: Create a Python script doing this!

19. Median sky subtraction • When your object is extended… or you need to make a pretty picture… • in near-IR: useful for 10-20 min depends on the quality of the night.

20. In Astronomical observations… • Three major noise sources… • Poisson noise (aka “shot noise”) • Sky noise • Electronic noise • Sky noise limited observation • Dark current limited observation • Readout noise limited observation(?)

21. Noise versus signal at various wavelengths Extremely bright objects 109 Sky noise Detector noise Typical stellar sources in the Solar Neighborhood 105 Signal Strength High red shift galaxies 101 Optical near IR mid IR

22. Coadds • Because of the increased sky background, frames will get saturated in short exposures. • variable sky is the limiting factor  sky limited observations. • Take as many unsaturated frames as necessary and combined them later. • too many files for a single object (several hundreds) • on-chip combine of multiple exposures = “coadd”

23. Near-IR observation Tips • Need to sample the sky • through dithering • or from a dedicated sky observation in the nearby field  needs to be dithered also. why? • Choose a right number of coadds. • Choose the maximum frame exposure time which will stay in the linear regime. • Then, from the required total exposure time, calculate the necessary number of coadds. • Sky level is varying faster at longer wavelengths! • at K-band (2.2 μm) : up to 20 minutes • at L-band (3.6 μm) : max is about a minute • at mid-IR (10 μm) : fraction of a second  different strategy • Dither • Repeat the dither sequence as many times as necessary.

24. Observation Planning : near-IR • You will be observing at the K-band (2.2 micron), and you expect that your source would have a count rate of about 1,000 counts per second where this signal will be spread over ~10 pixels due to the PSF structure. Some other relevant data are • Detector linear regime: count < 10,000 counts • Sky brightness at K = 100 counts per second per pixel • Readout noise = 10 counts per readout • Dark current = 25 counts per second per pixel What would be the best observing strategy for this object if you want your final S/N~100?

25. Observation Planning : near-IR • Source ~ 2,000 count per second over 4 pixels. Goal S/N >= 100 • Detector linear regime: count < 8,000 counts • Sky brightness at K = 1,000 counts per second per pixel • Readout noise = 10 counts per readout • Dark current = 100 counts per second per pixel • Per pixel per second signal = 500 source count + 1,000 sky count + 100 dark current = 1,600 counts/pxl/sec In 5 sec exposure, the source will get into a non-linear regime  max exposure time should be less than 5sec or we should “coadd” • In a single 5 second exposure : S/N = 500 / sqrt(1600)~12  if we coadd 16 frames, then a single exposure will be 40 second long and S/N=12*sqrt(16)=48 • For a box5 dither observation pattern, we will get five images  total readout noise = 10 * sqrt(5) = 22.4 counts. This is negligible to shot noises from source+sky+detector. • If we take one image (40 sec long with 8 coadds) using a box5 pattern  final S/N=48 * sqrt(5) = 107 • The total duration of the observation is 5 x 40sec + total overhead (~1 min) = 4 min. At K-band, the sky will be invariable for up to ~20 min. So, this will be OK.

26. Nod & Shuffle • "Nod and Shuffle" is a technique used to obtain very faint spectra • the natural glow of the nighttime sky would overwhelm the extremely faint objects (e.g., far, far away galaxies) • N&S allows astronomers to effectively subtract away the bright spectral emission lines and fainter continuum of our atmosphere's nighttime glow while retaining the faint spectra of dim, red galaxies. This also significantly reduces readout noise. • obtaining a spectrum of an object • moving the electrical charge built up by the image of the spectrum on the CCD to another location "buffer storage" on the CCD (Shuffling) • shifting the position of the telescope slightly (Nodding) so that the spectrum of the target object shifts to a different part of the CCD • obtaining another spectrum in the nodded position • shuffling the charges moved in step 1 back to the original position (while moving the charges from step 4 to the buffer storage) • nodding the telescope back to the position in step 1 • repeating until enough light is collected.

27. A B A-B Sky cancellation: ‘nod and shuffle’ Storage of ‘sky’ image next to object image via ‘charge shuffling’Zero extra noise introduced, rapid switching (60s) Typically A=60s/15 cy: 1800s exposure10-3 subtraction

28. Another example

29. Weird image artifacts Latent images : badly saturated pixels do not come back to normal right away!

30. Weird image artifacts ghost images : multiple reflections within an optical element (e.g., filter, beam splitter, entrance pupil) seen near a very bright star

31. Handling various problems at IR

32. In summary… Important Concepts Important Terms Dither observation Shift and add Shift and median Coadd Nod-and-shuffle • Various noise sources at different wavelengths and their relation to the observing techniques. • Median combining of images • Chapter/sections covered in this lecture : 9