Download
1 / 20
200 likes | 316 Vues
The UKIDSS project maintains a comprehensive archive of WFCAM data, processing approximately 2000-3500 images nightly and addressing operational challenges like data transfers and quality control. This document provides an overview of the operational aspects, including data lifecycle management, from the arrival of raw data and processing to data transfer to ESO. Insights into flat field stability and sky properties are discussed, emphasizing the importance of regular sky flat observations. For further details and data availability, refer to the CASU website.
E N D
UKIDSS SV • operational aspects, archives, • flat & sky properties Marco Riello CASU - IoA mriello@ast.cam.ac.uk
Outline • Operational aspects • WFCAM archives & status pages • Flats fields and skies properties
Operations: overview • Life cycle of WFCAM data at CASU: • Raw data arrive at CASU: dump LTO tapes to disk. • MEF creation + detection/logging of missing files. • Pipeline processing. • DQC information generation. • Transfer processed data to Edinburgh. • Raw MEF transfer to ESO (UKIDSS + calibrations only).
Operations: some figures • Data volume (½ UKIDSS, ½ Private P.I.): • ~13 (3.3) Tb of raw data • ~21 (5.8) Tb of processed data • Transfers to ESO fairly slow: 0.5-0.7 Mb/sec. Only UKIDSS and calibration data, 7 nights left to transfer. • Pipeline processing is keeping up well: • ~2000-3500 images per night. Typically ~24 hr/night • 8 nights simultaneously processed (1 CPU/night). • GPS data challenging: 120000-250000 detections/MSB
Pipeline processing • Reduction is group-based. • Dark and flat field correction + de-curtaining. • Creation of “sky” frames: correct for residual patterns (e.g. crud on chip #1) and reset anomaly rather than removing the sky. • Interleaving (if required) • Stacking • Catalogue creation & astrometry • Photometry
Public WFCAM facilities at CASU • WFCAM raw data archive: data availability for registered users according to UKIDSS policy. Private P.I. data. • WFCAM Reduction progress monitor. • Documents and information.
K H J Y Z K H J Y Z DQC Plots 1
chip #1 #2 #3 #4 chip #1 #2 #3 #4 DQC Plots 2
K H J Y Z DQC Plots 3
Flat Properties The stability of the flat fields have been checked over the period April-June 2005 in all the broad-band filters: • The noise properties are stable within 0.3% in JHKYZ. • Large scale variations (corner-to-corner gradients) are presents between different flats: • Variations over a 15-20 days period usually less than 0.5% • Variations over a ~1 month period usually less than 1% • Variations over a >1 month period can be usually ~1% but can be > 2% • Conclusion: take sky-flats twice a month or, at least not less than every 20 days
H June 19th April 7th
Y April 12th April 7th
Sky Properties • Algorithm similar to fringing correction: combine 4-12 science frames with minimum separation of 0.1 arcmin. • We have several sky frames per filter per night: each MSB is corrected using the closest sky frame in time. • The CRUD feature on chip #1: • It appears as a diffuse patch of light with some brighter spots. It is possibly due to scattered light. • It is due to the presence of dirt/dust on some of the camera lenses. Will disappear from semester 2005B.
CRUD on chip #1 K 100 px/40”
K NM FM
