Overview of SPIRE Photometer Pipeline

1. Overview of SPIRE Photometer Pipeline C. Kevin Xu (NHSC/IPAC)

2. Goals: • Show how SPIRE Photometer pipeline works (functionalities of major modules). • Explain what is new in HIPE 8. • Brief summary of remaining issues. • Will concentrate on scan map “user pipelines” (covering small map, large map, • SPIRE/PACS parallel mode).

3. User Pipelines • User pipelines (Jython scripts): Simplified version of Standard • Product Generation (SPG) pipelines. • You can find these “user pipelines” in HIPE:

4. SPIRE Pipeline & Data Products 5’ (in HIPE) Our focus (“user pipelines”)

5. Scan Map Pipeline Flow Chart A “reversed sequence” relative to the chain of data acquisition place-holder • FFT based • Default: concurrent deglitcher • + wavelet deglitcher • Alternative: σ-κ deglitcher • Telescope pointing & • orientation • SPIRE detectors positions FFT based (run after deglitch & repair to avoid ringing) • Choices: • Default: scan median baseline removal • Advanced : polynomial baseline removal • Most advanced: an iterative destriper • Include: • Non-linearity correction • Volt to Jy/beam conversion Baseline removal + Mapper Remove 1/f noise due to T-drift • Default: naïve mapper • Alternative: madmapper

6. Uniform scan speed distance Scan-By-Scan Processing • The pipeline processes timelines scan by scan (to ease the demand on RAM). • Problem: ringing at the two ends of each scan due to FFT based modules. • Solution: • (1) Before the process, attaching “turn-around” data blocks to ends of the scan. • (2) During the process, the ringing is confined to the “turn-around” data. • (3) After the process, cut-off the “turn-around” data blocks from the scan. Turn-around

7. Highlights of a User Pipeline (Jython Script) New in HIPE 8.0 (for extended source, optional)

8. Sudden (spontaneous?) jump in the timeline of a single channel. • Mostly seen in thermistors (see below). Very rarely seen in detectors. • The average frequency is ~ 1/day ( a few hundred instances since launch). • The cause is still unknown. • Effect: The pipeline uses thermistor timelines in the correction for detector signal • drift due to bath temperature drift (major source of 1/f noise for SPIRE). A • thermistor “jump” affects this correction, introducing artificial stripes in the final map. Signal “Jump” Stripe caused by the jump Jump

9. Thermistor Jump Solution (in HIPE 8.0) • a “reversed sequence” relative to the chain of • data acquisition • scan by scan processing New module Jump Detector (automatically identify & mask any thermistor affected by a jump) Destriper + Mapper Exclude masked thermistors

10. Status of SPIRE Scan Map Pipeline in HIPE 8.0. SPIRE 3-color map of NGC 5315 (a planetary nebula) • General assessment: • In many cases, data from HSA • are already science quality! • The official calibration accuracy is • ±7% (5% from model, 2% RMS). • An example (on the right): • The image from HSA looks good. • Caveat: detailed inspections show • thin stripes (can be improved using • a more advanced baseline remover • such as the iterative destriper). • A major issue yet to resolve: • stripes due to bias voltage • drift (the “burp”) • (affecting a few observations) (Public data taken from HSA)

11. Bias Drift in “Burp” Period • Every time when SPIRE is switched on after a • cooler recycle, the first ~6 h sees a rapid rise • of bias voltage (the “burp”). • This effect has not been corrected in pipeline. • It causes stripes in maps observed • during the “burp” period (see example below). • A pipeline module is now being developed to • correct this effect (available soon). • For point source surveys, the stripes might be • removed by polynomial baseline removal. Resistor voltage  After cooler recycle 2 (~40 h) After cooler recycle 1 (~40 h) An example of stripes caused by bias drift bias drift ~10σ Bias voltage “burp” ~ 6h (Standard pipeline product)

12. Summary • Scan-Map pipeline covers nearly all SPIRE PHOT AOTs (small map, • large map, map in SPIRE/PACS parallel mode). • It follows a “reversed sequence” relative to the chain of data acquisition. • For a general user, Level0_5 should be the best starting point. • From Level0_5 to Level 1, the pipeline processes any observation data set • scan by scan (to ease the demand on the RAM). • The current pipeline (HIPE 8.0.1) does a good job (“science ready”) in general. • However, the pipeline still has some open issues: that • (1) Stripes due to the “burp effect” (affecting ~10% observations). • (2) Residual stripes (can be corrected by the destriper module in HIPE). • (3) Residual glitches in maps (2nd level deglitcher being developed inside • the destriper module).

13. Home work: • Run Photometer Large Map user pipeline (with your own data or NGC 5315 data). • Next: A demo of SPIA (SPIRE Photometer Interactive Analysis), a GUI based • data reduction tool (including all pipeline modules and more).