1 / 6

The SPIRE Quality Control Pipeline

The SPIRE Quality Control (QC) Pipeline, developed by Dave Clements and George Bendo, aims to ensure the integrity of collected data. The pipeline checks for proper data collection, identifies issues such as high glitch rates, and verifies that SPG processing runs smoothly. Although it adopts the notion that "there is no QC pipeline," it automatically generates quality control reports using metadata generated by operational modules. The QC plug-in is applicable to various product levels and is designed to enhance data quality through thorough examination and reporting mechanisms.

marcie
Télécharger la présentation

The SPIRE Quality Control Pipeline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The SPIRE Quality Control Pipeline Dave Clements George Bendo

  2. Goals of the QC Pipeline • Determine whether the data was collected properly • Determine whether there are any problems with the data • Eg. excess glitch rate • Determine whether SPG processing proceeded normally • Produce quality control report flagging up any problems for human examination

  3. The QC Pipeline • The first rule of the Quality Control Pipeline is • There is no quality control pipeline! • All modules in normal operation generate quality control information • Collected in product metadata • A QC plug-in at the end of SPG operation examines these • Produces an automatic QC report and flags any problems for further investigation

  4. QC ‘pipeline’ Concept

  5. Application • Can apply QC plugin to • Level 0.5 products • Level 1 Products • Level 2 Products • In theory to intermediate products as well • Works with multiple building blocks where appropriate • eg. Level 0.5 products

  6. Status • QC plugin written and running • Some debugging and optimizations required • Past tests in E2E tests • Naming conflicts and full list of metadata from modules still need to be sorted • Module developers need to make sure modules fulfill QC requirements

More Related