1 / 24

Validation of Screening Methods

Validation of Screening Methods. Aisling Treacy, MSc. Prof. Tom Buckley, MSc, FIBMS, FAMLS. Background. Established in 1984 – Equine diagnostic laboratory Screening laboratory during the “Angel Dust Era”

dlail
Télécharger la présentation

Validation of Screening Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validation of Screening Methods Aisling Treacy, MSc. Prof. Tom Buckley, MSc, FIBMS, FAMLS

  2. Background • Established in 1984 – Equine diagnostic laboratory • Screening laboratory during the “Angel Dust Era” • Antibiotic residue testing for the porcine industry where there was approx. 25% non-compliance → currently < 1% non-compliance • Screening laboratory for the NRCP → obtained ISO 17025 Accreditation for NRCP tests in 2005 → Currently accredited for 94 analyte / matrix / species combinations • Rapid screening laboratory for Phenylbutazone (PBZ) during the horse meat scandal – 2013 • Self Monitoring Plan • Equine anti-doping testing → testing other species

  3. Pre-validation criteria to be considered • Analyte(s) – regulatory limit – MRL, MRPL, RPA • Selection of the method – LOD, cross-reactivity, single / multi-analyte method, reliability, matrix • Client requirements • Review of relevant regulatory or guidance documents • Commission Decision EC/2002/657 • CRL Guidance Paper of 7 December 2007 • Community Reference Laboratories Residues (CRLs) 20/1/2010 • ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories

  4. Pre-validation work • Test performance – recovery of spikes (various levels), reproducible results • Matrix / species effects • Sample treatment / optimum extraction

  5. Study design / Validation plan • Analyte, matrix and species to be validated • CCβ, Screening Target Concentration → validation spiking level • Number of samples to be analysed • Factors to be included in ruggedness • Specificity testing with related analytes • Applicability • Stability testing • Repeatability / method verification Initial validation Further validation

  6. Performance characteristics to be determined according to EC/2002/657 • Semi-quantitative screening: • CCβ • Precision • Selectivity / specificity • Applicability • Ruggedness • Stability

  7. CCβ Detection capability • CCβ: • Detection capability CCβ – is defined as the smallest content of the analyte that may be detected, identified and/or quantified in a sample with an error probability of β • The β error is the probability that the tested sample is truly non-compliant even though a compliant measurement has been obtained • For screening tests the β error (i.e. false compliant rate) should be <5% • For analytes with a regulatory limit, CCβ must be ≤ the regulatory limit • For analytes with no regulatory limit, CCβ must be as low as possible • CCβ must be established for analytes across individual matrices • LOD: • A consideration when selecting a method • CCβ and threshold values to describe the detection capability of the method

  8. Determination of number of samples required for validation and screening target concentration • The STC in relation to the regulatory limit will determine the number of samples required for validation

  9. Implementation of validation • Determination of STC and number of samples required for validation • Analyses carried out on 3 different days by different analysts

  10. Inclusion of other species and matrices / applicability • Separate validation is carried out for each individual matrix type, e.g. serum and muscle • If multiple species are to be validated within one matrix type, then the 60 samples could comprise different species, e.g. 20 x bovine, 20 x porcine and 20 x ovine • If an additional species is to be included after validation is complete, 20 spiked and 20 blank samples made up of the additional species may be analysed and added to the initial validation for the calculation of threshold value and cut-off value

  11. Test specificity • Testing of blank samples to determine threshold will demonstrate whether there is matrix interference – calculation of threshold values • Testing of similar analytes – recovery should fall below cut-off value established during validation

  12. Calculation of cut-off value • Two approaches to establishing cut-off levels outlined in CRL Guidelines for the Validation of Screening Methods for Residues of Veterinary Medicines 2010: • Approach I – Lowest response The lowest response in the spiked samples analysed during validation is taken as the cut-off giving a response greater than this level is deemed to be ‘screen positive’ and exceeds the CCβ of the screening method • Approach II – Statistical approach Cut-off factor (FM) = Mean Response – 1.64 x SD

  13. Calculation of cut-off value • The multiplication factor approach to establish a cut-off value • The IEC uses a multiplication factor applied to the mean of two spiked ‘cut-off’ samples analysed alongside routine samples to establish a cut-off value for each test run • A one-tailed 95% confidence interval of the validation data generated from spiked samples is calculated and then divided by the mean (M) • To apply this fraction to the results of routine spiked cut-off samples, this figure is subtracted from 1 to give a multiplication factor to be applied to the mean of the spiked cut-off samples • Multiplication factor = 1 - (1.64 x Std. Dev.) M • In routine testing, any sample with a reading at or above the calculated ‘cut-off applied’ will be considered ‘screen positive’ or non-compliant and forwarded for confirmatory analysis • This approach to cut-off calculation has been approved by regulatory authorities and INAB auditors

  14. Validation data generated from analysis of three related analytes

  15. Ruggedness • Introduction of minor reasonable variations and observations of consequences • The Youden approach to ruggedness is employed at the IEC – facilitates the introduction of several variations simultaneously • Factors that may influence measurement result are selected: • Pre-treatment • Clean-up, SPE • Analysis • Factors that influence results are subjected to further testing – these factors are described in the validation report and may be included in the SOP

  16. Ruggedness parameters investigated

  17. Ruggedness results Mean results for each parameter Differences between normal and varied parameters • Ruggedness Study Conclusion - The only parameter that showed significant difference was elution buffer of the immunoaffinity column procedure. This is controlled in the procedure. This test has proven to be robust against all other changes to the parameters included in the ruggedness testing

  18. Stability • The degradation of the analyte under different conditions relevant to the test / laboratory • Observed through ongoing monitoring of QC materials (negatives and spikes) – plotted on Shewhart charts • Stability experiments carried out if any degradation is observed over time • Stability experiments – assessment of stability of analytes: • Stability of analytes in solution • Stability of analytes in matrix • Assessment carried out in line with EC/2002/657 and SOP P5.5.135

  19. Repeatability • Assessment of spiked samples over three days • Calculation of Relative Standard Deviation measure of within lab reproducibility  must be < 35%

  20. Validation acceptability criteria – Was validation successful? • RSD < 35% • Approach I – No overlap between threshold value and cut-off value • Approach II – The rate of false positive is acceptable (i.e. 5%) when FM > T if B < FM < T the false positive rate is higher than 5% • According to Commission Decision 2002/657/EC, CCβ is validated when FM > B • If the above criteria are met the method is considered robust, specific and fit for purpose

  21. Validation data generated from analysis of three related analytes

  22. When is additional or re-validation required? • Additional validation carried out when extra species are introduced or minor changes to an existing method • Re-validation will occur if: • Significant change to the method • Method fails on a continual basis • Change in client requirements, e.g. lower STC

  23. Continuous verification • QC samples • Negative control (TQCN) • Positive controls (Cut-off 1 and 2) • Check sample (blind to analyst) • Spiking analyte – the analyte with the poorest recovery (worst case / lowest specificity) is used to spike QC or most relevant analyte according to test • Last 20 sets of QC analyses plotted on Shewhart charts - ≤ 1 outlier per 20 analyses, i.e. ≤ 5% • Verifies overall method performance, analyst accuracy / performance, spike stability, kit performance / stability • PT’s – CRL / State Lab submit samples for PT analysis for each analyte at least once yearly • External providers – Progetto Trieste, FAPAS • In-house • Out of specification results subjected to an in-house investigation procedure

  24. Thank You

More Related