Getting an estimate of % of GM in a sample 2. Qualitative laboratory methods

1. Getting an estimate of % of GM in a sample2. Qualitative laboratory methods May 8-10, 2006 Iowa State University, Ames – USA Jean-Louis Laffont Kirk Remund

2. Overview • Impurity estimators and confidence intervals • Quantitative information from a qualitative assay • Limitations to quantification with a qualitative assay ISTA Statistics Committee

3. µ=1% truth µ= lot impurity/purity (sometimes called p) impurity/purity estimate Impurity Estimate ? Our best guess of what the true lot impurity/purity is based on the sample… ISTA Statistics Committee

4. + - - - - - - - - + - - - - - + - - - - - - - - - + - - - - - + Estimate based on sample Lot Sample ISTA Statistics Committee

5. Oops, looks like one got away!! Confidence intervals are like nets… Then what are we trying to catch? Answer: true level of impurity (µ) in the lot Lot 3 Lot 2 µ3 Lot 1 µ2 µ1 ISTA Statistics Committee

6. Confidence level • Net “interval” size is function of sampling variability, assay errors and confidence level • If we fix the sampling and assay variability then: lower conf. level small higher conf. level large ISTA Statistics Committee

7. µ expect 5% of time µ will fall out of net What does 95% confidence mean? • Statement: “We are 95% confident that the true lot impurity is contained within the interval (net)” • Overall we expect that 95% of the time the interval will catch the true lot impurity (µ) µ µ µ µ µ µ µ µ ISTA Statistics Committee

8. Estimator of GM purity/impurity(Individual Seed Testing) • Estimator: • UCL: where F is the 1- quantile from an F-distribution with 2d+2 and 2n-2d degrees of freedom • Individual seed testing used to test purity of GM • material for proficency test • Used to test purity of GM variety seed • Implemented in Seedcalc ISTA Statistics Committee

9. Estimator of GM impurity(Seed Pool Testing) • Estimator: where m is the number seeds per pool, n is the number of seed pools and d is the number of deviant seed pools • UCL: • Used to estimate AP levels of GM in conventional seed • Used to estimate level if GM impurity in conventional • seed for proficency test • Implemented in Seedcalc ISTA Statistics Committee

10. 1 & 2 sided Confidence limits • Upper confidence limit (UCL) • “95% confident that true impurity is below upper confidence limit” • Caution: do not use as estimate • Two-sided confidence limit • “95% confident that the true impurity is between the lower and upper limit • Similar to form on earlier slide formulas and is implemented in Seedcalc ISTA Statistics Committee

11. Two-sided confidence interval (put ½ of alpha in each tail) 1- confidence that interval contains true purity of lot 1- confidence interval /2 /2 ISTA Statistics Committee

12. One-sided confidence interval (alpha in one tail) 1- confidence that interval contains true purity of lot 1- confidence interval  ISTA Statistics Committee

13. The following slides illustrate that a simple presence/absence answer per pool of seeds allows estimation of % of seeds presence The statistical computation takes into account the fact that for a given level of GM presence, some sub-samples will “by chance” contain 0 GM seeds, others 1 GM seed, others 2 GM seeds, etc.. The formula is : Where d is the number of deviant sub-samples , n is the number of sub-samples, m is the number of seeds per sub-sample ISTA Statistics Committee

14. ISTA Statistics Committee

15. From 1500 seeds, 10 pools of 150 seeds have been made ISTA Statistics Committee

16. Each sub-sample is tested for presence/absence of GM seeds 4 sub-samples are positives Positive control 0.34% Negative control ISTA Statistics Committee

17. % estimate can be obtained in Seedcalc or in ISTA documents 4 positive pools from 10 pools of 150 seeds => 0.34% of GM seeds ISTA Statistics Committee

18. Statistical computation take into account that some sub-samples may have more than a GM seed ISTA Statistics Committee GM seed ( 5 GM Seeds 3 times 5 positive, 1 time 4 positive)

19. - - - seed + - - - seed + + - - seed Qualitative Test/Quantitative Information Example of seed pool testing strategy: - <0.25% <0.46% <0.77% (4 pools of 300 seeds) ISTA Statistics Committee

20. + + - - seed # positive seeds Probability* 2-3 ~65.4% 4-5 ~24.9% 6-7 ~7.8% 8-9 ~1.4% >9 ~0.4% How confident are we that the qualitative data is appropriate to describe a quantitative result? Distribution of attribute within pooled samples: How many positive seeds in 2 positive pools? (4 pools of 500 seeds) <0.46% 0.14 = Best Estimate all seeds negative (1000 seeds) * Probability of set number of positives given that two pools are negative ISTA Statistics Committee

21. Inputs Outputs ISTA Statistics Committee

22. Threshold Testing VS UCL LQL UCL yields more information than threshold testing 0.0% 0.5% 1.0% 1.5% 2.0% ISTA Statistics Committee

23. Threshold Testing VS UCL ISTA Statistics Committee

24. Estimation limitations for small # of pools Real-time PCR assays also has Problem estimating higher AP impurity levels due to asymptote of cycles at higher impurity ISTA Statistics Committee

25. Limited information if all pools are positive • Test 10 pools of 300 seeds and all are positive • Impurity estimate = 100% BUT • 95% confident that impurity in lot is between 0.45% and 100%!!! • Test 3 pools of 1000 and all positive • 95% confident that impurity in lot is between 0.05% and 100%!!! ISTA Statistics Committee

26. Demonstration and Exercises in Seedcalc ISTA Statistics Committee