Center for Biofilm Engineering

1. Center for Biofilm Engineering The Importance of Statistical Design and Analysis in the Laboratory Al Parker, Biostatistician Standardized Biofilm Methods Research Team Montana State University Feb, 2011

2. Standardized Biofilm Methods Laboratory Al Parker Lindsey Lorenz Marty Hamilton Darla Goeres Paul Sturman Diane Walker Kelli Buckingham-Meyer

3. What is statistical thinking? • Data • Experimental Design • Uncertainty and variability assessment

4. What is statistical thinking? • Data (pixel intensity in an image? • log(cfu) from viable plate counts?) • Experimental Design • - controls • - randomization • - replication (How many coupons? experiments? technicians? labs?) • Uncertainty and variability assessment

5. Why statistical thinking? • Anticipate criticism (design method and experiments accordingly) • Provide convincing results (establish statistical properties) • Increase efficiency (conduct the least number of experiments) • Improve communication

6. Standardized Methods Why statistical thinking?

7. Attributes of a standard method: Seven R’s • Relevance • Reasonableness • Resemblance • Repeatability (intra-laboratory) • Ruggedness • Responsiveness • Reproducibility (inter-laboratory)

8. Attributes of a standard method: Seven R’s • Relevance • Reasonableness • Resemblance • Repeatability (intra-laboratory) • Ruggedness • Responsiveness • Reproducibility (inter-laboratory)

9. Resemblance of Controls Independent repeats of the same experiment in the same laboratory produce nearly the same control data, as indicated by a small repeatability standard deviation. Statistical tool: nested analysis of variance (ANOVA)

10. Resemblance Example: MBEC • 86 mm x 128 mm plastic plate with 96 wells • Lid has 96 pegs

11. MBEC Challenge Plate disinfectantneutralizer test control

12. Resemblance Example: MBEC ControlData: log10(cfu/mm2) from viable plate counts Mean LD= 5.55

13. Resemblance Example: MBEC

14. Resemblance from experiment to experiment Mean LD = 5.48 Sr = 0.26 the typical distance between a control well LD from an experiment and the true mean LD

15. Resemblance from experiment to experiment The variance Sr2 can be partitioned: 2% due to between experiment sources 98% due to within experiment sources

16. Formula for the SE of the mean control LD, averaged over experiments Sc= within-experiment variance of control LDs SE= among-experiment variance of control LDs nc = number of control replicates per experiment m = number of experiments 2 2 2 2 S S E c SE of mean control LD = + m nc • m CI for the true mean control LD= mean LD ± tm-1 x SE

17. Formula for the SE of the mean control LD, averaged over experiments Sc = 0.98 x (0.26)2= 0.00124 SE = 0.02 x(0.26)2= 0.06408 nc= 8 m = 2 2 2 0.06408 0.00124 = 0.1792 SE of mean control LD = + 2 8 • 2 95% CI for the true mean control LD= 5.48 ± 12.7 x 0.1792 = (3.20, 7.76)

18. Resemblance from technician to technician Mean LD = 5.44 Sr = 0.36 the typical distance between a control well LD and the true mean LD

19. Resemblance from technician to technician The variance Sr2 can be partitioned: 0% due to technician sources 24% due to between experiment sources 76% due to within experiment sources

20. Repeatability Independent repeats of the same experiment in the same laboratory produce nearly the same data, as indicated by a smallrepeatability standard deviation. Statistical tool: nested ANOVA

21. Repeatability Example Data: log reduction (LR) LR = mean(control LDs) – mean(disinfected LDs)

22. Repeatability Example: MBEC

23. Repeatability Example: MBEC Mean LR = 1.63

24. Repeatability Example Mean LR = 1.63 Sr = 0.83 the typical distance between a LR for an experiment and the true mean LR

25. Formula for the SE of the mean LR, averaged over experiments Sc= within-experiment variance of control LDs Sd= within-experiment variance of disinfected LDs SE = among-experiment variance of LRs nc = number of control replicates per experiment nd = number of disinfected replicates per experiment m = number of experiments 2 2 2 2 2 2 S S S E d c SE of mean LR = + + m nd • m nc • m

26. Formula for the SE of the mean LR, averaged over experiments Sc= within-experiment variance of control LDs Sd= within-experiment variance of disinfected LDs SE = among-experiment variance of LRs nc = number of control replicates per experiment nd = number of disinfected replicates per experiment m = number of experiments 2 2 2 CI for the true mean LR = mean LR ± tm-1 x SE

27. Formula for the SE of the mean LR, averaged over experiments Sc2 = 0.00124 Sd2= 0.47950 SE2= 0.59285 nc = 8, nd= 5, m = 2 0.59285 0.00124 0.47950 SE of mean LR = = 0.5868 + + 2 8 • 2 5 • 2 95% CI for the true mean LR = 1.63± 12.7 x 0.5868 = 1.63± 7.46 = (0.00, 9.09)

28. How many coupons? experiments? 0.59285 0.00124 0.47950 + + margin of error= tm-1 x m nd • m nc • m

29. Responsiveness A method should be sensitive enough that it can detect important changes in parameters of interest. Statistical tool: regression and t-tests

30. Responsiveness Example: MBEC A: High Efficacy H: Low Efficacy disinfectantneutralizer testcontrol

31. Responsiveness Example: MBEC This response curve indicates responsiveness to decreasing efficacy between rows C, D, E andF

32. Responsiveness Example: MBEC Responsiveness can be quantified with a regression line: LR = 6.08 - 0.97row For each step in the decrease of disinfectant efficacy, the LR decreases on average by0.97.

33. Summary • Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria. • Good experiments use control data! • Assess uncertainty by SEs and CIs. • When designing experiments, invest effort in • more experiments versus more replicates • (coupons or wells) within an experiment.

34. Any questions?