Estimation : Precision and accuracy , standard error , confidence intervals

1. Estimation:Precisionandaccuracy, standard error,confidenceintervals

2. Accuracyand Precision

3. Accuracy How close a measurement is to the actual or “true value” highaccuracy true value lowaccuracy true value

4. Precision How well several measurements agree with each other high precision low precision

5. Precision • Precision or the reproducibility of a set of measurements. • A precise sample estimate will have a very smallrandom error of estimation.

6. Accuracy

7. Precision

8. Accuracy and Precision What can you say about the accuracy and precision in each of the following: High precision, low accuracy High precision, high accuracy

9. Question: How will each of the following affect accuracy and precision? 1. A meter stick that is missing the first centimeter. 2. A scale that has a zero point that is really five pounds above zero.

10. Solution How will each of the following affect accuracy and precision? 1. The shortened meter stick will produce measurements that have poor accuracy, but good precision is possible. 2. The poorly calibrated scale will give a weight that is not accurate, but good precision is possible.

11. A blood sample was taken from a patient and four different assays were used to measure blood glucose. The true value of the blood glucose was known to be 4.5 mmol/L. State whether the accuracy and the precision of each assay is high or low: • Assay A: 4,6; 4,6; 4,8; 4,5; 4,5; 4,4; • Assay B: 4,3; 3,5; 5,3; 4,6; 5,5; 3,7 • Assay C: 3,5; 3,6; 3,3; 3,5; 3,4; 3,5 • Assay D: 8,5; 6,4; 5,3; 7,6; 4,8; 9,3

12. Literature incosistency Highaccuracy and lowprecision or lowaccuracyandlowprecision?

13. How confident are we in theestimationofmeanor proportionwe havecalculated?

17. Measures of precision: • Standard errorofmean, SEM Standard errorofproportion, SE(p) • Confidence interval for mean, CI Confidence interval for proportion, CI

18. Standard error of mean, SEM Standard deviation, SD • SEM is smaller (estimate is more precise): • the larger is N (number of patients) • the smaller is SD (dispersion of data) Number of patients

19. 95% confidence interval for mean, 95% CI • Together with SEM, 95% CI is also the measure of precision • Unlike SEM, 95% CI also estimates accuracy of the result ie. 95% is accurate that interval includes true (population) mean

20. 95% confidence interval for mean • Ifwedraw a 100 samplesfromourpopulationwewouldfindthetruepopulationvaluewithin 95% confidence interval in 95 samples. Inexample for 20 samples:

21. Criticalvalues for 90%, 95% and 99% levelofconfidence 90% CI => mean ± 1.65 SEM 95% CI => mean ± 1.96 SEM 99% CI => mean ± 2.58 SEM Level of Confidence - Critical Value 0.75, or 75% 1.15 0.80, or 80% 1.28 0.85, or 85% 1.44 0.90, or 90% 1.65 0.95, or 95% 1.96 0.98, or 98% 2.33 0.99, or 99% 2.58

22. Example 1 • The average systolic BP before treatment in study A, of a group of 100hypertensive patients, was 170 mmHg. After treatment with the new drugthe mean BP dropped by 20 mmHg. • If the 95% CI is 15–25, this means: we can be 95% confident that the true effect of treatment is to lower the BP by 15–25 mmHg.

23. Example 2 • In study B 50 patients were treated with the same drug, also reducingtheir mean BP by 20 mmHg, but with a wider 95% CI of -5 to +45. This CIincludes zero (no change). • This means: Notrue change in BPwasdetermined (the drug mightbe actuallyineffective)

24. Watch out for... • The size of a CI is related to the sample size of the study. Larger studies usually have a narrower CI.

25. Widthof CI

26. Example 3 – Meta analysis Fig. Plot of 5 studies of a new antihypertensive drug. Which study showed the greatest change? Did all the studies show change in favour of theintervention? Were the changes statistically significant?

27. Proportion

28. Proportion • Standard errorofproportion, SE(p) SE(p) = √(p(1 – p)/n) • Confidence interval for proportion

29. The standard deviation describes the variability of a sample; The standard error of the mean(SEM)doesnotdescribethesample but describes the uncertainty of how the sample mean represents the population mean.

30. Commonmistakeinthe literature Misuse of standard error of the mean (SEM) when reportingvariability of a sample. A critical evaluation of four anaesthesiajournals. P. Nagele* British Journal of Anaesthesia 90 (4): 514-16 (2001)

31. SD CI • Standard deviation tells us aboutthe variability (spread) in a sample. • The CI tells us the range in which the true value (themean if the sample were infinitely large) is likely to be.

32. Krebs NF, Westcott JE, Culbertson DL et. al. Comparison of complementary feeding strategies to meet zinc requirements of older breastfed infants.Am J ClinNutr. 2012; 96:30-35 Meat Fe&Zn Fe CI

33. Radial extracorporeal shockwave treatment compared with supervised exercises in patients with subacromial pain syndrome: single blind randomised study Mean shoulder pain and disability index scores with 95% confidence intervals for supervised exercises and radial extracorporeal shockwave treatment

34. What does a small standard error tell us about the sample estimate of the mean? (Y/N) • That it is highly variable • That the population standard deviation may be small • That the sample size is probably small • That it is imprecise

35. What will tend to make the standard error larger? (Y/N) • A small variance • A large standard deviation • Imprecise data • Inaccurate data