Observational Studies

1. Observational Studies Introduction to the Design and Analysis of Trials can be found on: http://www-users.york.ac.uk/~djt6/

2. Background • Stronger ‘quasi-experimental’ research methods involve control groups. Most health research use none randomly formed control groups. • Need to consider the advantages and disadvantages of non-randomised controlled studies.

3. Case control study • In a case control study ‘cases’ are identified (e.g., patients with angina) and these are matched with similar controls (e.g., patients of a similar age without angina). • The medical/lifestyle history of the two groups are then compared.

4. Case Control-study

5. Case control • Once cases have been matched to controls differences between the groups are examined. Any differences, (e.g., fruit and vegetable consumption) are then seen as potentially causative of the disease (e.g., Angina).

6. Case control data • Typically in a case control study we will observe a range of ‘risk factors’ for the disease. Among women with angina we will see: • Smoking; • Low oestrogen use; • Early menopause; • Low fruit and vegetable consumption.

7. Weaknesses • Case control studies have a number of weaknesses. • Recall bias – patients in the intervention group may selectively recall ‘risk factors’ more often than the control group giving a spurious correlation with outcome. • For example, people with Angina may selectively recall lower amounts of vegetable consumption compared with controls.

8. Weakness • Case control studies can be biased because of mortality effects. If a treatment incurs a mortality at the start of intervention then these patients won’t be available for a case control study.

9. Selection Bias • All non randomised studies are susceptible to selection bias. Case control studies are particularly susceptible. Selection of both the cases and controls by the researcher may be biased. • Controls may be selected from the same hospital which can lead to bias.

10. Interpretation of case - control • Assocations should NEVER been assumed to be causal in case-control studies. • Associations are pointers to further research – preferably using random allocation. • Why do case control studies?

11. Speed and Economy • Case control studies can usually be done quickly and relatively inexpensively; • For rare events (e.g. DVT among women on the pill) case control methods may be the only feasible approach. • Association of biochemical markers and Down’s syndrome was observed using case control methods.

12. Cohort Studies • These avoid some of the biases of the case control approach. • A cohort of people of selected and followed for a period of time and people who have an event are then compared with people who did not have an event.

13. Cohort study

14. Cohort • Because we measure various risk factors BEFORE the disease event then this eliminates some biases, such as recall bias, and therefore, cohort studies are more rigorous than case control methods.

15. Advantages • Relatively inexpensive (but not quick) compared to an RCT; • Can sometimes be quicker than an RCT as ethics committees ironically usually allow less scientifically rigorous studies through more quickly than the rigorous RCT; • Are an alternative when RCTs are not possible (either ethically or not appropriate).

16. Historical Controls • Some studies use historical controls to compare treatment effects. • This approach has even more potential for bias than when using contemporaneous controls (ie., usual care patterns might have changed over time).

17. Time Series analysis • When data are available over a long period of time we can look at changes due to policy changes. For example, data on percent of women being screened for cervical cancer showed little change for 25 years until government changed the payment mechanism.

18. Glue ear • No evidence that the use of grommets is an effective treatment for the treatment of glue ear. • A systematic review showing this was widely disseminated. • Did the review have an effect on practice? Mason et al, in a time series analysis suggested it did.

20. Does NICE work? • NICE guidance for hernia treatments does not support expensive laparoscopic surgery for this condition. • Not cost effective. • Did guidance reduce laparoscopic surgery? • Bloor et al in a time series analysis suggested it did not.

22. Problems with Time Series • The assumption is that any change in the trend is due to the intervention. There may be confounding due to other simultaneous policy initiatives. • Often when a problem is identified several initiatives may be introduced at around the same time and it is difficult to untangle what is the main effect.

23. Perils of Confusing Association with Causation • An association was noted that smokers who had high consumption of vegetables were less likely to develop lung cancer than smokers who did not. They also had higher blood levels of beta-carotene a natural anti-oxidant present in many vegetables.

25. Association gives rise to hypothesis • High vegetable consumption leads to high consumption of beta-carotene, which as an anti-oxidant neutralises ‘dangerous’ free radicals produced by the cells, which are a cause of cancer. Supplement with beta-carotene in pills among smoking men will reduce free radicals and reduce lung cancer risk.

26. Test hypothesis in RCT • Several RCTs launched to test this hypothesis. Male smokers randomised to receive beta-carotene supplements, several trials – Finland and USA. • What happened? Men taking beta-carotene INCREASED their risk of lung cancer. • Association does NOT mean causation.

27. Vitamin E • Another anti-oxidant, so beloved of ‘alternative’ practitioners, is the use of high dose vitamin E. • Cannot be harmful – BUT if it doesn’t have the potential for harm cannot do GOOD. • A meta-analysis of all the RCTs of vitamin E shows an INCREASE in mortality. Miller et al, Ann Intern Med 2005; 142:37-46.

28. Risk difference in all-cause mortality for randomized, controlled trials of vitamin E supplementation and pooled results for low-dosage (<400 IU/d) and high-dosage ([&ge;]400 IU/d) vitamin E trials Miller, E. R. et. al. Ann Intern Med 2005;142:37-46

29. High dose vitamin E • Most people who take vitamin E supplements (>60%) take daily doses above 400 IU a day, which is the dose associated with increased mortality.

30. Education • Hypothesis – use of ICT improves literacy and numeracy? • Observational data suggest ICT is ineffective for language teaching (in UK & Israel) and HARMFUL for numeracy acquistion. • Need RCT to test negative association?

31. Selection Bias • ALL observational data are susceptible to selection bias. • Selection bias is when people are ‘selected’ on characteristics that affect outcome. • This can often give a ‘spurious’ association.

32. Examples of selection bias • Women taking HRT are healthier than non-users; • Schools using ICT have more resources different teachers and pupils compared with schools not using ICT. • Male smokers with ‘naturally’ high beta-carotene intake are ‘different’ in some other way that makes them more resistant to carcinogenic effects of smoking.

33. Direction of Association • Use of dummies (pacifiers) are associated with early cessation of breast feeding. • 25% of children who use a dummy daily are weaned by 3 months compared with 13% who do not use a dummy (RR = 1.9, 1.1 to 3.3). • Conclusion: DON’T use Dummies?

34. Trial of Pacifier(Dummy Use) • WHO recommends that women should avoid using dummies as ‘observational’ data suggests a STRONG association with early weaning or no breast feeding with their use. • Kramer et al tested this hypothesis in a trial. In an intervention they reduced dummy use from 84% to 61%. Kramer et al, JAMA 2001;286:322.

35. Did this affect breast feeding? • NO – at 3 months 19% of those women with LOW dummy use had weaned their babies compared with 18% with HIGH dummy use. • HOWEVER, when the data were analysed as an observational study women using dummies were more likely to wean their babies. WHY?

36. Direction of Association • Women who for what ever reason did not or could not continue with breast feeding were using dummies to ‘pacify’ their infants. The decision to give up breast feeding CAUSED the association NOT vice-versa.

37. Can observational data ever infer causation? • Many epidemiologists believe it can, 6 requirements to assess causation: • Consistency of association • Proper time sequence • Dose response • Strength of association • Change in risk with change in exposure • Biological plausibility

38. Example, lung cancer and smoking • Consistent association • development of lung cancer and smoking in both sexes across numerous countries; • Time sequence • Smoking occurs before lung cancer (development of cancer does not preceed smoking); • Dose-response • Heavy smokers have higher risk than light smokers.

39. Smoking (cont) • Strength of association • Very high relative risk of lung cancer associated with smoking (e.g., 19/20 cancers in smokers). • Change in risk with exposure • Former smokers have lower risk than current smokers; • Biological Plausibility. • Animal and tissue data show that exposure to contents of tabacco smoke increase cell proliferation and cancer.

40. BUT • The power of selection bias means that a treatment can pass all these criteria and the relationship is still not causal.

41. Whither Observational Data? • Observational data is good for hypothesis forming, cheap easily collected before an expensive trial. • Good for looking at ‘risk factors’. • Helpful in areas where trials are not possible (e.g., policy changes).

42. Disadvantages • Prone to selection and other biases. • May often give the ‘wrong’ answer. • Cannot be used to ‘prove’ an causal effect. • If at all possible findings from observational data need confirming using trial data.

43. Observational Studies • Any controlled study IS better than an uncontrolled evaluation. • BUT non-randomised studies are susceptible to a wide range of bias and should only be used to convict a suspect if no randomised data are available.

44. Cohort Complete source population Estimate incidence rates and relative risks Takes long time Expensive Useful for broad range of diseases and exposures Case control Sampling from source population Estimate only odds ratios Shorter and cheaper Focused around one disease Comparing cohort and case-control

45. Threats to internal validity • Selection bias • Self selection • Investigator selection • Information bias • Biased effect size because some subjects are more likely to attain a positive or negative outcome score than others • Confounding • Effect size explained by other factors

46. Selection bias • When the association between the disease/event and exposure/intervention differs for those who participate in the study and those who don’t • Effect of breast cancer screening by comparing breast cancer rates in volunteers and those not tested • Selection bias through self-selection (patient/clinician) • Those who get the intervention are more likely to benefit from it (observed and unobserved characteristics) • Selection bias due to choices made by the investigator • Comparison of workers health with those of general population (healthy worker effect)

47. Volunteer Bias Breast Cancer Incidence Rates in a Screening Trial (per 1000/yr)

48. Investigator selection Case-control studies • Selection of cases • Exposed patients are more likely to be be included in the study than non-exposed patients • Selection of controls • Exposed controls are less likely to be included in the study than non-exposed controls

49. Information bias Information collected about exposure or outcome are wrong Misclassification • Non-differential • All epidemiological studies to some extent • Always biases estimate of association towards 1 • Differential • Recall bias (e.g. maternal recall bias) • Biased follow-up (e.g. unexposed people under diagnosed compared to exposed) • Biases estimate of association in different directions

50. Information bias Measurement errors in variables • cohort study • more adequate measurement of results in exposed in comparison to non-exposed • case-control • more adequate detection of exposure in cases in comparison to controls