Part I: Experimental Design Chapter 2

1. Part I: Experimental DesignChapter 2 Observational Studies Pages 11-12 only Thursday’s Lecture

2. Page 11 OBSERVATIONAL STUDIES CONTROLLED EXPERIMENTS The researcher has NO power over assignment into treatment and control groups. Researcher decides how to divide the subjects into treatment and control groups. • Subjects themselves or simply fate determines who gets the treatment and who doesn’t. • Researcher just observes what happens. • Subjects may be randomly assigned to treatment and control. • Subjects may be non-randomly • assigned to treatment and control. Note that both observational studies and controlled experiments compare treatment and control groups. The key difference is how the subjects got into the treatment or control groups. If the researcher assigned them, it’s a controlled experiment. If not, it’s an observational study. Some studies have to be observational. Examples: • Effects of malnutrition--researcher cannot starve people for a study. • Effects of alcohol on fetal development.

3. Page 11 Observational studies are done out of necessity. Whenever possible it is better to do a controlled experiment. Main Problem of Observational Studies: They can show association, but difficult to show causation. Since the treatment and control groups just "happened" they are often very different from each other. These differences confound (mix up) the results. Possible confounders make it difficult to prove causation by association. Did the treatment cause the response or is the treatment simply associated with the response?

4. Page 11 Maybe both treatment and response were caused by a third confounding factor? TREATMENT IS JUST ASSOCIATED WITH RESPONSE. A CONFOUNDING FACTOR IS CAUSING BOTH THE TREATMENT AND RESPONSE TREATMENT TRULY CAUSESRESPONSE. Treatment Treatment Response Confounder Response Statisticians adjust for these confounding variables by breaking the control and treatment groups into smaller more homogeneous sub-groups, where the confounding factor is the same.

5. Page 12 Examples of Observational Studies: The following studies show association. Do they also show causation? What are the likely confounders and how would you adjust for them? Let’s start with an extreme example that will help you remember what a confounder is: Lighters and Lung Cancer--Suppose observational studies showed that people who carry lighters have a higher rate of lung cancer. There are 2 possibilities: 1. Lighters cause cancer. If so, how? The arrow in the diagram on the left represents a possible causal (causal, as in cause, not casual!) link that could connect lighters to cancer to support the argument that lighters cause cancer. But remember in observational studies, this causal link is pure speculation, observational studies do NOT provide evidence for causation. We first have to eliminate the possibility of major confounders before investigating causation, which brings us to the second possibility… 2. Lighters don’t cause cancer. If not, then why would people who carry lighters get more lung cancer? The mystery box in the diagram on the right represents a confounder or lurking variable-- something else about people who carry lighters that is the true cause of lung cancer making it look like lighters are responsible when they’re not.

6. Page 12 Lighters Lung Cancer ? OR Lighters ? Lung Cancer

7. Page 12 Lighters ? Lung Cancer How to identify confounders? Remember this is an OBSERVATIONAL study, the treatment (lighter group) and controls (non-lighter group) are different in other ways that could tip the balance towards our response (lung cancer). What might they be?

8. Page 12 On the next slide are either possible confounders (that would fit inside the mystery box on the right) mixing up the results making it look like lighters are to blame when they’re not, possible causal links (that would fit inside the arrow) explaining how lighters may cause cancer, or neither (other causes of cancer but ones that don’t help explain why people with lighters are getting more cancer).

9. Page 12 Clicker Questions Genetics—Some people are more genetically prone to lung cancer than others. a) Confounder b) Causal Link c) Neither 2) Age—Lung Cancer rates rise with age. a) Confounder b) Causal Link c) Neither 3) Smoking Cigarettes—People who smoke cigarettes have a higher rate of lung cancer and are also more like to carry lighters. a) Confounder b) Causal Link c) Neither 4) Lighter fluid—Perhaps inhaling the fumes from lighters can cause lung damage leading to cancer. a) Confounder b) Causal Link c) Neither 5) Radon—Radon exposure raises one’s risk for lung cancer. a) Confounder b) Causal Link c) Neither

10. Page 13 Now how can we determine whether it’s the lighters or the confounders or (maybe some combination of both) that is the true cause of the lung cancer? In this case there’s no difference in cancer rates between those who carry lighters andthose who don’t within each group. Of course the heavy smoker group has the highest cancer rates but rates between those who carry lighters and those who don’t are the same in that group. In other words, the lighters are just a marker for people who smoke—smoking causes cancer, not lighters. Once smoking was balanced between the 2 groups, the lung cancer rate was balanced too.

11. Page 13

12. Page 13 Examples of Observational Studies: The following studies show association. Do they also show causation? What are the likely confounders and how would you adjust for them? • Smoking and Liver Cancer --Studies show that smokers have a higher rate of liver cancer. Does that show that smoking causes liver cancer?

13. s To find Confounders list the differences between the treatment and control groups besides the treatment that could be relevant in causing the response. Treatment Treatment Response Response TREATMENT CONTROL

14. Page 13 Break the subjects down into subgroups where the confounding factor is the same in each subgroup. If the difference goes away, you’ve found a confounder

15. Page 14 • Oral Contraceptives and Cervical Cancer-- Some studies show that women on the pill have a higher incidence of cervical cancer. Does that mean the taking the pill could increase your risk for cervical cancer?

16. Page 14 • Pets and Allergy Protection- A study showed that infants living in homes that have two or more dogs or cats are less likely than other babies to develop allergies. Do you think this shows that early exposure to pets provides protections against allergies as the study concludes?

17. Page 14 • Sleep and Death- A study of 100,000 people (reported in the Feb 2003 journal of Sleep) found that people who reported sleeping eight hours a night had a higher mortality rate than those who reported sleeping seven hours. The longer the subjects slept, the higher their risk. Does this mean that people who sleep long hours should reduce their sleep to live longer?

18. Page 15 The following are observational studies where cause and effect may be reversed: • Endometriosis and Sexual Activity, Orgasms and Tampon Use-- A recent study found that women who have endometriosis (a condition that often causes extreme pelvic pain during menstruation) are less likely to engage in sexual behavior during menstruation. The study also found they are less likely to experience orgasms during menstruation and less likely to use only tampons. The study concludes that sexual activity, orgasms and tampons use may protect against endometriosis. Is there a more plausible explanation?

19. Page 15 • Smoking and Depression: A study found that people who smoke as teenagers are more likely to suffer depression as young adults. This study made national headlines as evidence that teenage smoking causes depression and led to research to investigate which chemical in tobacco is responsible. Is there another plausible explanation for the association between smoking and depression?

20. Page 15 • Artificial Sweeteners and Obesity- A study showed that people who drink lots of diet soda are more likely to be obese than people who don't drink diet soda and concluded that artificial sweeteners may set up a craving for sugar that leads to weight gain. What is a more plausible explanation for the association?

21. Page 16 What do you do with people who drop out of a study, include them or not? Example: The Clofibrate Trial —observational data within a randomized controlled experiment. A randomized, controlled double-blind experiment was done on middle-age men with heart disease to see if clofibrate, a drug that lowers cholesterol would prevent heart attacks. The patients were followed for 5 years. The death rates between the 2 groups were basically the same (20% and 21%) , implying the drug didn’t work. But not everyone took their medicine. If you look at just the subjects who faithfully tookthe clofibrate ("adherers"), the death rate is 15%, compared to 25% for non-adherers.

22. Page 16 Which comparison should be used: adherers vs. non-adherers (15% vs. 15%) everyone in the treatment group vs. everyone in control (20% vs. 21%) Adherers in the treatment group vs. everyone in the placebo group (15% vs. 21%) Suppose the above data came from a study where it’s impossible to have a placebo—like, for example, a study to see if exercise prevented heart attacks. Then we wouldn’t know who was an adherer or non-adherer in the Control group. All we’d see is the Total.

23. Page 16 Clicker Questions Should we compare the death rates of everyone assigned to exercise to the controls? Or should I just compare those who actually exercised to the controls? a) You should compare only those who actually exercised to the controls, since exercise can only help those who did it. b) You should compare everyone assigned to exercise to everyone assigned to control, otherwise the treatment group might consist of a different type of population (subjects who take better care of themselves in general) than the controls which could confound the results. c) You should compare those in the treatment group who actually exercised to those in the treatment group who didn’t since both groups were given equal encouragement and training in the special exercises.

24. Page 17 Simpson's Paradox — a clear-cut case of confounding that is easily adjusted for by dividing into subgroups Example 1: Sex Bias in Berkeley Graduate Admissions? 8,442 men and 4,321 women applied for admission to graduate school at UCB. 44% of the men and 35% of the women were admitted. Clicker Questions: Is this a: a) Controlled experiment b) Observational study? Assuming that men and women are equally qualified, is this evidence of sex bias? a) Yes b) No c) Maybe

25. Page 17 Break it down by major: There is no sex bias against women if you adjust for major. Women's overall admission rate is worse simply because most of the women applied to the harder majors (C-F), while most of the men applied to the easier majors (A and B).

26. Page 18 Example 2: Could smoking prolong life? A 20-year (1974-94) study of 1314 British women compared death rates of smokers to non-smokers and found that 23.9% (139/582) of the smokers had died compared to 31.4% (230/732) of the non-smokers. Could this be evidence that smoking helps you live longer? Clicker Question: a) YES b) NO c) Maybe

27. Page 18 Example 2: Could smoking prolong life? A 20-year (1974-94) study of 1314 British women compared death rates of smokers to non-smokers and found that 23.9% (139/582) of the smokers had died compared to 31.4% (230/732) of the non-smokers. Could this be evidence that smoking helps you live longer? Clicker Question: a) YES b) NO c) Maybe

28. Page 18 Control for age: Non-smokers had a high overall death rate because 26.4% (193/732) of them were old, while only 8.4% (49/582) of the smokers were old. If you control for age you get the opposite: Smoking shortens your life. Simpson's Paradox again: Overall percentage is misleading because of confounder. Oncethe confounder is controlled for by looking at sub-groups separately, the overall effect is reversed.

29. In the previous examples we first said it was best to compare Everyone to Everyone (in studies where people drop out) and then in Simpson’s Paradox we said that it was misleading to compare Everyone to Everyone.Isn’t this a contradiction?NO, because the first example is a randomized experiment and the next examples (Simpson’s paradox) are observational studies.*Inrandomized experiments, always compare everyone assigned to treatment to everyone assigned to control.*In observational studies, never compare everyone in treatment to everyone in control. Treatment and control groups are always different and we adjust for those differences by breaking into more homogeneous subgroups.

30. RANDOMIZED EXPERIMENTS OBSERVATIONAL STUDIES Treatment and control are assigned randomly (no systematic differences between the two groups). So it’s always best to compare the responses of everyone assigned to treatment to everyone assigned to control. Do not break into subgroups such as adherers and non-adherers since they choose themselves (like an observational study). ***It’s best to stick to the original random assignment and look at the results of EVERYONE assigned to treatment compared to EVERYONE assigned to control. Treatment and control groups choose themselves (they are different in MANY ways). So we have to break them into subgroups to make them more similar. So we CANNOT compare everyone in treatment to everyone in control. There will always be confounders. Simpson’s Paradox is a clear case of such extreme confounding that the overall results are so misleading as to be the opposite of the truth.

31. Page 19 Summary: • Observational studies are not as reliable as controlled experiments. • In Observational studies the control and treatment groups are different since the subjects (and not the researcher) chose the groups. • Observational studies are likely to have confounders—hidden differences between treatment and control groups that cause differences in their responses. • Important to remove confounding variable by comparing sub-groups for which the confounding variable is the same. • Simpson’s Paradox—An Observational Study which such extreme confounding that once the confounder is removed by breaking into sub-groups, the result is reversed. • What do you do with people who drop out of a Randomized Experiment (called non-adherers)? You have to include them and compare EVERYONE randomly assigned to treatment to EVERYONE randomly assigned to control. Otherwise you end up changing an ideal randomized study into a bad observational one. • Difficult to show causation by association in observational studies. Cause and effect can even be reversed.