Download
1 / 77
790 likes | 1.06k Vues
Experimental Design. Hw 3. RECap : scientific method. The Water Cycle. Earthquakes & Tsunamis. Earth’s Seasons. Science. We’ve learned a lot of bad ways (fallacies) for figuring out whether claims are true. There is a good way of finding things out: science!
E N D
Science • We’ve learned a lot of bad ways (fallacies) for figuring out whether claims are true. • There is a good way of finding things out: science! • Science tries to discover the causal structure of the world, so it can predict, explain, and control nature for the benefit of humankind.
Science • We cannot directly observe the causal structure of the world, we can only observe correlations, and theorize about them. • The goal of science is to test our theories about the causal structure of the world. We try to show that they are false. • If we try very, very hard to show they are false, and we keep failing, then we can accept them as true.
Chocolate and Nobel Prizes In the article, “Eat more chocolate, win more Nobels?” Dr. Franz Messerli claims to have found "a surprisingly powerful correlation” between the chocolate consumption in a country and the Nobel rate.
Chocolate and Flavanols The theory outlined in the article is that chocolate contains flavanols; flavanols slow down age-related mental decline (though this is doubtful); and…? Well, it’s not really explained how lessened mental decline makes you more likely to win Nobels. Wouldn’t chocolate have to make you smarter and not just prevent you from being dumber?
B causes A Dr. Messerli, according to the article, admits that “it’s possible… that chocolate isn't making people smart, but that smart people who are more likely to win Nobels are aware of chocolate's benefits and therefore more likely to consume it.”
C causes A and B The article also quotes Sven Lidin, the chairman of the Nobel chemistry prize committee: “I don't think there is any direct cause and effect. The first thing I'd want to know is how chocolate consumption correlates to gross domestic product.” He seems to be suggesting that GDP causes higher chocolate consumption and more Nobel prizes.
The GDP Theory Here’s what I think Lidin is suggesting: Chocolate is a luxury. Wealthy individuals are more likely to be able to afford it. Education is also a luxury. Poor people can’t afford to go to college for 10 years to get a PhD in chemistry. But you can’t win the Nobel prize in chemistry unless you’re a chemist. So he expects that the GDP or “wealth” of a country will be correlated both with chocolate eating and with Nobel prizes.
The GDP Theory So he expects that the GDP or “wealth” of a country will be correlated both with chocolate eating and with Nobel prizes. Wealth causes chocolate eating & Nobels.
Spurious Correlation It’s also possible that Dr. Messerli has committed the ecological fallacy, assuming that a correlation between a country’s chocolate consumption and that country’s number of Nobel prizes means that there is a correlation between individual chocolate consumption and individual Nobel-prize winning.
Ecological Fallacy Explanation Maybe smart people tend to avoid chocolate, because they know it can cause obesity. When they live in a country that consumes lots of chocolate they have to exercise their will power frequently. And maybe smart people + strong willpower = more Nobels. So it’s not eating chocolate but avoiding chocolate that causes Nobel prizes.
Types of Scientific Studies There are two basic types of scientific studies (the stuff that gets published in scientific journals and reported in the “science” section of the newspaper): • Observational studies • Controlled experiments
Observational Studies An observational study looks at data in order to determine whether two variables are correlated.
Case Study In science, we want to know about the effects of something (exposure to radiation, living through a certain political crisis…) or the causes of something (a disease, having certain beliefs…). A case study finds people who have the condition we want to know about (they were exposed to radiation, or they have the disease) and looks back at their histories.
Example Suppose I want to know why people gamble. I might find a group of gamblers and give them all a survey: When did you first have sex? Do you smoke? Did your parents divorce? When you win money, how do you spend it? Do you eat meat?
Problems with Case Studies Suppose I find that 27% of the gamblers I survey have divorced parents. Does that mean divorce is significant cause of gambling? No. We need to know if this is more or less than the divorce rate among non-gamblers. (In fact, it’s about the same: HK divorce rate is 20%-30%.)
Case Control Study In a case control study we find not just a group of people we’re interested in (gamblers) but also a group of people we’re not interested in (the control group, non-gamblers). The goal is to compare the histories of one group to the histories of the other group.
Problems with Case Control Studies Correlation is not causation! What if I discover that more gamblers smoke than non-gamblers. I still don’t know: • Maybe smoking causes gambling. • Maybe gambling causes smoking. • Maybe poverty causes gambling & smoking. • Maybe it’s just a coincidence.
Why Do We Do Them? Case control studies can be done very easily, very fast, and with very little expense. Scientists will use them to suggest things to study more seriously, or to rule out certain hypotheses. After all, if gamblers smoke less than non-gamblers, smoking probably does not cause gambling!
Cohort Studies Cohort studies are more reliable than case control studies. In a cohort study, you follow two groups over time. One group, the cohort, has a certain condition (for example, smokes) and the other group doesn’t. Then you see what happens and compare the results.
Cohort Studies For example, an cohort study might ask women to record how much wine they drink, and also to report if they develop breast cancer. After many years, a correlation may be found between wine consumption and cancer.
Advantages over Case Control • Avoids recall bias. • Lets us study changes over time. • Useful for studying rare conditions. • Lets us investigate many effects. • Allows us to calculate the relative risk (the amount that a condition increases or decreases your risk of something.)
Problems: Confounding Variables Suppose my cohort is a group of smokers. Smokers tend to have more in common with one another than just smoking: • The poor smoke more than the rich. • The uneducated smoke more than the educated. • People who drink alcohol smoke more than people who do not.
Problems: Confounding Variables Anything that we discover in the cohort that is correlated with smoking will also be correlated with all the confounding variables! So if smokers get more cancer, is it because they smoke, or because they don’t have money to go to a hospital for checkups?
Observational Studies Importantly, observational studies can only show whether two variables A and B are correlated. They cannot show whether A causes B, or B causes A, or some third cause causes both, or if the correlation is accidental.
Controlled Experiments The first recorded controlled experiment occurs in the Book of Daniel, part of the Jewish Torah and the Christian Bible.
Daniel 1:1-16 In the book, Daniel is forced into the service of King Nebuchadnezzar of Babylon. He is fed the king’s meat and wine, but he refuses – the Jews have special laws about how things like meat and wine are prepared.
Daniel’s Experiment “Please test your servants for ten days: Give us nothing but vegetables to eat and water to drink. Then compare our appearance with that of the young men who eat the royal food, and treat your servants in accordance with what you see.” Daniel 1:12-13
Daniel’s Experiment “At the end of the ten days they looked healthier and better nourished than any of the young men who ate the royal food. So the guard took away their choice food and the wine they were to drink and gave them vegetables instead.” Daniel 1:15-16
Controlled Experiments In a controlled experiment there are two groups who get separate treatments. One group, the “control group” gets the standard treatment. For example, all of the king’s servants ate meat and wine before Daniel suggested a different diet might be better.
Controlled Experiments The other group, the “experimental group”, gets the treatment we plan to test. If the test group has better results than the control group, we have good evidence that our new treatment should be adopted.
Controlled Experiments Suppose I believe that eating chocolate makes you smarter. Maybe I have some evidence, in the form of observational studies that show a correlation between chocolate consumption in a country and the number of Nobel prizes won by that country.
But there are alternative theories: • Smartness causes chocolate eating • Wealth causes smartness and chocolate eating • Chocolate avoiding causes smartness • Etc.
Experimental Design I can rule out these possibilities with a well-designed experiment. What I want is two groups: one group (the experimental group) that eats chocolate because I tell them to, and another group (the control group) that does not eat chocolate, because I tell them not to.
Not: B causes A If the experimental group improves in intelligence over the course of the experiment, I know that this is not because higher intelligence leads to more chocolate consumption (even if that is true). In my experiment, intelligence does not cause chocolate consumption, I do. I am the experimenter and I say who eats chocolate.
Controlling for Additionally, if I make sure to put equal numbers of rich people in both groups, and equal numbers of middle-class people, and equal numbers of poor people, then I can make sure that improvements in the experimental group are not due to wealth: both groups have the same distribution of wealthy and non-wealthy people. This is called controlling for wealth.
Randomization Ideally, an experiment controls for as many variables as possible. To a large extent, this is done by randomly assigning individuals in the study to either the control group or the experimental group. This way, the members of the group are less likely to share features other than chocolate eating.
Randomization Randomization is not the only tool for controlling for confounding variables, and for certain variables, it can’t help. For example, suppose I want to test whether seeing pictures of babies makes people happier.
