The Scientific Method

1. The Scientific Method

2. Common Mistakes in Applying the Scientific Method • The scientist prefers one outcome over another (bias) • Ignoring or ruling out data that do not support the hypothesis; tendency to find something “wrong” with evidence that does not support the hypothesis (not treating all data in the same way) • Failure to estimate quantitatively systematic errors • Faulty interpretation of statistical data

3. Bad conclusions can be drawn when there is… • Selection bias: a distortion of evidence or data that arises from the way the data are collected • Reversed burden of proof: burden of proof should rest on those making a claim, not on the critic • Assertion that claims which have not been proven false must be true (and vice versa)

4. Correlation vs Causation • Correlation implies causation is a logical fallacy by which two events that occur together are claimed to be cause and effect. • It is dangerous to: • ignoring the possibility that the correlation is coincidence or that both correlated events have another common cause • draw conclusions about causation from statistical correlations. If you only have A and B, a correlation between them does not let you infer A causes B, or vice versa.

5. The Simpsons(Season 7, “Much Apu about Nothing”) • Homer: Not a bear in sight. The “Bear Patrol” must be working like a charm! • Lisa: That’s specious reasoning, Dad. • Homer: Thank you, dear. • Lisa: By your logic I could claim that this rock keeps tigers away. • Homer: Oh, how does it work? • Lisa: It doesn’t work. • Homer: Uh-huh. • Lisa: It’s just a stupid rock. But I don’t see any tigers around, do you? • Homer: Lisa, I want to buy your rock.

7. Case Study: Willie Soon • Associated with the George C. Marshall and Fraser Institutes • Has received funding from the American Petroleum Institute (this is acknowledged in his main paper) • While it doesn’t prevent him from doing good science, this calls into question his ability to be unbiased

8. Claim: “[T]he 20th century is not unusually warm or extreme.” • This was not based on a quantitative analysis. • The authors considered anomalous conditions to be warm, cold, wet or dry relative the 20th century. • The relationship between wet/cold and warm/dry is not 1:1. • Authors do not properly distinguish between local and global temperature changes • No attempt was made to estimate the errors in using proxy data • No way to resolve short time scales

9. Claim: CO2 has nothing to do with global climate change • The authors can only responsibly claim one point in their paper • They have a model with two free parameters • They can fit these parameters to global temperature change data • In their model, their best fit includes a higher contribution from the sun than green house gases • Just because the data fits a model does not imply the model is true. (One would have to test all possible models in order to prove that any one model was true.) • A convincing argument would entail performing a simulation that produced output that matched the data without fitting *Acknowledged funding sources in paper: Electric Power Research Institute, Mobil Foundation, Texaco Foundation and American Petroleum Institute

10. Some useful references about Correlation and Causation… • http://www.onpedia.com/encyclopedia/Correlation-implies-causation-(logical-fallacy) • http://www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml • http://teacher.pas.rochester.edu/phy_labs/appendixe/appendixe.html • http://en.wikipedia.org/wiki/Pseudoscience • http://www.venganza.org/ Willie Soon Papers and Rebuttal • http://adsabs.harvard.edu/full/1996ApJ...472..891S • http://www.cfa.harvard.edu/~wsoon/1000yrclimatehistory-d/Jan30-ClimateResearchpaper.pdf • http://www.atmos.washington.edu/~davidc/ATMS211/articles_optional/Mann_on_Soon2003.pdf