Nullius in Verba The Royal Society in London is the oldest scientific society in the world. It’s motto is “Nullius in Verba,” which translates to: “Take nobody’s word for it” or “Trust no one.”
Authority That’s impossible: we all need to trust other people for knowledge about lots of things. I’m not a climate scientist, I don’t have stations that collect and analyze data about global temperatures, so I can’t determine myself whether global warming is happening. I have to trust an authority to tell me if it is.
Appeal to Authority Sometimes you’ll hear about the “fallacy of appealing to authority.” But basing your views off on appeals to authority is not always fallacious. Your authority needs to be: • An expert, or better, an agreeing group of experts, on the subject in question. • A proven truth-teller on the subject.
Conflicting Reports However, sometimes there are conflicting reports between two seeming experts or two groups of seeming experts. How do we reconcile such conflict? How do we know what to believe?
Appeal to Motive It’s tempting to argue that one side is wrong because it has a motive for its position. Sometimes climate change deniers argue that scientists are lying about global warming because that brings in research dollars. Sometimes people who are pro-evolution argue that creationists are lying about the facts because they want to indoctrinate people into their religion.
Appeal to Motive It may be true that people have motives to lie and that they are acting on those motives and not the evidence. But we need to evaluate the arguments themselves. It is not enough to point out that someone has a motive to lie. That doesn’t mean what they are saying is actually untrue.
Credentials One way society has of distinguishing genuine authorities from other people is via credentials. A person’s authority can be certified by their having certain degrees, like a PhD or MD (doctor of medicine). A person’s ideas can have their authority certified by being published in a top peer-reviewed scientific journal.
Academic Degrees Of course, having a PhD doesn’t make you an expert on everything, only on what your PhD is in. For example, global warming skeptics have made a big deal about the Oregon Petition, a petition to have the U.S. government not base its policies on the supposition that global warming is happening.
The Oregon Petition Supporters of the petition point out that it is signed by over 31,000 people. However, only about 9,000 of them have PhD’s. Furthermore, only about 1,400 of the PhD holders have PhD’s in climate science and related fields.
The Oregon Petition The journal Scientific American polled the 1,400 who worked in climate science and found that many of them hadn’t ever heard of the petition, or said that though they had signed it, they would not sign a similar statement today. Scientific American estimated that the petition represented only about 200 genuine authorities.
Worthless Credentials Some “PhDs” aren’t real degrees awarded by serious institutions. Real universities are “accredited” meaning that an independent body has judged that they meet widely held academic standards. Many quacks and crazies try to acquire false authority by obtaining worthless credentials.
“Dr.” Gillian McKeith Gillian McKeith is a popular nutritionist in the UK. For a while she had a TV show where she represented herself as “Dr. McKeith,” and she also did this in books and on her website. This was on the basis of her degree from the non-accredited correspondence college Clayton College of Natural Health.
Suspicious Claims Ben Goldacre, a real medical doctor and science writer suspected that McKeith’s credentials were fake, because her work contained rudimentary scientific errors, like the claim that chlorophyll “oxegenates your blood.” Goldacre points out that chlorophyll doesn’t contain oxygen, it makes it in sunlight, but there’s no sunlight in your stomach. Even if it could make oxygen in your stomach, you can’t absorb oxygen there (like in your lungs): you’d just fart.
Fake Credentials It turns out Clayton College is not a real academic institution. It’s non-accredited, and it sells its degrees: USD$6800 for a PhD and a Master’s, $12,100 for two PhD’s and a Master’s. It’s a correspondence college, meaning you never go to any classes. And although it has you write a PhD thesis, it refuses to make them available (this is very unusual– you can find my thesis online at the Rutgers University website).
“Dr.” No More To make matters worse, McKeith trumpeted her membership in the American Association of Nutritional Consultants. To prove that this was not a real credential, Goldacre signed his dead cat up for membership for $60. After receiving complaints, the UK’s Advertising Standards Authority required her to stop using the title “Dr.” because it was misleading.
Degree Mills A “degree mill” is a university that is not accredited by official accrediting organizations. Many creationists have “doctorates” from degree mills. For example, prominent creationist Kent Hovind has a PhD from Patriot University, a degree mill. Patriot University is accredited by an unofficial accreditation mill, which accredits any university for $100.
Peer Review Peer review is the accepted method of research approval among scientists and other academics. If you write a scientific or academic article, typically you send it to a “peer reviewed journal.” Your submission is “blinded” so that no one can tell who wrote the article. The editor of the journal then sends the article to “peer reviewers”: other people who have expertise in the subject.
Peer Review The reviewers read the article and write up criticism and a publication judgment (“yes it should be published,” or “no there are too many errors, not high enough quality,” etc.). The editor then makes a judgment to publish or not based on the reviews. Peer reviewed articles in leading journals are the “gold standard” of academic achievement.
Climate Change Consensus Lots of work has been published arguing that global warming is not happening. But what about the peer-reviewed literature? A study by Oreskes (2004) of the previous 10 years (1993 to 2003) showed that of the 928 peer-reviewed studies that used the phrase “climate change” in the abstract, 0% of them said that global warming was not happening.
Oreskes 2004 75% of the papers either explicitly endorsed the consensus view that global warming was happening or implicitly endorsed it, by for example, making recommendations for how to slow down or stop warming trends. The other 25% made no commitment, often because they were about climate change long ago, now climate change today. 0% disagreed with global warming.
Non Peer-Reviewed Literature The public often doesn’t understand the distinction between peer-reviewed scientific research and other types of publications. Climate skeptics use such misunderstandings to mislead and manipulate the public. There is scientific literature that is against global warming– but this literature is not peer-reviewed: it consists of editorials, letters, or reviews with no original research in them.
The Sokal Affair In 1996, a physicist at NYU named Alan Sokal submitted an article to the journal “Social Text,” a non-peer reviewed journal about postmodern cultural studies. The article, entitled “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity” was full of nonsense and silly mathematics jokes.
Nonsense “[A]s Bohr noted, ‘a complete elucidation of one and the same object may require diverse points of view which defy a unique description’ -- this is quite simply a fact about the world, much as the self-proclaimed empiricists of modernist science might prefer to deny it. In such a situation, how can a self-perpetuating secular priesthood of credentialed ‘scientists’ purport to maintain a monopoly on the production of scientific knowledge?”
Silly Math Jokes “Just as liberal feminists are frequently content with a minimal agenda of legal and social equality for women and ‘pro-choice’, so liberal (and even some socialist) mathematicians are often content to work within the hegemonic Zermelo–Fraenkel framework (which, reflecting its nineteenth-century liberal origins, already incorporates the axiom of equality) supplemented only by the axiom of choice.”
Lax Standards After the journal published the article, Sokal revealed that it was a hoax, intended to prove the lax standards in “postmodern studies.” If the article had been appropriately peer reviewed by experts who actually knew about the subjects (“quantum gravity”) they would have immediately seen that it was phony.
The Limits of Peer Review Peer review isn’t perfect, however (of course, nothing is perfect). Academic journals can be poorly peer reviewed: the review process might not be “blind,” it might not involve real experts, and it might be “too easy”– some journals make money off of fees that authors pay to publish in them. These journals have an incentive to publish anything, because everything they publish makes them money.
http://pdos.csail.mit.edu/scigen/ For example, students at MIT created a computer program that randomly generates nonsense Computer Science papers. Various people have used the program to submit nonsense papers to conferences and non-peer-reviewed journals. One group of students actually had a nonsense paper accepted at a peer reviewed conference in Wuhan, China.
Impact Factor One way of distinguishing good peer-reviewed academic journals from lousy ones is by a measure called the “impact factor.” Journals that publish papers that are cited by further scientific or academic research have higher impact factors. Impact factor is thus a measure of how much influence a journal has.
Sometimes the people promoting crazy views, instead of trying to fake credentials, trick people with real credentials into appearing to endorse their views.
Ramtha’s School of Enlightenment JZ Knight is a charlatan who claims to “channel” (i.e. speak for) Ramtha, a 35,000 year old spiritual entity from the fictional land of Lemuria who once conquered the fictional land of Atlantis. She runs Ramtha’s School of Enlightenment, where rich stupid people pay thousands of dollars to hear new age spiritual nonsense married with quantum physics mumbo jumbo.
https://www.youtube.com/watch?v=3QlZ5O8_bGk Some of “Ramtha’s” views are outlined in the 2004 film “What the Bleep Do We Know?” a combination of interviews and a fictional story about a deaf woman using the power of positive thinking to influence the events in her life. Several legitimate scholars show up in the film, including one of my former teachers, David Albert.
Creative Editing Albert says, “I was edited in such a way as to completely suppress my actual views about the matters the movie discusses. I am, indeed, profoundly unsympathetic to attempts at linking quantum mechanics with consciousness. Moreover, I explained all that, at great length, on camera, to the producers of the film… Had I known that I would have been so radically misrepresented in the movie, I would certainly not have agreed to be filmed.”
Creative Editing This is a common trick: you tell people you are working on a film about X, when really you are working on a different film about Y. You interview them for many hours, and then you edit the interview so it seems to be agreeing with you. Now you have expert testimony about Y. (Second example: the documentary “Expelled!”)
Expelled from ‘Expelled!’ “Expelled!” had many prominent scientists like Richard Dawkins in it. Those scientists were told that they were being interviewed for a pro-Darwin movie called “Crossroads,” when the movie was actually an anti-Darwin movie arguing that the scientific establishment “expelled” dissent, and didn’t allow criticism. At the movie’s screening, the producers actually threw out one of the scientists in the movie, because they knew he was critical of it!
News as an Information Source Unfortunately, most people don’t get their beliefs from the genuine consensus of real scientific authorities with real credentials. People get their beliefs from newspapers, which at their best are horribly misleading, and full of falsehoods and propaganda.
Editorials vs. Reporting Newspapers contain opinion pieces called editorials. These are written by the editorial staff of the newspaper, and are not reports about scientific findings or consensus, they are opinions and (often very bad) arguments of (very often) know-nothing journalism graduates. So if you read The Washington Times or The Australian, you’re likely to be told that global warming is a hoax and is not happening.
Infotainment Most people don’t make sharp distinctions between what’s on the “Opinion” page of the newspaper and what’s in the rest. Sometimes news sources themselves stop making the distinction. “Infotainment” is a term that covers this fusion of information and entertainment.
Initial Studies Scientific studies cost money, so often scientists conduct “initial” or “pilot” studies. These are studies that are too small to prove any real effect (reject the null hypothesis, that there is no effect). Still, if a correlational claim is confirmed in a small study, that might suggest it’s worth pursuing in a good, big study.
Most Research Findings are False Most effects, however, will turn out not to be real in subsequent research (there are lots more correlations that are NOT there than there are correlations that ARE there). That’s why we shouldn’t trust just any old research: we want systematic meta-analyses of large, statistically significant double-blind randomized controlled trials.
Ioannidis (2005) “The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field…Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.”
Reader$ However, newspapers are most likely to publish interesting positive results, and they don’t tend to care if those results are just in a pilot study and will probably later be shown to be wrong. Newspapers care about attracting readers, and if they can say truly, “New scientific research finds correlation between video games and eyeball cancer,” they will, even if it’s misleading.
Hacks and Flacks Newspapers don’t make a lot of money anymore (there’s free news on the internet). So they employ fewer reporters who are expected to generate more stories. This has led to a symbiosis between many journalists and public relations firms. The PR firm will write a “news” story that the newspaper will simply reprint.
Churnalism This is known as “churnalism”: According to Wikipedia (I couldn’t find the original source) “Justin Lewis at Cardiff University and a team of researchersfound that 80% of the stories in Britain's quality press were not original and that only 12% of stories were generated by reporters.” A lot of these “news stories” are intended to make you buy products or accept propaganda.
Free Advertising http://www.dailymail.co.uk/news/article-2232056/Gardeners-florists-named-UKs-happiest-workers-90-horticulturalists-enjoy-going-work.html Here’s an example: the entire article is mostly an advertisement for the vocational education company City & Guilds.
Propaganda Frequently as well, anonymous government sources will intentionally “leak” classified information to the press, so that on the basis of no evidence (besides the “leak”) newspapers will report the government’s side of a story. Other citizens and journalists cannot check and make sure the story is true, because there is no evidence, only the “leaked” report, from an unidentified source.
Journalistic Embargoes Even scientists are not above manipulating newspapers for undeserved fame and sometimes even money. There is a practice whereby a scientist(s) will release a forthcoming article to the press, but not allow them to talk about it with anyone before it is published, and there is a press conference. This is called an “embargo.”