Statistics as Evidence

1. Statistics as Evidence Arguing by the numbers

2. impressing with statistics • Approximately one in five children ages 10-17 who use the Internet frequently have received a sexual solicitation or approach over the Internet. • The mortality rate for avian flu is more than 50 percent. • The average L.A. driver spends 136 hours per year in rush hour traffic • A typical child has seen 4,286 acts of violence on TV by age 18. • Bill Gates makes more money than the poorest 100 million Americans

3. But are the numbers really that impressive? • An article in the New England Journal of Medicine stated “a person who purchases a handgun is 57 times more likely to commit suicide within a week of buying the weapon than the general population as a whole.” • The study said suicide is the leading cause of death among gun buyers, in the first year after a weapon is purchased. • The statistic is misleading, however, because buying a handgun doesn’t make one more likely to commit suicide, wanting to commit suicide makes one more likely to buy a handgun. • People who are bent on committing suicide probably buy more knives, rope, sleeping pills, and razor blades, too.

4. watch out for “unknowable” statistics • example: The average person tells 13 lies per week • example: “in up to 98% of the cases, simply brandishing a gun is sufficient to stop a crime.” John Lott Jr., Wall street Journal July,30, 2001 • example: Judy Garland sang “over the Rainbow” 1,476 times.” Esquire, Feb. 1964

5. comparing non-comparable units (apples versus oranges) • A study conducted by the dean of admission for Harvard University concluded that Harvard students who took prep courses scored lower than students who didn’t take prep courses. The study concluded “the coaching industry is playing on parental anxiety.” • However, the comparison is based on non-equivalent samples of students. Students who didn’t even bother taking a prep course might be much stronger academically than students who opted to take such a course.

6. watch out for unrepresentative samples • Non-random samples: • voting on the American Idol show • phone calls or emails to a politician about a particular bill • testimonials on infomercials • talk radio callers • To be truly valid a sample must be: • random • representative • sufficient

7. questionable extrapolations • Using a limited sample (limited in time or number) and extrapolating the percentages nationwide or globally • example: According to the National Self Defense Survey in 1994, the rate of Defensive Gun Uses can be projected nationwide to approximately 2.5 million per year -- one Defensive Gun Use every 13 seconds. • http://www.pulpless.com/gunclock/framedex.html

8. keep the numbers in perspective • Compare “scary” statistics to other, well-established health risks. • “mad cow disease” or the “flesh eating virus” sound scary, but the odds are much greater of dying of dog bites, drowning in the tub, or being struck by lightning. • Chicken pox kills 100 people per year in America. • Flu kills about 35,000 people annually in the U.S.

9. School killings Total school-related violent deaths, August 1, 2005, July 31, 2006: Shooting: 15Suicides: 1  Murder-Suicide: 4 Fight-Related: 0 Stabbing: 3 Other: 4 Total = 27 Compared to drownings and car crashes In 2003, 782 children ages 0 to 14 years died from drowning (CDC 2005). In the United States during 2004, 1,638 children ages 14 years and younger died as occupants in motor vehicle crashes Keep the numbers in perspective

10. misleading statistics • Some studies have reported that 90% of students are bullied at school. • But the results depend on what the researcher defines as “bullying.” Does bullying included physical and verbal abuse? If bullying means “people being mean to you,” then it is surprising that 10% of students haven’t been bullied!

11. misleading graphs and charts • This CDC graph, comparing causes of death is misleading, because it leaves out deaths from heart disease, cancer, and stroke. The omission tends to make smoking look much more harmful by comparison.

12. not defining the units to be counted clearly • example: statistics on “juvenile violence” • the legal ages of juveniles vary from state to state, as do the ages at which juveniles may be tried as adults. • example: statistics on “child abductions” • can refer to kidnappings by strangers or parental custody disputes

13. carefully considering statistical measures • Failure to report the margin of error ( “sampling error”) • example: +/- 5% in a national poll of 1,200 respondents • Making conclusions within the margin of error • If candidate A is preferred by 43% of voters, and candidate B is preferred by 46% of voters, all one can say is that they are tied in a statistical dead heat. • Not stating whether the results are statistically significant or what the threshold for statistical significance is. • example: p < .05

14. media slanting: What to watch and listen for • Remember: news media are after ratings, not necessarily the truth. • Watch for bias during the opening or lead-in. • Ask how representative or generalizable case studies are. • Don’t let pictures overwhelm the facts. • Beware of quoted sources with an agenda. • Beware of “black hats” or “white hats.” • When statistics are presented is the underlying methodology explained? • Watch out for rhetorical questions with no follow-up proof.