Download
1 / 57
570 likes | 694 Vues
Hyperskepticism. The dunning- kruger effect. 孔子. “Real knowledge is to know the extent of one's ignorance.”. Charles Darwin. “Ignorance more frequently begets confidence than does knowledge”. Expertise and Evaluation. To be good at a task requires certain skills and knowledge.
E N D
孔子 “Real knowledge is to know the extent of one's ignorance.”
Charles Darwin “Ignorance more frequently begets confidence than does knowledge”
Expertise and Evaluation To be good at a task requires certain skills and knowledge. Often, to know whether one is good or bad at that task requires the same skills and knowledge one needs to be good at the task.
English as an Example To write good, grammatical English, you need language skills and knowledge acquired over years of training. But to know if you’re writing good or bad English, you need the same skills! You can’t know that you’re wrong or how to correct yourself unless you have the skills + knowledge.
Physics One finding showed that people who were new to physics were less good at deciding how hard a physics problem was, compared to experts.
Dunning-Kruger Effect The Dunning-Kruger effect is when people: • Perform badly because they lack the skills and knowledge for the task. • Think they are performing well because they lack the same skills and knowledge to evaluate themselves.
Study #1 The experimenters (Dunning and Kruger) asked some professional comedians to rate how funny certain jokes were. The comedians did their ratings, and the experimenters found that they were highly correlated: what one professional said was funny, the other professionals said was funny.
Example: Bad Joke Question: What is big as a man, but weighs nothing? Answer: His shadow. Expert Rating: 1.3 on 1-11 scale.
Example: Good Joke “If a kid asks where rain comes from, I think a cute thing to tell him is ‘God is crying’… And if he asks why God is crying, another cute thing to tell him is probably because of something you did.” Expert Rating: 9.6 on 1-11 scale.
Then the researchers asked undergraduate psychology students both: • To rate the jokes and • To decide how well their answers agreed with the experts.
Regression to the Mean Remember that when we have two correlated variables on average extreme values of one variable will be paired with extreme values of the other.
Correlated Variables A = how good you are at knowing what’s funny. B = how good you are at knowing how good you are at knowing what’s funny.
Regression to the Mean Therefore, we expect a regression effect: the people who most disagree with the experts will (low A) will have a more average opinion of themselves (less low B).
Two Problems • Humor is subjective. Even the experts disagreed (r = .76). • It’s possible that the unfunny students weren’t overestimating their own abilities, but underestimating others’ abilities: • They thought that though they were bad, others were really bad. (Instead of thinking that others were good, but they were really good.)
Logic! Buses 1, 2, and 3 make one trip each day, and they are the only ones that riders A, B, C, D, E, F, and G take to work. • Neither E nor G takes bus 1 on a day when B does. • G does not take bus 2 on a day when D does.When A and F take the same bus, it is always bus 3. • C always takes bus 3.
Logic! Traveling together to work, B, C, and G could take which of the same buses on a given day? (A) 1 only(B) 2 only(C) 3 only(D) 2 and 3 only(E) 1, 2, and 3
Logic! Traveling together to work, B, C, and G could take which of the same buses on a given day? (A) 1 only(B) 2 only(C) 3 only(D) 2 and 3 only(E) 1, 2, and 3
No Problems Logic is not subjective. Here we can see that the worst students thought they scored better than average AND thought they were better than average.
Learning from Others In the final experiment, the researchers invited the worst students and the best students in each task to “grade” the other students’ tests. Prediction: the worst students cannot learn they are bad by looking at others’ work, because they cannot tell which answers are right or wrong.
Real Life Consequences People like Deepak Chopra use scientifically flawed theories to sell products to unsuspecting consumers. Chopra is not knowledgeable enough about quantum physics to see that he’s wrong, or to recognize the flaws others point out.
Sample Size Students in this class and previous classes have critiqued scientific studies by claiming “there are too few participants” when the participants are in the thousands. This is a case where students are wrong, they cannot see that they are wrong, and this prevents them from learning what’s right.
Ioannidis “Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.”
Important Points Most studies are not large-scale RCTs or even better large-scale meta analyses of lots and lots of RCTs. So most studies are likely to be false. But that doesn’t mean that science can’t find the truth: we can always conduct more research and do meta analyses.
Important Points Some things we know are true. Science has conclusively proven them. Smoking causes lung cancer. We did not always know that.
Important Points Prior probability matters. Most (positive) studies on homeopathy are probably false, because it’s highly unlikely magic water cures people. But a new study showing smoking was linked to some other cancer? Probably not false.
Important Points We often have lots of reasons to suspect a treatment will work– cell studies or animal studies or pilot studies. Strong prior probability decreases the chances of false positives.
(I’ll admit, I’m doubtful about acupuncture and acne, because of its low prior probability.)
Important Points As we saw, science relies on replication. When we fail to replicate a finding, we get rid of it. So just because most scientific findings are false does not mean that most scientific beliefs are false. We have already gotten rid of the bad beliefs by trying to replicate the findings and failing!
Conclusion If you want to know what’s true, ask a scientist. If you think this class taught you to ignore science, you are a victim of the Dunning-Kruger effect.
“Inverse” Base Rate Neglect Base rate neglect is when tests become unreliable because a condition is very rare. But the opposite can happen too. When a condition is very common, tests that show you don’t have that condition are unreliable.
Lawyer vs. Skeptic A lawyer’s job is to prove that her client is right. Any argument that supports the view is good. Any doubt that can be cast on the opposing view is good. Truth is not the goal: winning is.
Lawyer vs. Skeptic It’s different for critical thinkers. We are supposed to accept what’s true and reject what’s false– even if it’s what we believe that’s false.
Hyperskepticism Hyperskepticism, as I define it, is being a lawyer when you need to be a critical thinker. It’s using all your intelligence and ability to show that you are right instead of using it to find out the truth.
Cognitive Dissonance Sometimes how we behave and how we believe are different in a way that conflicts. I smoke, but I believe that smoking is terrible.
Cognitive Dissonance “Cognitive dissonance” is the bad feeling you get when what is happening conflicts with your beliefs about what should be happening. There are many ways people can resolve their cognitive dissonance. I could stop smoking. Or I could stop believing that smoking is bad.
Motivated Reasoning People whose beliefs or attitudes are a strong part of their personality/ identity may be tempted to irrationally change their beliefs instead of their behaviors. (Alt. medicine, climate change, religion, evolution…)
Barack Obama In 2008, America elected its first black president: Barack Obama. Many Americans think that blacks are lazy, stupid, and inferior. They have treated blacks poorly in the past and have voted in ways that harm blacks.
Barack Obama This is where cognitive dissonance comes in. People decided that Obama was not legitimate. He was born in Kenya! He was anti-American! He wanted to hurt, and not help, Americans.
Foreign Domestic Helpers Amnesty international found that “they worked on average 17 hours a day, half were denied their statutory weekly day off and two-thirds had been physically or verbally abused.” [SCMP]
