The Limitations of Personal Experience

1. The Limitations of Personal Experience • As we discussed in the last class, the way people tend to gather information tends not to be very systematic • we tend to seek information that is consistent with one’s beliefs and ignore inconsistent information • we tend to base our inferences on little information • our expectations tend to influence what we see • we’re not attentive to base rates and comparisons • we tend to assume that propositions that feel wrong to us are invalid

2. Why are these “limitations?” • In conjunction, these factors can lead to two problems for developing an accurate understanding of the world. • First, these factors can lead us to the wrong answer. • Second, the process itself is not self-correcting.

3. 1. The Wrong Answer • As we demonstrated in the “Josh” exercise, people who were seeking to answer different questions recalled different kinds of information about Josh. • Although Josh was neither more extraverted than introverted (i.e., there were two pieces of data pertinent to each interpretation), people came to different conclusions about Josh based on the question they sought to answer.

4. 2. Why the Process is not Self-correcting • The confirmation bias helps to ensure that, once an expectation or theory has been developed, the belief will be self-perpetuating. Is Josh extraverted? We conclude that Josh is extraverted We then recall extraverted-consistent information better, and fail to notice introverted information

5. So, what we want • are methods that are more likely to lead us to the right answer • a process for understanding the world that will enable us to correct the inevitable mistakes that we will make

6. The Scientific Method • The scientific method is a way of dealing with these concerns. • Science is theprocess of constructing, testing, and refining theories about natural phenomena though the use of systematic empirical observation.

7. Systematic • By systematic, I mean that that all information counts, regardless of whether it is consistent or inconsistent with one’s assumptions or how it makes us feel. • Moreover, by systematic I mean attending to base rates, collecting a sufficient amount of information, recognizing and correcting for potential biases. • In short, trying to be as true as possible to what really happens in the world.

8. How is the process self-correcting? theory about how something works generate predictions (what would the theory lead you to observe?) testing: comparing the observations with the theory systematic empirical observations

9. How is the process self-correcting? theory about how something works generating predictions (what would the theory lead you to observe?) testing: comparing the observations with the theory ensures that inconsistent observations will be recorded systematic empirical observations

10. How is the process self-correcting? theory about how something works generating predictions (what would the theory lead you to observe?) testing: comparing the observations with the theory ensures that inconsistent observations will be counted against the theory systematic empirical observations

11. How is the process self-correcting? theory about how something works generating predictions (what would the theory lead you to observe?) testing: comparing the observations with the theory theory is revised in light of the tests, and, hopefully, becomes more accurate systematic empirical observations

12. What is the scientific process all about? • Let me say a few more words about the term “theories.” • In this class, I will be using the term theory pretty loosely to refer to an explanation for a phenomenon; a conceptual model of how something works.

13. Good and bad theories • What is the difference between a good theory and a bad one? We all have theories for things, so what gives a theory scientific credibility?

14. How do we decide whether a theory is good? • Historians of science have argued that “good theories” tend to have the following qualities: (1) They are generative (2) They make precise (i.e., risky) predictions (3) They can be unambiguously tested (falsifiable) (4) They are simple (parsimonious) (5) They have Good Track Records (previous predictions have been tested and supported by systematic observation)

15. Cautions about “science”: Process vs. subject matter • It is important to keep in mind that science is a process for understanding the world. • It is not a topic or subject matter. Some people believe that chemistry, for example, is inherently scientific. However, one can study chemical phenomena in scientific ways or in non-scientific ways (as exemplified by some of the more esoteric branches of alchemy).

16. Cautions about “science”: Pseudo-science • Just because something is called a “science” doesn’t mean that it is scientific. creation science astrological science • What is astrology missing?

17. Cautions about “science”: Pseudo-science • In large part, scientific inquiry is accompanied by what might be called a “scientific attitude” • an excitement about discovery • a willingness to bend over backwards to prove oneself wrong • a healthy mixture of skepticism and open-mindedness • A good litmus test for whether someone is thinking scientifically: What does the person do with information that is inconsistent with his or her expectations? (Binny Hinn)

18. Cautions about “science”: Pseudo-science • Some signs that the “science” may not be good science • Failures are rationalized or explained way • Reliance on anecdotes • Lack of tests • Lack of supporting evidence

19. Cautions about “science”: Science as a Process versus Science as an Ideal • Obviously, people who are using the scientific method suffer from many of the problems we’ve described • The scientific process is an ideal; one that we strive to obtain, but do not necessarily achieve • The onus is on you to be able to separate good science from pseudo-science. (In taking this course, I hope you’ll learn how to make these distinctions.) • We’ll discuss pseudo-science on Monday.