Construct Validity Theory: The Philosophy of Science Replacing Positivism and Operationism

1. Construct Validity Theory:The Philosophy of Science Replacing Positivism and Operationism Lee Cronbach: 1916-2001 Paul Meehl: 1920-2003

2. Cronbach and Meehl Paul Meehl was elected president of the APA in 1962 Lee Cronbach invented the famous statistic called Cronbach’s alpha…he was ALSO a president of the APA (1957) These could be the most important pair of researchers in the history of psychology In 1955 they published the famous and incredibly influential paper… Construct Validity in Psychological Tests 2

3. Construct Validity Theory A philosophy of science Specifically, a method for justifying the claim that a test is a valid measure of a construct Is an IQ test a valid measure of intelligence? Cronbach and Meehl suggested that one way to determine this is by using the methods and logic of CV Theory 3

4. The Logic of Construct Validity Theory There are 4 types of validity: Predictive validity: A test has predictive validity as a measure of a CRITERION if the test is correlated with the criterion and the criterion is measured after the test. E.G: High school GPA has predictive validity as a measure of college GPA because they are correlated with each other and college GPA is measured some time after high school GPA. Concurrent validity: A test has concurrent validity as a measure of a CRITERTION if the test is correlated with the criterion and the criterion is measured at the same time the test. E.G: Diameter of a tree has predictive validity as a measure of height of a tree because they are correlated with each other and diameter of a tree is measured at the same time as height of the tree. Content validity: A test has content validity as a measure of a CRITERION if the test items are representative of the criterion. Quiz 2 has content validity as a measure of chapters 8-10 if the items on the quiz are taken from chapters 8-10 (and not say chapter 5). Construct validity: A test has construct validity as a measure of a CONSTRUCT if the test correlates with what it should be correlated with if it ere a measure of the construct. 4

5. The Logic of Construct Validity Theory Construct validity is relevant when there is no operational definition 5

6. The Logic of Construct Validity Theory Construct validity is not just a way to validate tests, it’s a way to validate constructs as well. We ask: Is test X a valid measure of construct Y. E.G: Are IQ tests valid measures of intelligence? What does construct Y denote: E.G: What is intelligence? 6

7. The Logic of Construct Validity Theory What is the difference between a construct and an operational definition? An operational definition is unique, clear, public, shared, etc. It is…precise A construct is not well defined…it is open to interpretation, perhaps vague, not formally defined, etc. 7

8. The Logic of Construct Validity Theory So…according to CV Theory, how do we determine what something is and/or what a test measures? There are a number of fundamental principles Each of the next slides states one principle 8

9. Fundamental Principles of CV Theory Scientifically speaking, to "make clear what something is" means to set forth the laws in which it occurs. We shall refer to the interlocking system of laws which constitute a theory as a nomological network. What something correlates with tells you what it is. Depression correlates with insomnia, therefore, depression is partly physical. 9

10. Fundamental Principles of CV Theory What some thing is, is given by a theory about what it is. There is a theory that depression is a chemical imbalance in the brain. 10

11. Fundamental Principles of CV Theory Initially, when there are few empirical findings relating to a construct (the nomological network is small), the construct is vague, or we say, underdetermined. We have not done much research on the nature of ADHD yet, so we do not know much about what ADHD is. 11

12. Fundamental Principles of CV Theory "Learning more about" a theoretical construct is a matter of elaborating the nomological network. This means that learning more about something is the same thing as learning what it is. Definition and description are the same thing. The more we know about the brains of people with depression, the more we learn about what depression is. 12

13. Fundamental Principles of CV Theory What a test measures is given by what the test correlates with. If the test correlates with what it should correlate (if it were a measure of what we claim it measures), we say the test as convergent validity as a measure of the construct we claim it measures. If IQ tests are measures of intelligence, they should correlate with academic achievement. If they do, then IQ tests have convergent validity as measures of intelligence. 13

14. Fundamental Principles of CV Theory What a test does not measure is given by what the test does not correlate with. If the test does not correlate with what it should not correlate (if it were a measure of what we claim it measures), we say the test as discriminant validity as a measure of the construct we claim it measures. If IQ tests measure intelligence, they should not correlate with the weather. If IQ test scores do not correlate with the weather, IQ tests have discriminant validity as measures of intelligence. 14

15. Fundamental Principles of CV Theory There is no single, clear measure of the validity of a test. The data and theory supporting a particular interpretation of what a test measures are not definitive. There will always be some amount of interpretation and/or uncertainty. There is no such thing as the validity of IQ tests. There is disagreement and debate about the validity of IQ tests as measures of intelligence because different researchers may have different interpretations of the meaning of the data that have been gathered with IQ tests. 15

16. Fundamental Principles of CV Theory The validation of a test as a measure of a construct is never fully completed. We will never know exactly what intelligence is because we could always make a new discovery tomorrow. 16

17. An Example Hypothesize that a test is a measure of the construct “attitude towards the church”. Theorize that if the test is a measure of attitudes towards the church scores should be somewhat different for churchgoers than non churchgoers (if the scores were very different then we would conclude that the test measures church attendance and not attitudes towards the church) Conduct the experiment. If the scores are somewhat different, this is evidence that the test measures attitudes towards the church. If the scores are not somewhat different conclude: a) the test does not measure attitudes towards the church, or b) the theory that attitude towards the church is related to church attendance is incorrect. 17

18. The Case of ADHD Hypothesize that ADHD is an abnormality of the brain. Theorize that if ADHD is an abnormality of the brain, children with ADHD should have different brain structure to children without ADHD Conduct the experiment. If brain structure is somewhat different, this is evidence that ADHD is a brain abnormality. If the scores are not somewhat different conclude: a) We did not measure the right brain abnormality, or b) the theory that ADHD is a brain abnormality is incorrect. 18

19. Some Examples From our Introductory Psychology Text "The neurological approach to understanding intelligence is currently in its heyday. Will this new research reduce what we now call the g-factor to simple measures of underlying brain activity? Or are these efforts totally wrongheaded because what we call intelligence is not a single general mental trait but several culturally adaptive skills? The controversies surrounding the nature of intelligence are a long way from resolution." 19

20. An Example of a “Validated” Psychological Test: The HAM-D 20

21. Question? Why are there questions on the HAM-D that do not denote depression? The questions on the test denote things that are correlated with depression. In CV Theory, what something is correlated with is what it measures. 21

22. Question? Recall the logic of inductive definition…how does it relate to the logic of CV Theory 21

23. Question? What is the difference between the logic of operationism and the logic of CV Theory? Read the Cronbach and Meehl paper carefully and you will find the answer to this question. 22

24. Something VERY Interesting Did Cronbach and Meehl, the inventors of CV Theory, agree with it? “Without in the least advocating construct validity as preferable to the other three kinds (concurrent, predictive, content), we do believe it imperative that psychologists make a place for it in their methodological thinking, so that its rationale, its scientific legitimacy, and its dangers may become explicit and familiar. 23

25. The Logical Problem of CV Theory Suppose we measure daily depression using the HAM-D depression scale. Suppose, on the same days we measure depression, we also measure the amount of rain on the day. Now, suppose we correlate depression scores with amount of rain and notice that levels of depression are correlated with amount of rain. According to the logic of CV Theory, this means that a measurement of rainfall is a measurement of depression. 24

26. Question? Does that make sense to you? 25

27. Question? What would an operationist say about that? 26