Lecture 1

1. KV Petrides Lecture 1 Psychometrics Dr. K. V. Petrides www.psychometriclab.com

2. KV Petrides Individual differences • Most branches of psychology are interested in human (or animal) behaviour under different experimental conditions. • The assumption is that all people (or animals) are pretty much the same. • This assumption is patently wrong: • Some children develop faster than others • Some adults are less conscientious than others • Some dogs are more intelligent than others

3. KV Petrides Correlational vs experimental psychology • There has always existed a divide between correlational and experimental psychology. • In what is perhaps the most famous presidential address to the annual convention of the American Psychological Association (APA), Cronbach (1957) pointed out the existence of two opposing camps in psychology: • correlationists versus experimentalists • This module concerns primarily correlational psychology. L J Cronbach

4. KV Petrides Sir Francis Galton (1822-1911) • Sir Francis Galton was one of the greatest polymaths of all time. His credits include: • Father of individual differences. • Pioneer of correlation and regression. • Discoverer of ‘regression to the mean’ effect. • Pioneer of fingerprint identification and classification. • Inventor of the weather map. • Inventor of the Galton whistle (produces sounds into the ultrasonic range). • Founder (with Pearson and Weldon) of Biometrika.

5. KV Petrides Measurement & psychometrics I • “Everything that exists, exists in some quantity and can therefore be measured.” - E L Thorndike • Measurement concerns the systematic assignment of numbers to represent quantitative attributes of objects or events. • The concept of measurement is fundamentally important to the study of individual differences and of psychology, more generally. • The ability to measure and quantify constructs allows us to test falsifiable theoretical hypotheses, which is what establishes psychology as a science and distinguishes it from other disciplines (e.g., literature).

6. KV Petrides Measurement & psychometrics II • Psychometrics is the branch of psychology concerned with the scientific measurement of individual differences. • The accurate assessment of individual differences is important because theories can be tested if and only if the constructs involved can be measured. • In addition, a working knowledge of psychometrics is a sine qua non for the understanding of: • Advanced personality and intelligence theories. • Advanced psychological applications (occupational, educational, clinical, medical, military, etc.).

7. KV Petrides Psychometrics H Goldstein K G JÖreskog • Boosted by cheap computing power and some truly gifted mathematical statisticians, the field of psychometrics and behavioural statistics has been expanding with unprecedented speed. • These rapid and technically rigorous developments have created a need for a new breed of quantitative scientist who can work at the interface between mathematical statistics and applied research.

8. KV Petrides Reliability and validity • Two fundamental notions in psychometrics and measurement theory more generally (e.g., blood pressure) are reliability and validity. • Reliability:Very broadly, the dependability of a measurement instrument. • Validity: The property that a measurement instrument measures what it claims to measure. • A reliable test may be invalid but a valid test may not be unreliable.

9. KV Petrides Reliability I • There are several different types of reliability: • Internal consistency (Cronbach’s α): • measures the homogeneity of the test (the degree to which the various parts of an instrument measure the same variable). • technically, the proportion of true score variance accounted for by a fallible measure. Its square root indicates the correlation between observed and true scores on a variable.

10. KV Petrides Interpretation of Cronbach’s alpha • The sampling distribution of Cronbach’s alpha is unknown and, therefore, it is not possible to establish significance levels, rejection regions, etc. • Certain benchmark values have been proposed in order to interpret Cronbach’s alpha: • α < .60 poor • .60 < α < .70 adequate • .70 < α < .80 good • .80 < α < .90 very good • α > .90 perhaps too good

11. KV Petrides Reliability II • Alternate-forms reliability: Involves the construction of two tests of equal length by randomly sampling from the same domain (‘universe of items’). The Pearson correlation between these two tests (forms) is an estimate of the reliability. • Parallel-forms reliability: Involves the construction of two tests comprising items of the same difficulty, which leads to similar score distributions. The Pearson correlation between these two tests (forms) is an estimate of the reliability.

12. KV Petrides Reliability III • Split-half reliability: Randomly splits the test into two halves, calculates the Pearson correlation between them, and applies the Spearman-Brown prophecy formula to estimate reliability. Still used, but not recommended. • Test-retest reliability: Also known as ‘temporal stability’. Same test administered on two different occasions spaced about one month or more apart. The Pearson correlation between the two scores is an estimate of the reliability. Note this is very different from the foregoing types of reliability.

13. KV Petrides Validity I • There are very many different types of validity and, in some cases, there is more than one label to describe them. Here are the most common and important ones: • Construct validity: The degree to which a test measures a specified construct as determined by the interpretation of the psychological meaning of test scores and the implications of this interpretation. Term introduced by Cronbach and Meehl (1955). Construct validity is important from a scientific perspective and rests on the psychological theory underpinning an instrument.

14. KV Petrides Validity II • Criterion validity: the degree to which scores on a test correlate with scores on a relevant external criterion. It is a broad type of validity encompassing several more specific types (e.g., convergent, predictive, concurrent). • Convergent validity: the degree to which scores on a test correlate with variables they are supposed to correlate with, given the nature of the construct. • Discriminant validity: the degree to which scores on a test do NOT correlate with (are ‘independent of’ or ‘orthogonal to’) variables they are NOT supposed to correlate with, given the nature of the construct.

15. KV Petrides Validity III • Predictive validity: the degree to which scores on a test predict future behavior on a criterion variable. • Concurrent validity: is based on the correlation between predictor (test) and criterion scores obtained at approximately the same time (e.g., self-reported and clinically diagnosed depression). • Congruent validity: the degree to which a new test correlates with extant measurement instruments of the construct. It is a weak type of validity because extant measures may themselves have low validity. • Face validity: ‘experts’ review test contents to determine if they are appropriate ‘on their face’.

16. KV Petrides Internal validity • So far, we have looked at types of validity that are relevant to individual differences constructs, specifically, and correlational (nonexperimental) research, more generally. • There is another type of validity that is important mainly in experimental research, viz., internal validity. • Internal validity: The extent to which the observed effect on the dependent variable is caused only by the experimental treatment condition.

17. KV Petrides Threats to internal validity • There are six well-known extraneous and, therefore, potentially confounding variables that must be controlled. • History • Maturation • Mortality • Instrumentation • Statistical regression • Selection • Christensen, L. B. (2006). Experimental methodology (10th ed.). Needham heights: Allyn and Bacon.

18. KV Petrides Factor analysis

19. KV Petrides Factor analysis I • Principal component analysis (PCA) and principal factor analysis (PFA) are statistical methods used to reduce the dimensionality of a correlation (more often) or a covariance (less often) matrix. • Typically, the researcher’s interest is to find which of the variables in a particular set (items, scales, response latencies, etc.) form coherent and relatively independent subsets. • Variables that correlate highly together, but lower with other variables in the set, are grouped into components (PCA) or factors (PFA).

20. KV Petrides Factor analysis II

21. KV Petrides Factor analysis in SPSS Variables to be factored Do NOT change this default Click ‘extraction’ to get menu on the right Click ‘Scree plot’ and run the analysis directly afterwards

22. KV Petrides How many factors? • A crucial consideration in PFA concerns the number of factors to be extracted from the matrix. There are a host of extraction criteria available, two of which are especially common: • Kaiser eigenvalue criterion: This is the default in SPSS and extracts all factors with eigenvalues > 1. It is often very misleading and should never be adopted uncritically. • Scree plot: Eigenvalues are plotted against the principal components. The cut-off point for factor extraction is where the line changes slope. We extract all the factors that lie above the break. If there are two breaks in the slope, we extract all the factors above the leftmost break.

23. KV Petrides Scree plot The scree test is one of the best criteria for extracting the right number of factors. Its major disadvantage is that it often requires considerable experience and expertise to interpret it correctly. In many cases, it should be supplemented by additional criteria (e.g., parallel analysis). In the example below, the scree points to a 4-factor solution. However, the correct number of factors happens to be 3 in this case (I know this from additional analyses).

24. KV Petrides Factor rotation I • The main aim of factor rotation is to maximize interpretability. • Factor solutions with multiple variables having moderate loadings on multiple factors are virtually uninterpretable. • Rotational algorithms try to identify solutions in which: • Each factor comprises a few variables with high loadings and many variables with low loadings. • Each variable has a high loading on one factor and low loadings on all the other factors. • Such solutions are called ‘simple structure’ solutions.

25. KV Petrides Unrotated matrix

26. KV Petrides PCA/PFA II

27. KV Petrides Factor rotation II • There are two types of rotations: • Orthogonal rotations (e.g., VARIMAX) keep all factors at right angles (i.e., they do not allow them to intercorrelate). • Advantages: simple and easily interpretable solutions. • Disadvantages: solutions are often unrealistic. • Oblique rotations (e.g., OBLIMIN) allow factors to intercorrelate, if necessary. • Advantages: Solutions are precise, flexible, and realistic. • Disadvantages: Solutions are more difficult to interpret due to factor intercorrelations and the involvement of several matrices.

28. KV Petrides The structure of cognitive abilities • Cattell’s gc (crystallized) and gf (fluid) are second-order factors. • gc: consolidated knowledge arising from educational opportunities. • gf: the capacity to figure out new problems. Most loaded on tests with virtually no scholastic or cultural content (e.g, Raven’s). Cooper (2002)

29. KV Petrides The structure of personality

30. KV Petrides FA output I Percentage of item variance accounted for by factors. Percentage of total variance accounted for by each factor.

31. KV Petrides FA output II • We ignore the part of the output labelled ‘component matrix’ and focus on the ‘rotated component matrix’. Factor loadings

32. KV Petrides On the Web • http://issid.org/issid.html • International Society for the Study of Individual Differences (ISSID). • http://www.personality-project.org/ • Maintained by William Revelle.