Psych 818 Psychological Measurement

1. DeShon - Introduction Psych 818Psychological Measurement

2. Preliminaries • Who am I? • Your job - Ask questions and find a way to become intellectually engaged • Self-handicapping is unproductive • You have a huge role in the success of the class • All work is your own! • Syllabus and Grading

3. Purpose Measurement Theory Measure Development Measure Use Measure Interpretation Exposure to Measurement thinking Exposure to Measurement papers Exposure to Measurement Standards

4. Responsibility! As a Psychologist YOU will likely have substantial effects on the course of many lives through the use of tests and measures Clinical Diagnosis Policy Making & Risk Factors Educational Opportunities Work Opportunities Example: Police promotion & expert witness You will have the power to help or hurt many people (and their families!)

5. Measurement & Policy • Measurement often starts as an academic investigation • Then, over time, transfers to policy and decision making • Examples: • The Kilogram conundrum • MPG vs GPM

6. Standards for Measurement APA standards for educational and psychological testing and assessment This is your professional bible. You will be held professionally accountable to these standards Therefore, you better know them!

7. Beginning Questions Why do we measure? What is measurement?

8. Why do we measure? Decision making Prediction Research

9. What is Measurement? S.S. Stevens The assignment of numerals to objects or events according to a rule. Warren Torgerson The assignment of numbers to perceived attributes of objects or events according to a rule. Lyle Jones Measurement is the determination of the magnitude of a specified attribute of an object or event in terms of a unit of measurement. Classification (including ordering) is not measurement.

10. What is Measurement? Cliff’s definition Good measurement is the assignment of numbers to perceived attributes of objects or events according to rules that are easily understood; are easily used; yield numbers that are as simply related as possible to as many other sets of measurements as possible.

11. Measurement Questions Fundamental measurement questions What is our best estimate of a psychologically relevant quantity Validity How precise is our estimate or... How much error is there in our estimate Reliability

12. Analogy with statistics Measurement questions look a lot like statistical questions What's our best estimate of a population mean How precise is our estimate (e.g., standard errors and confidence intervals)

13. History of Measurement Most early measurements occurred in physics, chemistry, and much later in biology (e.g., bacteria) Much attention paid to measuring specific quantities such as temperature, pressure, pH, etc... But no serious attention given to a general theory of measurement Also, little attention paid to a general model of reliability and validity

14. Two Physical Measurement Examples Temperature Fundamental to virtually all future discoveries in physics PH Fundamental to the understanding of the basic elements (i.e., hydrogen) and chemistry

15. Temperature • In the beginning there was the PERCEPTION of hot and cold • Aristotle’s fundamental qualities • Hot and cold were conceived of as opposing qualities of object • Virtually all measurements are rooted in subjective perception • Perceptions are fallible! • 200BC Philo’s air thermoscope

16. Temperature (cont.) • 1592 Galileo reinvents Philo’s (200BC) air thermoscope (instrument or indicator) • This is an indicator with no scale • Not a measurement? • Very sensitive to temperature changes • But also open to the air so sensitive to other variables (e.g., Air Pressure) • validity problem

17. Problems with Air Thermoscopes No scale – so a qualitative instrument Construct contamination - Validity open to the air so sensitive to other variables (e.g., Air Pressure) What liquid to use? (water, spirits, quicksilver) State changes, sensitivity and linearity of expansion rates No standardized construction so other instruments are not equivalent Inaccurate conceptualization of heat

18. Fix the validity problems Seal the thermoscope Duke Ferdinand II (1632) Pascal (1663) proved that air pressure caused errors in open air thermometers Are measurement errors reliability or validity Use quicksilver (i.e., mercury) Fareinheit

19. Fix the scaling problems Many individuals developed scales to assign numbers to the level of liquid in the thermometer (now a thermo – meter)! Newton (1700) "zeroth degree of heat" as melting snow and "33 degrees of heat" as boiling water Rømer (1708) Réaumur (1731) freezing point of water is 0 degrees, the boiling point 80 degrees Fareinheit (1724) 0 as the melting point of an equal mixture of ice and salt and 96 as blood heat Delisle (1732) 2400 graduations: 0 degrees as the boiling point and 150 degrees as the freezing point of water. Celcius (1742) freezing point of water at 0 °C and the boiling point at 100 °C

20. Proliferation of Scales

21. Units These scales result in “degrees” or units What are some psychological units? Check out these physical units! http://www.unc.edu/~rowlett/units/

22. Comparison of temp. scales kelvin Celsius Fahrenheit Rankine Delisle Newton Réaumur Rømer Absolute zero 0 −273.15 −459.67 0 559.725 −90.14² −218.52 −135.90 Fahrenheit's ice/salt mixture 255.37 −17.78 0 459.67 176.67 −5.87 −14.22 −1.83 Water freezes (at standard pressure) 273.15 0 32 491.67 150 0 0 7.5 Average human body temperature 310.0 36.8 98.2 557.9 94.5 12.21 29.6 26.925 Water boils (at standard pressure) 373.15 100 212 671.67 0 33 80 60 Titanium melts 1941 1668 3034 3494 −2352 550 1334 883 The surface of the Sun 5800 5526 9980 10440 −8140 1823 4421 2909

23. Transformations to Equate scales Celsius [°C] = [K] − 273.15 [K] = [°C] + 273.15 Fahrenheit [°F] = [K] · 9/5 − 459.67 [K] = ([°F] + 459.67) · 5/9 Rankine [°Ra] = [K] · 9/5 [K] = [°Ra] · 5/9 Réaumur [°Ré] = ([K] − 273.15) · 4/5 [K] = [°Ré] · 5/4 + 273.15 Newton [°N] = ([K] − 273.15) · 33/100 [K] = [°N] · 100/33 + 273.15 Rømer [°Rø] = ([K] − 273.15) · 21/40 + 7.5 [K] = ([°Rø] − 7.5) · 40/21 + 273.15 Delisle [°De] = (373.15 − [K]) · 3/2 [K] = 373.15 − [°De] · 2/3

24. Fix the standardization or calibration problems The technology of glass blowing made it very difficult to construct exactly equal instruments (the size of the liquid resivoir, the amount of liquid, the bore of the tube, etc…all must be identical to yield identical readings. Instead, calibrate or equate using fixed reference points Royal Society thermometers

25. Theoretical Understanding The various scales of measurement and the calibration of thermometers to fixed points allowed systematic investigations That resulted in improved understanding of what thermometers measure (heat) Kelvin’s scale of temperature (1848) Rational scale with absolute zero Based on gas laws and carnot cycles Adopted in 1954!

26. Example 2: pH - Acids & Bases In the beginning there was … perception Greeks attempted to characterize nature One fundamental character of nature was taste. Salty Sour the greek word for sour is the base of our word for acid Bitter Sweet

27. Acids and Bases Next, it turned out that sour tasting substances had other regular properties changes the color of litmus (a dye extracted from a lichen commonly known as dyer's weed.) Corrodes metals No real understanding of acids

28. Acids Huge proliferation of acid types (named by the method used to form the acid) Muratic acid, acetic acid, sulphuric acid, hydrofluoric acid, phosphoric acid, pyruvic acid No clear picture of the cause for the common functioning of acids Lavoisier (1776) thought it was oxygen that caused substances to be acidic Greek word oxein meaning 'sour' and gennan meaning 'to generate

29. Acids Leibig (1850s) noticed that all acids have hydrogen in common This insight required knowledge of the elements, molecules, and weights of molecules (another measurement problem) Sorenson (1909) – developed the scale for indexing the activity of hydrogen in a substance

30. Acids pH = (power of hydrogen ) is a measure of the activity of hydrogen ions (H+) in a solution Log of number of H+ molecules in a substance pH value has no unit but it is not an arbitrary scale The number arises from a definition based on the activity of hydrogen ions in the solution – has an absolute zero

31. Measurement of pH Many ways now exist to measure pH. Early measurements were based on the color of a substance resulting from a chemical reaction pH indicator A pH indicator is a halochromic chemical compound that is added in small amounts to a solution so that the pH of the solution can be determined easily. Hence a pH indicator is a chemical detector for protons (H+). Normally, the indicator causes the color of the solution to change depending on the pH. -Wilkpedia

32. pH Measurement Indicator Color - low pH Transition pH range Color - high pH Methyl violet yellow 0.0-1.6 blue-violet Thymol blue red 1.2-2.8 yellow Methyl yellow red 2.9-4.0 yellow Congo red blue 3.0-5.2 red Methyl red red 4.2-6.3 yellow Litmus (Azolitmin) red 4.5-8.3 blue Phenol red yellow 6.6-8.0 red Thymol blue yellow 8.0-9.6 blue Phenolphthalein colorless 8.2-10.0 pink Thymolphthalein colorless 9.4-10.6 blue Alizarin Yellow R yellow 10.1-12.0 orange-red Indigo carmine blue 11.4-13.0 yellow

33. pH Measurement Adding the indicator does not result in an absolute shift from one color to the other. Instead, it is a gradual shift and the resulting color must be compared to a SCALE to determine the pH.

34. Scaling pH Indicators

35. pH and IRT similarities The colormetric approach to pH has a great deal of overlap with current Item Response approaches to measurement – IRT Keep this in mind when we get to IRT

36. Causality and Measurement Measurement requires causal inference Changes in the level of the latent entity cause changes in the level of the indicator variable(s). This inference requires the same scientific method as any causal research question. Must show that the latent variable is the only cause of the indicator (construct validity)

37. Constructs What is a construct? Constructs become better defined via research Temperature, pH, bacteria, gas pressure, electricity 1. An unobserved cause of variation in an observable variable(s) 2. A label used to describe a pattern of observed covariances

38. Model of Measurement Indicator = an observable variable that is solely caused by the construct and sensitive to changes in the level of the construct Indicator / Instrument Scale: the rule for assigning numbers to the levels of the indicator or instrument Construct Construct = an unobservable but real causal variable (aka latent variable)

39. Basic Measurement Model Error 1 Error 2 Error 3 Measure 2 Measure 1 Measure 3 Construct

40. Some quotes to think about "I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.“ - Sir William Thomson, Lord Kelvin. 1889 “Whatever exists at all, exists in some amount. To know it thoroughly involves knowing its quality as well as its quantity” - Thorndike, 1918