Scaling

1. Scaling

2. Survey Research • Questionnaires and Interviews • Both experimental and nonexperimental research • Read pages 212 through 223 in Martin • Scaling = construction of instruments for measuring abstract psychological constructs.

3. Thurstone Scales:Method of Equal-Appearing Intervals • Define the Concept • Generate Potential Scale Items • about 100 statements • differ with respect to the extent to which agreement indicates presence of the attribute to be measured

4. Thurstone • Rating the Potential Scale Items • Judges rate the items on 11-point scale • 1 = agreement indicates very low amount of the attribute • 11 = agreeing indicates very high amount of the attribute • encourage judges to use entire range of scale, assigning some statements to each of the 11 values – sort them into 11 piles.

5. Thurstone • Computing the Scale Score Values for Each Item • Find median and SD or inter-quartile range • Arrange in table • Sort by median • Within items with same median, sort by SD or inter-quartile range

6. Thurstone • Select the Final Scale Items • 1 (or 2 or 3) item(s) for each possible scale score value • Prefer items with low variability among judges • End up with 10-30 items • See this example • Items and scale score values are shown on the next slide.

7. People with AIDS deserve what they got. (1) AIDS is good because it helps control the population. (2) AIDS will never happen to me. (3) I can't get AIDS if I'm in a monogamous relationship. (4) It's easy to get AIDS (5) Because AIDS is preventable, we should focus our resources on prevention instead of curing (5)

8. People with AIDS are like my parents (6) If you have AIDS, you can still lead a normal life (8) AIDS doesn't have a preference, anyone can get it (9) AIDS is a disease that anyone can get if they are not careful (9) Aids affects us all (10) People with AIDS should be treated just like everybody else. (11)

9. Thurstone • Administer the Final Scale • Randomize the order of the items • For each item, respondent chooses Agree or Disagree • For each item the scale score is the median from the judges’ ratings • Total Score = mean scale score for items on which the respondent agreed.

10. Thurstone • Thurstone scales are rarely used these days • They are just too much trouble to create. • Likert scales were developed in response to this difficulty

11. Guttman Scaling • Define the Concept • Generate Potential Scale Items • Evaluating the Potential Items • For each item, judges are asked if someone high in the attribute would agree with the statement – Yes or No.

12. Guttman • Conduct a Scalogram Analysis of Judges’ Responses • use special software to do this • if successful, it will create an ordered list of items such that • agreeing with the first item indicates you have at least a little of the measured attribute • agreeing with the second indicates you have at least a little more of the attribute • etc.

13. Guttman • The scalogram analysis also computes a scale score value for each statement. • See the example in Trochim’s Internet document Guttman Scaling (reproduced on the next slide). • It is assumed that anybody who would agree with the nth item would also agree with all preceding items. • The order of the items may be scrambled prior to administering the scale.

14. I believe that this country should allow in more immigrants. I would be comfortable with new immigrants moving into my community. It would be fine with me if new immigrants moved onto my block. I would be comfortable if a new immigrant moved next door to me. I would be comfortable if my child dated a new immigrant. I would permit a child of mine to marry an immigrant.

15. Guttman • Administer the Final Scale • Respondents are asked to check items with which they agree • Respondent’s score = sum of the scale score values for checked responses. • Like Thurstone scales, Guttman scales are not often used these days.

16. Likert Scales • Define the Concept • Generate Potential Items • About 100 statements. • On some, agreement indicates being high on the measured attribute • On others, agreement indicates being low on the measured attribute

17. Likert • Instead of a dichotomous response scale (agree or disagree), use a multi-point response scale like this:

18. Likert • Evaluating the Potential Items • Get judges to evaluate each item on a 5-point scale • 1 -- Agreement = very low on attribute • 2 – Agreement = low on attribute • 3 – Agreement tells you nothing • 4 – Agreement = high on attribute • 5 – Agreement = very high on attribute • Select items with very high or very low means and little variability among the judges.

19. Likert • Alternate Method of Item Evaluation • Ask some judges to respond to the items in the way they think someone high in the attribute would respond. • Ask other judges to respond as would one low in the attribute. • Prefer items that best discriminate between these two groups • Also ask judges to identify items that are unclear or confusing.

20. Likert • Pilot Test the Items • Administer to a sample of persons from the population of interest • Conduct an item analysis • Prefer items which have high item-total correlations • Consider conducting a factor analysis

21. Likert • Administer the Final Scale • on each item, response which indicates least amount of the attribute scored as 1 • next least amount response scored as 2 • and so on • respondent’s total score = sum of item scores or mean of item scores • dealing with nonresponses on some items • reflecting items (reverse scoring)

22. Psychometric Analysis • Whenever you use such an instrument, you should conduct basic psychometrics • See “Cronbach’s Alpha and Maximized Lambda4” • Factor analysis can help determine if the instrument is unidimensional or not. • Factor analysis of the Cultural Values Survey • Factor analysis of Patel’s SBS.

23. PCA of Cultural Values Survey • 45 items • Reduced to seven orthogonal components • Weighted linear combinations of variables • Used as outcome variables in Culture x Sex x Age Group factorial ANOVA • Loadings used to decide what the components are

24. 7 Components • Family Solidarity (respect for the family) • Executive Male (men make decisions, women are homemakers) • Conscience (important for family to conform to social and moral standards) • Equality of the Sexes (minimizing sexual stereotyping)

25. Temporal Farsightedness (interest in the future and the past) • Independence (desire for material possessions and freedom) • Spousal Employment (each spouse should make decisions about his/her own job)

26. ANOVA Results • US Students (especially the women) • Sexually egalitarian • Desire independence • Family not important to younger students • Taiwanese Students • Temporally farsighted • Men more sexually egalitarian than women • Women more interested in independence

27. Mexican Students • Like the Taiwanese, family is important • Like the US students , not temporally farsighted • Men thought independence more important than did women

28. Patel’s SBS • 21 items, measures aggression toward homosexual persons • Cronbach’s alpha = .91 • FA revealed three factors • Avoidance Behaviors (13 items) • Moving away from gay • Staring to communicate disapproval of proximity

29. Aggression from a Distance (6 items) • Writing anti-gay graffiti • Damaging gays’ property • Making harassing phone calls • Up-Close Aggression • Physical fighting