Levels of Measurement in Social Science Research

1. Measurement, Part II The challenge

2. Levels of measurement • Nominal Ordinal (nonparametric) • Interval Ratio (parametric) • Nominal: lowest level, simply classifying observations into categories • Categories should be mutually exclusive and exhaustive • Examples: gender, major, religion, state

3. Levels of measurement (continued) • Numbers assigned to the categories have no numerical meaning. Assign individuals, and report the % falling into each category. Fewer statistical techniques can be used • Ordinal measurement: one observation represents more of a given variable than another observation

4. Levels (continued) • Rankings • Newly developed tests • Ranks tell whether one observation represents more or less than another, but not how much more or less--nothing is known about the exact difference between any two ranks • Rankings of crime seriousness

5. Levels (continued) • Interval: like an ordinal scale, but has equal intervals between the units of measurement. Not only an ordering, but also the same distance or degree of difference between observations • For example, 81 is 1 point away from 80, etc. • Well-developed tests are interval level

6. Levels (continued) • With interval measurement, can do addition, subtraction, multiplication and division, more statistical tests • Ratio measurement: like interval, with the additional property of a true zero. • an individual could have two or three time as much of a trait as another with ratio measurement

7. Levels (continued) • height or weight. A 200 lb person weighs twice as much as a 100 pound person • Not true for interval. For example, no such thing as an IQ of 0, and a person with an IQ of 100 is not twice as smart as someone with an IQ of 50

8. Levels (continued) • Most measurement in the social sciences is interval measurement

9. Developing questions • Most common method of measuring in the social sciences is to ask questions • Types of questions: open ended and closed ended • Open: provide own answer. Provides more information, may result in ideas or considerations the researcher had not tthought about

10. Questions (continued) • Disadvantages of openended: harder to categorize answers. • They take longer to answer. Some subjects might not answer such questions, and you may end up with biased results • Hite reports. Ann Landers

11. Questions (continued) • Closed ended questions: select an answer from a list of choices. • Advantage: quick, easy to code • Problem: making sure all the possible reasons are covered

12. Guidelines for asking questions • Clear, terms should be defined • Avoid double-barreled (asking two questions in one) • Subjects should be competent to answer the questions • Questions should be relevant to the subjects • Short

13. Guidelines (continued) • Avoid negative, or emphasize NOT • Unbiased • Recognize social desireability as a factor when developing questions (imagine how you would feel giving any of the answers) • Use of contingency questions

14. Guidelines (continued) • Use of matrix questions • Try to determine if the ordering of the questions will affect the answers • Rule of thumb: if questions are written, start with interesting but not threatening questions, and put routine questions at the end. With interviews, ask routine first so subjects feel comfortable.

15. Guidelines (continued) • Always include instructions • Pretest questions

16. Sampling • Population: all subjects one is interested in. Very large or very small • Element • Sample: portion of population • Sampling frame: list of people (elements) in the populaiton

17. Sampling: continued • Representative sample: if the overall characteristics of the sample approximate the important characteristics of the population • Biased sample: not representative • Parameters and statistics • Why sample? time and money

18. Sampling in the U.S. • Literary Digest polls. Accurate until 1936, when Landon was predicted as winner of the presidential election • Reasons: (1) low return rates (2 million out of 10 million) and (2) sampling frame (telephone directories and lists of auto owners) • Poor sampling frames result in bias

19. Sampling in the U.S. (continued) • 1948 Gallup poll predicted Dewey would win. Problems: (1) stopped polling in Oct.; (2) quota sampling • Two types of sampling: probability and nonprobability sampling • Probability sampling uses the laws of probability, whereas nonprobability does not

20. Probability • p = number of times an event could occur / total number of outcomes. • Can be express as a fraction, a %, as chances out of 100, or as a decimal. • P can range from 0 (no probability to 1 (certainty)

21. probability (continued) • A sample will be more likely to be representative of a population from which it is selected if all members of the population have an equal chance of being selected in the sample • Sampling error: error due to the fact that the sample is not representative • Necessity of a complete sampling frame

22. Probability sampling methods • Simple random sampling: (out of a hat, random numbers) • Systematic random sampling: every nth element is cnosen, select first element at random (random start) • Stratified random sampling • 1. Divide sample into subgroups based on important population characteristics

23. P sampling methods (continued) • 2. Randomly sample from those subgroups in proportion to their percentage in the population • Choice of stratification variables will often depend on what variables are available, and how much is known about the population • This technique most likely to be representative

24. Nonprobability sampling • Probability sampling only works if there is a sampling frame of the population. Sometimes that is not possible (i.e., criminals, drug addicts, etc.) • Nonprobability sampling methods, while running the risk being unrepresentative might be the only option

25. Nonprobability sampling • Convenience: the captive audience • College students and prisoners • Purposive: researcher uses judgment • for Example, the mentally ill. Works best if the criteria for inclusion are clear. • Quota: like stratified random. Groups are selected on the basis of known variables

26. Nonprobability (continued) • In quota sampling, however, subjects are not selected randomly--subjects with the desired characteristics are selected until a quota is filled for each subgroup • Snowball: each subject is asked to suggest other subjects

27. Tips about sampling • Sample size: unusually the number of subjects needs to be at least 30. If several groups within the sample are to be compared, there needs to be at least 10 per group. • The larger the number of subjects (N), the less likely sampling error • There will always be “mortality”

28. Tips (continued) • The greater the heterogeneity of the sample, the larger the sample must be. The less population diversity, the smaller N might be. • N is often determined by time and money factors

29. Ethics • No harm to subjects • Ethics boards or committees, especially with captive populations such as prisoners or children (children--parents must give permission; correctional systems have their own boards to protect rights)

30. Ethics (continued) • Subjects’ right to privacy • Confidentiality and anonymity • The only exception: if someone is in danger • Voluntary participation • Informed consent: nature of the study, possible effects, being able to withdraw at any time

31. Ethics (continued) • Deception and debriefing • Analysis and report: do not “fake” results, or cheat, or conceal technical shortcomings of the study • Milgram’s obedience study • Zimbardo’s mock prison study • Tearoom trade study

32. Politics and ethics • Research can be used for good or for evil • Ex: development of the atom bomb • Project Camelot: assessing internal potential for war, actions governments might take

33. Politics and ethics (continued) • Misinterpretation of studies • Theory of evolution led to social Darwinism, which led to eugenics and justifications for Hitler’s purges • Politics may affect how studies are interpreted (pornography)