Sampling

1. Sampling

2. Goals of Sampling Maximize External Validity The extent to which the results of a study generalize to the population of interest To be confident about such a generalization, the sample must be representative of the population of interest. External Validity is NOT Everything.

3. In Defense of External Invalidity (Mook, 1983) Sometimes researchers DO NOT necessarily care if their findings generalize to �real-life behavior in the real world� Sometimes the point is to test theoretically-derived hypotheses. Proving it wrong in a non-random sample is enough. The task is to decide what kinds of conclusions that you want to draw from a particular research project.

4. Goals of Many Experiments Minimize Threats to Internal Validity The random assignment of individuals to treatment conditions means that confounding variables are equally distributed across conditions As such, confounding variables are unlikely to be responsible for observed differences between treatment conditions.

5. Terminology Population Set of all people, objects, or events of interest to the researcher Stratum A variable that divides the population into mutually exclusive segments (e.g., gender, SES, political group) Population Element A single member of the population Sample A subset of the population used in research

6. Definitions Redux A population refers to �the aggregate of all of the cases that conform to some designated set of specifications� (Chein, 1981, p. 419) The aggregate is the target of generalization. A sampling frame is the list that identifies the elements of the population.

7. Two Ways to Get a Sample Probability Sampling: (a) Every element of the population has a known nonzero probability of being selected; (b) Random selection is used at some point in the process Nonprobability Sampling: Something else. Bottom Line: With nonprobability sampling it is NOT possible to estimate sampling errors. Moreover, judgments about external validity are rarely on firm ground.

8. Probability Sampling Any method of sampling that ensures that the elements in a population have a KNOWN and CERTAIN probability of being chosen. It is not the case that all elements MUST have the SAME probability. Simple Random Sampling: All elements have the SAME probability of being selected. Sampling Frame: List or Specification of the Population

9. Simple Random Sampling All elements have an equal probability of being selected. Population size is N. Sample size is n. If N = 1000 and n = 100 then the chance that any one element would be selected is .10.

10. A Word About Sample Size The precision of our estimates increases with sample size. It is sample size and NOT the size of the population that plays the vital role in determining precision. The function between sample size and precision is non-linear.

11. The Standard Error Decreases as Sample Size Increases (Example SD =10)

12. Another Example of Probability Sampling Stratified Random Sampling Divide the population into strata and then take a simple random sample from each subgroup. Sometimes called proportional random sampling Here we can over-sample a group if more statistical precision is desired for that group. This is called disproportionate stratified random sampling.

13. Disproportionate Stratified Random Sampling We can over-sample particular groups. This way we can obtain more precise estimates for strata that are small relative to the total population. Say there are 100 green people in State X which has 10,000 people (Green people are 1% of the population). If we really want to know what green people are thinking then we should over-sample them! We must still draw from the entire population of green people at random!

14. Nonprobability Sampling This method does not involve random selection. Let�s be Blunt: None of these methods are terribly good for supporting inferences about external validity. However, sometimes you cannot use probability sampling or do not need to use probability sampling.

15. Types of Nonprobability Samples Accidental, Haphazard or Convenience Sampling The "person on the street" interviews College student samples Clinical practice samples

16. Purposive Sampling  One or more specific predefined groups being sought. (They are the purpose!) Example: People in a mall with a clipboard Modal Instance Sampling: Sampling the most frequent case, the "typical" case, or the �average� person OK: What is the typical or modal person (e.g., the average voter)?

17. Quota Sampling  Specify the minimum number of sampled units that you want in each category. Not always concerned with having numbers that match the proportions in the population. Sometimes, you simply want be able to talk about even small groups in the population.

18. Snowball Sampling Begin by identifying someone who meets the criteria for inclusion in your study Then ask them to recommend other potential participants who meet inclusion criteria. Useful when you are trying to reach populations that are inaccessible or hard to find (e.g., homeless)

19. Experience Sampling

20. Sampling of Events In addition to sampling people, it is possible to sample Events or Experiences Examples: Age differences in emotional experiences; Job satisfaction in the workplace; Motives for drinking Often we use technology to help record thoughts, feelings, or behavior in the moment.

21. Sampling Strategies Sampling at Random Signaling device beeps at random periods throughout the day. Participants are asked to record thoughts, feelings, or behaviors. Event-Contingent Sampling Answer a short questionnaire after an event of interest occurs. Timing of response is determined by the participant

22. Triggers other than Events Signaling Use an electronic device to signal/remind the participants to complete a questionnaire Timing Controlled by Researchers Daily Diary End of day reporting about experiences and reactions Subject to more biases due to memory Does not require electronic devices

23. General Issues Brevity � Keep it Short Frame of reference (�at this moment�) Duration of the Study? Common for 7 Days 14 Days is probably better Possible for much longer intervals

24. Programs for conducting ESP ESP: The Experience Sampling Program A free software package for conducting experiments by experience sampling Runs on PDAs (Palm Pilots) Asks questions of the participant and records the answers and the participant's response time. The data may later be uploaded to a computer for analysis.

25. Experience sampling How do you feel right now? Please rate each feeling on the scale given. A rating of 0 means that you are not experiencing that feeling at all. A rating of 6 means that this feeling is a very important part of the experience. Not at all Very Much Happy . . . . . . . . . . . . . . . 0 1 2 3 4 5 6Frustrated/annoyed . . . . 0 1 2 3 4 5 6

26. The research team: Daniel Kahneman, Princeton University Alan Krueger, Princeton University David Schkade, University of Texas Norbert Schwarz, University of Michigan Arthur Stone, Stony Brook University

27. Day Reconstruction Method

28. Sample 909 women who worked the previous day 49% were European American 24% were African American 22% were Latinas/Hispanic 5% were Other Average Age: 38 years Average Income: $54,700

29. Step 1: Construct a Short Diary of the Previous Day �Think of your day as a continuous series of scenes or episodes in a film. Give each episode a brief name. Write down the approximate times at which each episode began and ended.� Average Number = 14.1 (SD = 4.8) Average Duration = 61 minutes (Range 15 minutes to 120 minutes)

30. Step 2: Answer Structured Questions about Each Episode

31. What were you doing? (check all that apply) __ commuting __ working __ shopping __ preparing food __ doing housework __ taking care of your children __ eating __ pray/worship/meditate __ socializing __ watching TV __ nap/resting __ computer/internet/email __ relaxing __ on the phone __ intimate relations __ exercising __ other (please specify________________) �

33. �How did you feel during this episode?Please rate each feeling on the scale given. A rating of 0 means that you did not experience that feeling at all. A rating of 6 means that this feeling was a very important part of the experience. Please circle the number between 0 and 6 that best describes how you felt. � Not at all Very much Impatient for it to end . . . . . . . . . 0 1 2 3 4 5 6 Happy . . . . . . . . . . . . . . . . . . . . . . 0 1 2 3 4 5 6 Frustrated/annoyed . . . . . . . . . . . 0 1 2 3 4 5 6 Depressed/blue . . . . . . . . . . . . . . 0 1 2 3 4 5 6 Competent/capable . . . . . . . . . . . 0 1 2 3 4 5 6 Hassled/pushed around . . . . . . . 0 1 2 3 4 5 6 Warm/friendly . . . . . . . . . . . . . . . 0 1 2 3 4 5 6 Angry/hostile . . . . . . . . . . . . . . . . 0 1 2 3 4 5 6 Worried/anxious . . . . . . . . . . . . . 0 1 2 3 4 5 6 Enjoying myself . . . . . . . . . . . . . . 0 1 2 3 4 5 6 Criticized/put down . . . . . . . . . . . 0 1 2 3 4 5 6 Tired . . . . . . . . . . . . . . . . . . . . . . . 0 1 2 3 4 5 6

35. Affect Calculation Positive: Average of Enjoyment, Warm, Happy Negative: Average of Frustrated, Worried, Depressed, Angry, Hassled, Criticized

36. Some Results - Activities

37. Some Results � Interaction Partners