Research Methods Psych 402

1. Research MethodsPsych 402 • LECTURE 9 – • Sampling • VHS 514G2 (#14 – sampling)

2. Efficacy & Effectiveness studies • Efficacy studies • Active vs Placebo condition where there is no change in behavior expected • (Placebo effect - spontaneous recovery - is around 30 % for numerous conditions) • Effectiveness study: • Active vs current best treatment e.g., new drug vs Prozac in depression More ethical

3. Sampling

4. What is Sampling? “Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.” - William Trochim, 2002

5. Sample Characteristics • At issue: representative of population of interest?

7. Sampling Key Terms • Population - complete set of individuals having some common characteristic – e.g., Australians, Buffalo Bills fan • Sampling frame – subset of the population from which the sample is actually drawn – e.g., White pages, game attendees • Sample – the set of people actually studied (i.e., selected from the sampling frame) – e.g., Every 1000th person in the white pages, every 10th season ticket holder (which would not be a representative sample of this population)

8. Sampling Techniques • Probability sampling - each member of population has a specific probability of being chosen. • Non-probability sampling - arbitrary sample, often chosen with a purpose, that is not representative of the population

9. Probability Sampling Techniques • Random Sampling- everyone in population has an equal chance of being selected.

10. Probability Sampling Techniques • Stratified Random Sampling - population divided into strata, then random sampling from within each stratum

11. Probability Sampling Techniques • Cluster Sampling - identify ‘clusters’ of individuals, randomly select among cluster, then sample exhaustively inside a cluster

12. Video • Against All Odds, VH 514G part 2 • Episode 14 - Surveys and Sampling

13. Nonprobability Sampling • Self-selected • Haphazard • (e.g., canvas laundromat, beach, door-to-door, by phone) • Convenience • (e.g., clients, nearby clinic, psych students) • Systematic • (e.g., 1st 50 people in clinic) • Restricted • (e.g., SAT MS>NJ - 3 v 65%) • Purposeful

14. Non-Probability Purposeful Sampling • Quota Sampling – meet a quota • e.g., 50% psych, 30% economics, 20% law students • Modal – typical registered voter • Expert – ask only experts for a topic • Heterogeneity – seeking as many different individuals or type of opinions as possible • Snowball Sampling - ask each respondent if they know someone else suitable for survey • e.g., studying drug-users.

15. Representativeness of Sample depends on: • adequacy of sampling frame • accidental stratification with strata missing • selection strategy • adequacy of sample size • response rate • percent & representativeness of those who complete Small, good sample is better than large, poor sample (Gallup’s 50,000 vs Literary Digest’s 3 million poll of 1936 Roosevelt election)

16. External Validity

17. Threats to External Validity

18. Sampling Example:Sher Hite‘American Sexology’

19. Hite Report: Male-Female Relations • 127-item questionnaire about marriage and relations between men and women • 4500 USA women, 14 to 85 years

20. Some of Hite’s findings.... • 70% married for 5 years had one or more affairs... • 87% had a closer female friend than husband • 98% wanted “basic changes” to love relationships • 84% were emotionally unsatisfied • 13% were still in love with husband (after 2+y of marriage) “She goes in with prejudice & comes out with a statistic.”

21. Methodology 100,000 questionnaires to a variety of women’s groups - feminist organisations, church groups, garden clubs, etc. 4,500 replied(4.5% return rate)

22. “We get pretty nervous if respondents in our survey go under 70%. Respondents to surveys differ from nonrespondents in one important way: they go to the trouble of filling out what in this case was a very long, complicated, and personal questionnaire.”Regina Herzog, Michigan Institute for Social Research Sample Representative? - selection bias - nonresponse bias

23. Hite Revised by others • When representative sample studied by others, rate of female infidelity fell to single digits, etc.

24. Sources of Error Sampling Error(non-representative sampling frame) Nonsampling Error • non-response bias (those who respond doesn’t equal those who don’t) • inaccurate response • unreliable measurements • misunderstanding of instructions • social desirability or opposite • researcher expectancy

25. Minimizing Error • Sampling - take pains to ensure sampling frame is representative • Non-sampling • well-designed measures • trained interviewers • call backs (to eliminate nonresponse) • data checks

26. Research MethodsPsych 402 • LECTURE 10 – • Library Research & Hypothesis Testing • VHS 514J (#20)

27. Are motor skills impaired when you’re anxious? Are driving skills impaired by marijuana use? Does smoking pot make you less aggressive? Does drinking alcohol make you more aggressive? Are teenagers who smoke more antisocial than teenagers who don’t? Are children of alcoholics more likely to divorce than other groups? Are children of alcoholics more likely to develop mental illness? Does divorce alter a child’s self-esteem? Does teenage sex alter a girl’s self-esteem? Does teenage sex alter a boy’s self-esteem? Does watching violent TV make boys more aggressive? Does watching violent TV make girls more aggressive? Does watching TV impact school performance? Is episodic memory impaired by drinking alcohol? Is semantic memory impaired by drinking alcohol? Are photographic memories true records of events? Is eyewitness testimony reliable? Are women better at perceiving emotions than men? …. Library Query Assignment

29. Authors (Year). Article title. Journal Title, Volume Number, pages. Roosevelt, F.D., and Truman, H. (1945). Evolutionary strategies in politics. American Journal of Research, 35, 124-138. Authors (Year). Book Title. City published: Publisher. Arnheim, R. (1971). Art and visual perception. Berkeley: University of California Press. APA Citation Format: Journal or Book • Last Name, F. M., Last Name, F.M, and Last Name, F.

30. APA Reference Format: Book • Authors (Year). Book Title. City published: Publisher. • Arnheim, R. (1971). Art and visual perception. Berkeley: University of California Press.

31. Research MethodsPsych 402 • Hypothesis Testing

32. What is a hypothesis? 1. It is a clear statement about the state of the world, a prediction between variables and/or conditions. It is not a question 2. Identifies specific relationships between variables 3. Based in theory or existing knowledge 4. Concise 5. Testable 6. Readily understood by others

33. Example Hypotheses “Older people drive less aggressively than younger people” “The number and severity of driving infractions is negatively correlated with age, from 16 to 80 years of age”

34. Hypothesis testing • Research hypothesis - “It is predicted that older drivers will have fewer driving infractions than younger drivers” • Test the null hypothesis: i.e., that there is no relationship – can H0 be rejected or not?

35. Reject the null hypothesis (accept H1) or fail to reject the null hypothesis (accept H0) • Probability for rejecting the null - conventionally set at p < .05 (significance level)

37. VH 514 J #20

38. Designs • The simplest of all experimental designs is the two-group posttest-only randomized experiment. In design notation, it has two lines -- one for each group -- with an R at the beginning of each line to indicate that the groups were randomly assigned. One group gets the treatment or program (the X) and the other group is the comparison group and doesn't get the program (note that this you could alternatively have the comparison group receive the standard or typical treatment, in which case this study would be a relative comparison).

40. Research MethodsPsych 402 • LECTURE 11 – Unobtrusive Measures • VHS Natl Geographic Apes

42. Three ways to “observe” people • Directly • Observational • Survey (sometimes indirect) • Indirectly [unobtrusive measures] • Use Traces & other physical evidence • Archival (may be primary, direct source)

43. Birds vs Cards: 7-3 Birds vs Cats 6-0 Example: Which game was more exciting? Today’s Red Sox/Yankees or the Championship Series game

44. Sound volume (overall or maximum) # viewers # lead changes Self-report # fans left early Score # extra innings Importance of game Food sales Example: Which game was more exciting? Bills or Patriots game

45. Use Traces Routine Records Documents Products indirect measures Unobtrusive Measures

46. Erosion: measurement of selective wear on material Broken windows in public bldgs – community pride Tile replacement in front of museum exhibit – public’s interest Accretion: evidence of some deposit of materials Ticker tape volume – parade popularity Dust on library books – recency and famount of use Use Traces – any changes in environment due to human actions

47. Advantages of physical traces • Actual behavior can be measured • Physical properties are quantifiable • Cumulative – often reflects long process • Nonreactive and unobtrusive (usually)

48. Archival Research • Not all research means collecting new data • Advantages – non-reactive, cheap to collect (not so to interpret), usually a high quality source, continuous • Content analysis – inferences about specific characteristics of messages • Multi-method approach (converging datasets) • Possible problems • Selective deposit (bias in production) • Selective survival (discontinuous, outdated) • Possible errors in record keeping, storage • Perceiving spurious relationships

49. Examples of Routine Records

50. Observational Research • Decide on: • Artificiality of Setting • Any interaction with Observed • Awareness of the Observed