Sociology 202 Martin Lecture Outline 15: October 25, 2005

1. Sociology 202 (Martin)Lecture Outline 15: October 25, 2005 Sample design: based on Babbie, pages 199-218 issues related to sampling frames simple random samples and some alternatives Description of the GSS sample design Information you can use to write the first paragraph of your data and methods section.

2. For discussion According to a study of 4,165 U.S. military personnel in October 2004, 73% said they would vote for Bush 18% said they would vote for Kerry Which of the following might be true? “You can’t dismiss” the results, even though the results are probably a bit extreme. The Bush campaign has been effective in creating the impression that, if elected, Kerry might “cut and run” in Iraq. This is “an inaccurate e-mail survey” and Kerry has “the vision and values to keep faith with military families and America’s veterans.”

3. Sampling frame: the link between a sample and a population sampling frame: The list or quasi-list of units composing a population from which the sample is selected. If an individual has no chance of appearing in the sample, that individual is not in the sampling frame. If the sample is to be representative of the population, the sampling frame should include all or nearly all of the members of the population.

4. Sampling Frames: Issues to watch out for (1) 1.) Findings based on a sample can be taken as representing only the aggregation of elements that compose the sample frame. For example: About a million persons are U.S. military personnel. About 31,000 persons subscribe to Army Times Publications. What is the sampling frame? What is the population the survey refers to?

5. Sampling Frames: Issues to watch out for (2) 2) Often, sampling frames do not truly include all of the elements their names might imply. Omissions are almost inevitable. For example: About 31,000 persons were emailed information about the survey. 4,165 responded back to a secure web site. Are the persons who responded typical representatives of the whole sampling frame? One columnist commented that this survey is fair, because if anything, the people most likely to reply would be the ones most upset about the Iraq war and most pro-Kerry. Do you agree?

6. Sampling Frames: Issues to watch out for (3) 3) To be generalized even to the sampling frame, all elements must have equal representation in the frame. Typically, each element should appear only once. For example: The web survey probably had safeguards in place to prevent multiple ballots. We don’t know with certainty that this is the case, but the survey is probably okay in this respect.

7. Types of sampling designs Simple random sampling: Use a random number generator to pick observations from the sample one at a time, thereby insuring that every individual has an equal chance of being in the sample, and every possible sample of individuals has an equal chance of being the sample. Advantages: on average, the sample characteristics will be the same as the population characteristics the confidence intervals and statistical significance are as you calculate them in Socy 201. Disadvantages: it is time consuming to draw a random sample one individual at a time. in a nationwide study, you might end up sending an interviewer to eastern North Dakota just to interview one person.

8. Types of sampling designs (2) Systematic random sampling: Use a random number generator to pick a number from 1 to k, then choose that individual and every kth individual in the sampling frame. Advantage: saves time compared to drawing a true random sample one individual at a time. Disadvantage: while every individual has the same chance of being in the study, only a few patterns of samples are possible (usually this is not a problem).

9. Types of sampling designs (3) Stratified sampling: Break the sampling frame into groups based on social characteristics, then take a sample from each group. Advantage: Makes the sample less likely to deviate substantially from the population in one way or another. Disadvantage: when you put the whole sample back together, you might need to use sampling weights to calculate estimates.

10. Types of sampling designs (4) Multistage cluster sampling: Sampling that proceeds in stages. First a sample of groups is drawn from the sampling frame (as in sampling cities in the U.S.), then a sample of individuals is drawn from each cluster. Advantage: Much less expensive than a true random sample. Disadvantage: Although the sample statistics still correspond to population parameters on average, they tend to be a bit further off (high or low) than in a true random sample.

11. Other definitions sampling ratio: the chance of one individual in the sample frame appearing in the sample: sampling ratio = sample size / population size sampling interval: the number of spaces k between cases when you draw a systematic random sample sampling interval = population size / sample size weight: when different groups in a stratified sample have different sampling ratios, the weight of each observation is a multiplier used to give every observation the same sampling ratio. a properly weighted sample gives you an appropriate guess of the population parameter.

12. Sample characteristics of the General Social Survey The sampling frame of the General Social Survey is all U.S. adults living in households. The sampling frame includes 97.3 % of all U.S. adults. Who does not live in a household? college students in dorms military personnel in barracks prisoners elderly persons in retirement homes.

13. Age and the sampling frame of the GSS For U.S. adults age 18 – 24, only 90.6 % of adults live in households. Do you? Some researchers limit the GSS to individuals 25 and older if they worry that college students in dorms would have scores well outside the sampling frame. For U.S. adults age 25 – 64, about 99% of adults live in households. For U.S. adults 75 and older, about 88.6% of adults live in households. Dead people by definition do not live in households. remember this if you are trying to do a study of how smoking affects self-reported health.

14. Does the GSS sample really draw from all the adults in its sample frame?. After the GSS is sampled, only 70% of persons in the sample actually respond to the survey (in the 2004 study). 23% refuse or cut the survey off in the middle 2% are unavailable or can’t be found 5% are missing for other reasons In general, a response rate of 60% or more is considered minimally acceptable, but you should check your results in any way you can.

15. Other characteristics of the GSS The sampling ratio of the GSS in 2002 was about 2,800 (in the sample) / 200,000,000 (adults 18+ in households in the U.S.) 2,800 / 200,000,000 = 1 / 70,000 What is your chance of appearing in the GSS if you live in a household (apartment or house?) if you live in a dorm? How on earth can you get meaningful data on society when you look at only one person in 70,000?

16. Weights in the GSS There were some years (like 1987) where the GSS used stratified weights for black oversamples (variable = formwt) The 2004 GSS provided weights based on nonresponse rates. For years before 2004, researchers often used the GSS without a weighting variable. However, the GSS has always been weighted in the sense that only one adult in each household is ever sampled. If you live alone, you are automatically sampled if the GSS comes to your door If you live with a spouse and his/her mother-in-law, you have only a 1/3 chance of being sampled if the GSS comes to your door.

17. Summary Questions 1.) Using a 2004 e-mail survey of 31,000 subscribers to Army Times publications, a researcher concludes that U.S. military personnel strongly supported George Bush in the 2004 election. Identify the sampling frame and the population of interest, and evaluate the researcher’s conclusion. 2.) Describe three issues or concerns one might encounter when identifying a sampling frame. 3.) Identify two types of probability sampling designs other than simple random sampling. Describe a potential advantage and disadvantage of each type of sampling design. 4.) What types of people are not included in the sampling frame of the General Social Survey? Give an example of a research question that might be limited by the incomplete sampling frame of the GSS. 5.) What are three possible ways that weighting occurs in the GSS – that is, three ways that a person in one household might have a higher or lower chance of being in the survey than another person in another household?

18. Assignment for next lecture Write a paragraph describing how the General Social Survey was conducted Try to make it suitable for use in the data and methods section of a term paper.