Analysis of Factors Related to Nonresponse for Landline and Cell Phone Surveys in China

1. ITSEW 2010 Analysis of Factors Related to Nonresponse for Landline and Cell Phone Surveys in China Yan Jiang School of Statistics, Renmin University of China Visiting Professor, Iowa State University, Department of Statistics and Center for Survey Statistics and Methodology

2. Outline • Introduction • Survey Design • Nonresponse Rates • Call AttemptAnalysis • Interviewer Analysis • Conclusions

3. Introduction • Surveys in China • High nonresponse rates • Low contact rates (RDD Sampling) • High refusal rates • Change of coverage • Steady increase in cell phone frame • Decrease in landline phone frame since 2006

4. Amount of telephone users (Ten thousands) • 2009 • Cellphone Users: • 747.38 million • Landline Users: • 697.02 million • Overlap: unknown • Total : • 1335 million

5. Survey Objectives • To evaluate the utility of selected methods (e.g., increased call effort) in telephone surveys of adults in China • To identify factors related to nonresponse rates (e.g., interviewer voice quality, call effort) • To study how the effect of these factors on nonresponse vary by sample frame (i.e., landline, cell)

6. Survey Design • Population: Adults in China (over 18 years old) • Two-phase Sampling Design • Phase 1: Dual frame sample of phone numbers to sample adults • Phase 2: Sample of nonrespondents • Data Collection Mode: CATI • Phase 2: Modified methods

7. Survey Design in First Phase • Sample Size: 17,162 phone numbers • Cell phone: 9678 • Landline: 7484 • Stratified RDD Sampling • Strata: Provinces • Allocation: proportional to Province HH pop. • Standard call methods • Up to 3 call attempts – usually 2 • Ring for about 20 secs

8. Survey Design in Second Phase • Population: nonrespondents in first phase • Sample size: 1549 • Strata: province, frame source, noncontact and refusal numbers • Effort to improve the response rate (both cell phone and landline survey) • Up to 15 call attempts were made to contact • Wait until at least one minute in each dial process

9. Nonresponse Rate in First Phase Survey

10. Nonresponse Rate in Second Phase Survey

11. Landline Cell phone Number of call attemptsin refusal sample (stratum) of Phase 2 • Cumulated Refusal Conversion Rate of landline survey is higher than cell phone survey when the number of attemptsis less than 5. • Cumulated Refusal Conversion Rate increases rapidly up to about 6 call attempts. Number of call attempts

12. logistic model for Refusal ConversionDep. Variable=1, refusal unit convert to respondent The relationship between above factors and refusal conversion rates differs for the landline survey and the cell phone survey.

13. Number of call attemptsin non-contact sample (stratum) of Phase 2 • Cumulated Phase 2 Response Rate of landline survey is lower than cell phone survey in non-contact sample. Landline Cell phone Number of call attempts

14. Distribution of the number seconds until the phone is answered • On average,10 additional seconds per interview are required for the phone to be answered in the cell phone survey relative to the landline survey. Cell Phone Land line

15. 20 15 Frequency 10 Mean = 0.375 5 Std. Dev. = 0.083 N = 108 0 0.20 0.30 0.40 0.50 0.60 0.70 Cooperation Rate Phase 1 Interviewer AnalysisCooperation rate distribution of interviewers

16. Phase 1 Interviewer Analysis • Which interviewer characteristics affect cooperation rates? • Interviewer characteristics • Skill level • Vocal quality • Control for respondent characteristics • Gender, education, career, income, location of residence, sample frame

17. Phase 1 Interviewer Analysis • Hierarchical Generalized Linear Model • Level 1 model (respondent level) • Level 2 model : (interviewer level) i : respondent; j: interviewer; :refusal rate : variables of respondent : variables of interviewer

18. Variable Definitions

19. Results Both the demographic characteristics of respondents and interviewers have significant impacts on refusal rates. Respondent Interviewer Survey type does not have significant effects on refusal rates in the model Both the interviewers’ speaking skill and natural vocal characteristics have significant impacts on refusal rates.

20. Conclusions • Dual frame sampling is necessary for reducing coverage bias for telephone surveys in China. The ineligible (e.g., non-working, business) rate in cell phone surveys appears to be lower than in landline surveys in China. • High non-response rates can be reduced by effective voice training of interviewers, more call attempts, and extending waiting time before giving up on the call. • The demographic characteristics and behaviors of cell phone users and landline telephone users are different, which may have implications for nonresponse bias. • Interviewers should wait longer for the respondent to answer the phone in the cell phone survey relative to the landline survey. • Refusals occur more quickly in the cell phone survey than in the landline survey.