Sampling Issues for Telephone Surveys in Scotland

1. Sampling Issues for Telephone Surveys in Scotland Gerry Nicolaas Survey Methods Unit National Centre for Social Research 13 January 2004 email: G.Nicolaas@NatCen.ac.uk

2. This session will focus on: • Surveys of the general population • Probability sampling

3. Potential sampling advantages of telephone interviewing: • No need for geographical clustering. • Inclusion of more remote areas. • Cheaper recalls

4. Telephone methods are not widely used in the UK for social surveys among the general population

5. Obstacles to selecting a representative and unbiased probability sample of the general population: • Households with no telephone • Unlisted telephone numbers • Mobile phones

6. (1) Households without telephone: • Proportion of households with no phone in Scotland declined from 5% in 1998 to less than 0.5% in 2002. • Households with no phone tend to belong to the most socially and economically deprived groups. • The exclusion of households with no phone would not introduce notable bias in a general population survey. Source: Taylor, 2003

7. (2) Unlisted telephone numbers: • In 1998 26% of telephone owning households (fixed lines) in Scotland were ex-directory • Households with unlisted phone numbers tend to be: - Smaller than average - Headed by someone in social classes IV or V Source: Beerten and Martin, 1999

8. (3) Mobile phones: • Proportion of households in Scotland with at least one mobile phone increased from 13% in 1998 to 80% in 2002 • 5% of households only have mobile phone • Mobile phone owners without a fixed line tend to be younger, lower socio-economic group, lower income, unemployed. • Mobiles tend to be not listed in phone directories. • Mobile phone numbers can’t be linked to geogr. area. • Mobiles belong to individual rather than household. Source: Taylor, 2003; Ofcom 2003

9. Sampling from telephone directories: • Select samples systematically or use random numbers to select page numbers and phone numbers. • Non-coverage of - households with no telephone, - households with only mobile phones, and - households with unlisted telephone numbers. • Coverage= about 65% of Scottish population. • Under-representation of - most socially & economically deprived groups, - young mobile people, - people living in small households.

10. Plus digit sampling: • Sample is selected from the telephone directory and a fixed number is added to the last digit. • Similar procedure replaces the last one or more digits with a random number. • Sample will include unlisted numbers, but: • - proportion of unlisted numbers is lower than in the population; • - profile of the achieved sample reflects that of listed telephones rather than that of all telephones; • - probabilities of selection are unknown & vary because of unequal distribution of listed & unlisted numbers.

11. Random Digit Dialling (RDD): • RDD uses comprehensive list of of valid area codes and prefixes and adds randomly generated suffixes. • OFCOM database of blocks of 10,000 numbers; e.g. 0131 557 xxxx • All telephone households have a known non-zero chance of selection (fixed telephones and mobiles). • Exclusion of households without phone will not introduce notable bias • But: High proportion of non-working numbers and other ineligibles (e.g. business): about 80% Sources: http://www.ofcom.org.uk/working_w_ofcom/numbers/numbers_administered/?a=87101#geog1 Nicolaas and Lynn (2002)

12. RDD hit rates can be improved: • Advanced telephony systems can screen out most non-working numbers; • RDD samples can be matched against yellow pages to remove listed business numbers; • Adopt sample designs that improve hit rate, - Mitofsky-Waksberg two-stage method - List-assisted methods

13. Should mobile phone numbers be included in RDD sample? • Exclusion of mobile phone numbers will result in under-coverage of younger people, lower socio-economic group, lower income group, the unemployed. • Inclusion of mobile phone numbers will result in further reduction in effective sample size due to post-weighting of households with more than one phone (fixed & mobile) • Inclusion of mobile phone numbers will increase ineligibles for surveys covering areas smaller than UK (e.g. Scotland).

14. Dual frame sampling: • E.g, a sample of listed telephone numbers supplemented with a RDD sample. • Directory status of each RDD interview must be known • Sampling administration is more complicated. • Data to be combined using post-stratified dual-frame estimators. • Unclear whether this approach is feasible and whether the gains are worthwhile.

15. Single frame, dual mode approach: • Use a frame with good coverage (e.g. PAF) • Telephone interviews for those sampled cases with a matched telephone number. • Postal questionnaires or face-to-face interviews for those without a matched telephone number. But: • - low matching rates; • - costly if unmatched cases interviewed face-to-face; • - relatively low response rates if postal questionnaires are sent to unmatched cases; • - possibility of mode effects.

16. Conclusion: • None of the sampling methods are perfect. • Choice of sampling method depends on specific survey. • On the whole, RDD appears to be superior method: • - Complete coverage of all households with phones. • - Non-coverage of households without phone can be ignored • - Sampling frame is accurate (Ofcom database). • - Precision of estimates is high (unclustered sample). • - Relatively cheap and easy to select RDD sample. • - Fieldwork efficiency can be improved. • But the proportion of mobile only households needs to be monitored. • Response rates?

17. Another sampling issue: Respondent selection • Include all eligible household members or select one at random with post weighting for unequal selection probabilities. • Random selection of one eligible household member tends to be most common approach for telephone surveys • “Gold standard” is Kish Method but some claim this requires too much info up front and may reduce response. • Most common method is Last/Next Birthday method. • NatCen experiment showed no significant difference between the two methods in response rates nor sample compositions. Source: Tipping & Nicolaas (2001)

18. Suggested reading: • Beerten & Martin (1999) Household ownership of telephones and other communication links: implications for telephone surveys. Surv. Methodol. Bull., 44: 1-7. • Collins (1999) Sampling for UK telephone surveys. JRSS(A) 162: 1-4. • Collins & Sykes (1987) The problems of non-coverage and unlisted telephone numbers in telephone surveys in Britain. JRSS(A)150(3): 241-253 • Lepkowski (1988) Telephone sampling methods in the United States. In Groves et al (eds), Telephone Survey Methodology. John Wiley and Sons: New York • Nicolaas & Lynn (2002) Random-digit dialling in the UK: viability revisited. JRSS(A) 165: 297-316. • Taylor, S. (2003) Telephone surveying for household social surveys: the good, the bad and the ugly. Surv. Methodol. Bull., 52: 10-21. • Tipping & Nicolaas (2001) Respondent selection procedures for telephone surveys. Survey Methods Newsletter, 21(1): 4-7, Nat. Centre for Soc. Res.