To Cell or not to Cell

1. To Cell or not to Cell Presentation to Presentation to the Ottawa Chapter of the Marketing Research and Intelligence Association June 11, 2008 the Ottawa Chapter of the Marketing Research and Intelligence Association June 11, 2009 Presentation to the MRIA Ottawa Chapter June 11, 2009 Randa Bell ASDE Survey Sampler, Inc. We get what you need

2. Current RDD sample does not include cell phones

3. Residential Telephone Service Survey RTSS The RTSS monitors residential phone penetration rates and reasons for non-subscribing Since 2005, the RTSS has been conducted in December only The collection period is the Labour Force Survey (LFS) interview week

4. Household composition of the 6.3% cell only households in Canada *Source: Statistics Canada: Residential Telephone Service Survey (RTSS) Dec 2007

5. Household composition of the 6.3% cell only households in Canada Source: Statistics Canada: Residential Telephone Service Survey (RTSS) Dec 2007

6. Using IVR to screen for cell only households Programmed and recorded survey with one question only Nature of cell phone we were reaching: Cell only household or in addition to landline Simplified wording

7. The questionnaire 20 seconds, National Study Only one question, nothing to sell Phone line which is not cell phone = PRESS 1 Cell phone only household = PRESS 2 Reimbursement of call = PRESS 3 Repeat question = PRESS 9

8. Two days dialing results of using IVR to screen for cell only households

9. Ratio of cell + landline vs. Cell onlyAmong participants

10. BBM inclusion of cell phones onlyhouseholds (MO) Assess the viability of recruiting into the TV panel Recruiting a subset from the establishment survey to participate in phase 2

11. Household Technology Inclusion of Mobile-Only Households in Canadian Radio & Television Ratings PanelsApril 2009 *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

12. BBM Canada Household Technology • BBM Canada, Sondages BBM in Québec: not for profit, member-owned tripartite industry organization, operating since 1944 • Provide broadcast measurement and consumer behaviour data, as well as industry-leading intelligence to broadcasters, advertisers and agencies • Services include a digital broadcast-ready TV & Radio people meter system, diary surveys for 100+ radio and television markets, and a variety of syndicated and custom research studies *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

13. Panel Establishment Household Technology • Establishment survey conducted monthly (except Dec, Jul and Aug) for each panel • Large scale telephone survey • Establish sample frame (households) for recruitment into Panel *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

14. Mobile-Only Test Household Technology • To overcome under-coverage in sample frame, the addition of mobile-only homes was tested in two phases • Existing Panel Establishment survey to assess participation by mobile-only homes • First three weeks of May, 2008, concurrent with the regular May Establishment • National sample of 4,316 mobile-only homes provided by ASDE • Safety assured at beginning of call. A few questions regarding mobile use were added to survey. • All respondents, regardless of completion of survey, were offered $5 as compensation for air time if address provided. *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

15. Establishment Survey Response Metrics – Cell-Only Households Nationally Establishment Survey Response Metrics – Cell-Only Households Nationally

16. Establishment Survey Response Metrics – Total Quebec Establishment Survey Response Metrics – Total Quebec Significant at p<0.05

17. Establishment Survey Response Metrics – Montreal EM Establishment Survey Response Metrics – Montreal EM Significant at p<0.05

18. Establishment Survey Response Metrics – Quebec Balance Establishment Survey Response Metrics – Quebec Balance Significant at p<0.05

19. Household and Respondent Data Household and Respondent Data *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

20. Household Technology Household Technology *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

21. Other Points of Interest Household Technology • Respondents did not call back to call centre based on • number on call display (as experienced in US) • Only 1% of respondents did not live in same geography • as area code indicated • 88% indicated they were contacted in the city • in which they live • Reimbursement for air time did not appear to be factor. • Very few (less than 1%) of non-completes • requested reimbursement *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey

22. Harris-Decima Benchmark study • Numbers were randomly generated for Canada • The sample was to be used as a benchmark → Wanted to analyze: Age composition Male/Female composition Cost and performance Cooperation rates We get what you need

23. No changes to the standard screener no special accommodations for providing incentives because they are reaching people on their cell phones • 2 separate call dispositions. We get what you need

24. At first glance, it looks like the cell phone sample performed considerably worse than the RDD sample 4.6 times the ratio of sample to complete We get what you need

25. Comparing the disposition of all numbers used • RDD sample vs. cell phone sample 20% of the RDD VS. 55% of the cell sample was NIS sample was NIS The use of a dialer means that the extra numbers cost time on the dialer and not on interviewing costs. We get what you need

26. The next step was to exclude the NIS numbers from the call disposition Ratio of sample to completes dropped by 56% for the cell sample and by 20% for the RDD sample • No answer/ Answering Machine/ Busy was 65.2% for cell numbers and 51.8% for RDD sample. • Refusal rate on cell phone was 25.7% and on RDD was 34% We get what you need

27. Benefits Looking at the actual cell phone sample completes: 41.1%58.9% 18-34: 53.4% 35-54: 31.5% 55+: 9.6% We get what you need

28. Conclusion Preliminary results show that people are just as responsive to being called on their cell phone if not more than on their home line One hypothesis could be the novelty of getting a call on their cell phones. We get what you need

29. Futuremoving on Focus on testing for various scenarios: Response rate for various length surveys Incidence of cell only and cell mostly Incentives vs. reimbursement Respondent experience

30. To include or not to include With cell only households at 6.3% Including them in a national sample may not substantially affect survey findings BUT It does improve the overall representativeness of the sample Reduces sampling error for these low incidence groups

31. Other considerations Under 18 with cell phones Reimbursement Safety and scheduling for call backs Geographical location vs. area code location Quality of responses, better or worse

32. How to incorporate cell only households sample into RDD Blended frame sampling Portion from cell phones only households Portion from RDD Augment traditional sample with cell phone numbers (no overlap) Dual Frame sampling Portion from cell phones Portion from RDD Qualify if overlap

33. Thank you Randa Bell rbell@surveysampler.com www.surveysampler.com 819-770-3651 Sorry no cell number provided