Use of Incentives in Surveys Supported by Federal Grants

1. Use of Incentives in SurveysSupported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December 9, 2008

2. Overview • What we know about incentives from the survey methods literature • IRB issues in providing incentives • Results of a survey of NIH grant recipients who planned to do surveys • Conclusions

3. Survey Literature Findings on Use of Financial Incentives • Incentives improve response across modes • Effects appear more or less linear, size matters • Prepaid incentives more effective than promised • Lotteries less effective than payments • Lower income respondents more responsive than higher income respondents • Incentives can help increase representativeness of the sample

4. How Financial Incentives Fit Into Survey Response • Respondents consider “costs” of survey participation • Time, inconvenience • Loss of privacy, concerns about other possible consequences • How it will feel to be interviewed or complete the survey • How others might view them for taking part • Whether the purpose of the survey is a potential loss for them

5. Non-Financial Reasons for Survey Participation • Possible “benefits” other than payment • Belief that the survey will be beneficial to them or to others, desire to help • Interest in the survey, desire to talk about the topic • Helping or being associated with a survey sponsor • Belief that participating in the survey will be pleasant • Desire to help and/or talk with interviewer • Prestige from being a study participant

6. Institutional Review Boards • All federally supported surveys should be reviewed by an IRB • IRBs operate under guidelines from DHHS Office for Human Research Protections • Requirements for approval: • Risks are minimized • Subject selection is equitable, vulnerable populations protected • Informed consent • Protection for subjects’ privacy and data confidentiality • Each IRB can and does interpret guidelines in light of local circumstances

7. IRBs Wrestle With Incentives • Do they “coerce” people who would not otherwise freely agree to participate in research to do so? • How should the level of incentives be determined? • As “wages” for time and effort of participation? • Based on the value researchers place on participation? • Equitably for all participants or recognizing different values participants may place on their effort and time?

8. Special Problems with Incentives for IRBs • Lotteries • Provide unequal rewards across participants • Undermine informed decision since chances of winning are overvalued • Disadvantaged populations • Especially vulnerable to coercion • May be induced to lie or conceal information in order to participate • Higher payments for people who refuse to participate

9. Survey Literature on Incentives Does Not Address Ethical Issues • Focus is on practical concern with effectiveness in terms of response rates and/or MSE • “AAPOR Best Practices” suggests considering use of incentives to stimulate cooperation • Ability to enhance participation of reluctant respondents and under-represented groups is a plus • Not part of AAPOR disclosure requirements • Not addressed in AAPOR advice on dealing with IRBs

10. Web Survey of NIH Grantees Doing Surveys • Sampling frame: NIH CRISP database of grants • Lists about 250 in each year • 482 unique grants in 2004-2006 • Reviewed abstracts: • Included those with mention of survey data collection • Excluded methodological studies • 145 grants selected for data collection

11. Incentives Web Survey • Sent e-mail invitation to participate and three follow ups to Principal Investigator listed in CRISP • Provided link to COPAFS and Incentives Conference web pages • Promised copy of the paper as “incentive” • Allowed PI or designated proxy to fill out the survey • Asked PI to select one survey, the “most important” in terms of research goals • Received 92 responses - 63%

12. Most Surveys Used Incentives

13. Why Didn’t Surveys Use Incentives? • Shorter surveys • 40% under 15 minutes, nearly all under an hour • Mean time to complete 27 minutes vs. 45 minutes • Reasons: • 60% expected good response without incentives • 60% did not have budget for incentives • 27% survey team did not want to pay incentives • None reported IRB was a factor

14. How Did Surveys Use Incentives? • Mix of expectations and timing • 44% for completed survey • 32% for partial complete • 13% for considering participation • 11% prepaid incentive • About half the surveys included tasks other than the survey • Half of those provided a separate payment

15. Kinds and Amounts of Incentives • Kinds of Incentives • Cash - 31% • Gift cards or certificates - 27% • Checks - 25% • Amounts of monetary incentives • $10 or less - 34% • $20-49 - 48% • $50 or more - 14% • Other kinds of incentives • Lottery for an iPod or gift certificates, mugs, bags, water bottles, etc.

16. Why Are Incentives Used? • 73% said to increase response rates was main reason • Rated as main or very important reason: • Reduce non-response bias - 71% • Reward participants for research participation - 56% • Not rated as important reasons: • Reduce data collection time or follow up costs • Help interviewers feel more comfortable • IRB wanted incentives

17. What Kinds of Surveys Were These? • Potentially Sensitive Topics • Health status or health conditions - 74% • Personal financial information - 44% • Sexual behavior - 31% • Drug use or drug use history - 25% • Immigration status - 14% • Special requests • Linkage to other databases (e.g. Medicare) 7%

18. Modes Used With Incentives

19. How Were Amounts Determined? Based on open-ended comments: • Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone charges • Time and contribution of personal information • Going rate for surveys of this kind of subject • Accounting or safety issues for interviewers or subjects or budget issues as constraints • Experiments to determine effective amounts

20. What Was the Role of IRBs? • For paying incentives or not: • Only one participant cited IRB preferences as a main reason for paying incentives • 88% said IRB preferences were not important in making this decision • For kind or amount of incentives: • 23% said IRBs raised questions or placed limitations • IRBs capped amount or required same incentive for all • Limited how and when incentives could be mentioned

21. Conclusions • At least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$50 • 75% of telephone surveys paid incentives, range was $15-20 • Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requests • Most paid incentives to increase response rates • IRBs were a factor, but not a major limitation