1 / 22

Use of Incentives in Surveys Supported by Federal Grants

Use of Incentives in Surveys Supported by Federal Grants. Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December 9, 2008. Overview. What we know about incentives from the survey methods literature IRB issues in providing incentives

isaiah
Télécharger la présentation

Use of Incentives in Surveys Supported by Federal Grants

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Use of Incentives in SurveysSupported by Federal Grants Sandra H. Berry, Jennifer S. Pevar, and Megan Zander-Cotugno CDC PRAMS National Meeting December 9, 2008

  2. Overview • What we know about incentives from the survey methods literature • IRB issues in providing incentives • Results of a survey of NIH grant recipients who planned to do surveys • Conclusions

  3. Survey Literature Findings on Use of Financial Incentives • Incentives improve response across modes • Effects appear more or less linear, size matters • Prepaid incentives more effective than promised • Lotteries less effective than payments • Lower income respondents more responsive than higher income respondents • Incentives can help increase representativeness of the sample

  4. How Financial Incentives Fit Into Survey Response • Respondents consider “costs” of survey participation • Time, inconvenience • Loss of privacy, concerns about other possible consequences • How it will feel to be interviewed or complete the survey • How others might view them for taking part • Whether the purpose of the survey is a potential loss for them

  5. Non-Financial Reasons for Survey Participation • Possible “benefits” other than payment • Belief that the survey will be beneficial to them or to others, desire to help • Interest in the survey, desire to talk about the topic • Helping or being associated with a survey sponsor • Belief that participating in the survey will be pleasant • Desire to help and/or talk with interviewer • Prestige from being a study participant

  6. Institutional Review Boards • All federally supported surveys should be reviewed by an IRB • IRBs operate under guidelines from DHHS Office for Human Research Protections • Requirements for approval: • Risks are minimized • Subject selection is equitable, vulnerable populations protected • Informed consent • Protection for subjects’ privacy and data confidentiality • Each IRB can and does interpret guidelines in light of local circumstances

  7. IRBs Wrestle With Incentives • Do they “coerce” people who would not otherwise freely agree to participate in research to do so? • How should the level of incentives be determined? • As “wages” for time and effort of participation? • Based on the value researchers place on participation? • Equitably for all participants or recognizing different values participants may place on their effort and time?

  8. Special Problems with Incentives for IRBs • Lotteries • Provide unequal rewards across participants • Undermine informed decision since chances of winning are overvalued • Disadvantaged populations • Especially vulnerable to coercion • May be induced to lie or conceal information in order to participate • Higher payments for people who refuse to participate

  9. Survey Literature on Incentives Does Not Address Ethical Issues • Focus is on practical concern with effectiveness in terms of response rates and/or MSE • “AAPOR Best Practices” suggests considering use of incentives to stimulate cooperation • Ability to enhance participation of reluctant respondents and under-represented groups is a plus • Not part of AAPOR disclosure requirements • Not addressed in AAPOR advice on dealing with IRBs

  10. Web Survey of NIH Grantees Doing Surveys • Sampling frame: NIH CRISP database of grants • Lists about 250 in each year • 482 unique grants in 2004-2006 • Reviewed abstracts: • Included those with mention of survey data collection • Excluded methodological studies • 145 grants selected for data collection

  11. Incentives Web Survey • Sent e-mail invitation to participate and three follow ups to Principal Investigator listed in CRISP • Provided link to COPAFS and Incentives Conference web pages • Promised copy of the paper as “incentive” • Allowed PI or designated proxy to fill out the survey • Asked PI to select one survey, the “most important” in terms of research goals • Received 92 responses - 63%

  12. Most Surveys Used Incentives

  13. Why Didn’t Surveys Use Incentives? • Shorter surveys • 40% under 15 minutes, nearly all under an hour • Mean time to complete 27 minutes vs. 45 minutes • Reasons: • 60% expected good response without incentives • 60% did not have budget for incentives • 27% survey team did not want to pay incentives • None reported IRB was a factor

  14. How Did Surveys Use Incentives? • Mix of expectations and timing • 44% for completed survey • 32% for partial complete • 13% for considering participation • 11% prepaid incentive • About half the surveys included tasks other than the survey • Half of those provided a separate payment

  15. Kinds and Amounts of Incentives • Kinds of Incentives • Cash - 31% • Gift cards or certificates - 27% • Checks - 25% • Amounts of monetary incentives • $10 or less - 34% • $20-49 - 48% • $50 or more - 14% • Other kinds of incentives • Lottery for an iPod or gift certificates, mugs, bags, water bottles, etc.

  16. Why Are Incentives Used? • 73% said to increase response rates was main reason • Rated as main or very important reason: • Reduce non-response bias - 71% • Reward participants for research participation - 56% • Not rated as important reasons: • Reduce data collection time or follow up costs • Help interviewers feel more comfortable • IRB wanted incentives

  17. What Kinds of Surveys Were These? • Potentially Sensitive Topics • Health status or health conditions - 74% • Personal financial information - 44% • Sexual behavior - 31% • Drug use or drug use history - 25% • Immigration status - 14% • Special requests • Linkage to other databases (e.g. Medicare) 7%

  18. Modes Used With Incentives

  19. How Were Amounts Determined? Based on open-ended comments: • Reflected actual costs of participation, e.g. travel, lost wages, child care, cell phone charges • Time and contribution of personal information • Going rate for surveys of this kind of subject • Accounting or safety issues for interviewers or subjects or budget issues as constraints • Experiments to determine effective amounts

  20. What Was the Role of IRBs? • For paying incentives or not: • Only one participant cited IRB preferences as a main reason for paying incentives • 88% said IRB preferences were not important in making this decision • For kind or amount of incentives: • 23% said IRBs raised questions or placed limitations • IRBs capped amount or required same incentive for all • Limited how and when incentives could be mentioned

  21. Conclusions • At least half the NIH sponsored projects that used surveys paid incentives that were substantial - $10-$50 • 75% of telephone surveys paid incentives, range was $15-20 • Surveys that paid incentives were longer (mean=45 minutes), involved sensitive topics, and often included additional kinds of data collection or other requests • Most paid incentives to increase response rates • IRBs were a factor, but not a major limitation

More Related