1 / 40

What works? Getting the General Population To Go Online in a Mixed Mode Local Health Survey

What works? Getting the General Population To Go Online in a Mixed Mode Local Health Survey. Louis-Robert Frigault Sadoune Ait Kaci Azzou Evi Jane Kay Molloy Fatima Ammarguellat Maude Couture Jean Gratton Montreal Public Health Department

felcia
Télécharger la présentation

What works? Getting the General Population To Go Online in a Mixed Mode Local Health Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What works? Getting the General Population To Go Online in a Mixed Mode Local Health Survey Louis-Robert Frigault Sadoune Ait Kaci Azzou Evi Jane Kay Molloy Fatima Ammarguellat Maude Couture Jean Gratton Montreal Public HealthDepartment Agence de la santé et des services sociaux de Montréal Web surveys for the general population: How, why and when? Workshop 6-7 june 2013 University of Essex, Colchester

  2. Montreal Public Health Surveillance Legal mandate – Public Healht Act Surveillance of health status and risk factors for 12 Local Health Units (LHU) Surveys to complement other administrative data

  3. Quebec Health System structure Provincial level – 16 regions Regional – Montreal Island (1.9M) Local - 12 Local Health Units (LHU)

  4. Survey challenges Improve or maintain response rates Ensure sample representativity at the local level Minimize measurement biases Reduce costs

  5. Our first steps … Mixed mode pilot studywith Statistics Quebec in 2011 Establish feasibility Estimate fieldwork effort Evaluate data quality and comparability

  6. TOPO 2012 Study population Personsaged 15 yearsold and older, living in Montreal in privatehouseholds and registeredwith the QuebecHealthInsurance Plan Objectives Guide local public health planning + baseline for surveillance Producereliable and precise (CV ≤ 15%) estimatesat the local level (12 LHU) for healthindicators of a minimum prevalence of 5% 900 respondents per local healthunits

  7. TOPO 2012 Stratified probability sample drawn from the the Quebec Health Insurance Registry (95% population coverage) 168 strata : 2 sex – 7 age groups– 12 local health units Full name, age, sex, complete postal address, day and night telephone numbers (70%), language preference, name of address holder, if different, linkage number

  8. Questionnaire Topic: Chronicdiseases + determinants Length: max. 109 questions – 23 minutes Validated questions (ESCC, EQSP, DSP) • French and English / CAWI and CATI • Adapted for a mixed mode survey • Answerchoicewereread in CATI whenlisted in CAWI • Use same question sequence (i.e. no grid)

  9. Inciting participation • Branding exercice • Name: TOPO • Logos and key art for website, letterhead, bulletins, etc. • Media relations – press kit – press release – interviews • Social media strategy • YouTubevideos • Facebook campaign • Google AdWords • Montreal Public Health Facebook and Twitteraccounts • Comprehensivewebsite : topomtl.ca • Bulletins – six issues duringfieldwork

  10. Fieldwork

  11. topomtl.ca

  12. Sampling

  13. Invitation letter • Personnalized • Survey purpose & content • Confidentiality • Unique access code • Website address • Tel. # of survey company • Advance letter to parentsof 15-16-17 y.o.

  14. Cummulative impact of invitation letters + reminders

  15. Facebook campaign

  16. Results

  17. Final results

  18. Response rates by LHU

  19. Web participation by LHU Global: 56% Range: 48% to 63%

  20. Sex – Unweighted results

  21. Age groups – Unweighted results

  22. Language– Unweighted results

  23. Immigrants – Unweighted results

  24. Materialdeprivation Index Unweightedresults

  25. Incomplete questionnaires Analyzed by sections – not onlyat « break-off » Indicators • % of completion: 75% • Time required to complete: detailedanalysis of questionnaires completed in 11 minutes or less • Number of connexions required to complete • last connextioncannotbe the onlyindicator 25 rejected questionnaires

  26. Validation of respondent’sidentity Age and sex of respondentswascomparedwith the information contained in sample frame database 114 rejected questionnaires

  27. Item non reponse Item by item analysis Generally low : inferior to 5% Except revenu: 24% (21% web – 28% tel)

  28. Non response and weighting Non response adjustments based on info from sample frame database • age • sex • place of residence (local public health unit) • day or night night phone (yes or no) in database • address holder name (yes or no) in database • preferred language for correspondence (French or English) CHAID – Chi-Square Automated Interaction Detection Post-stratification based on census data • Age, sexe, CSSS population projections based on Census data

  29. Quality of indicators

  30. Results on health indicators Limited mode effect on mostindicators • Adjustments: age, educationlevel Combined (web + tel) results comparable to othersurvey sources

  31. What works? And what does not?

  32. Access to a detailedsample frame • Registry information useful for personnalized invitations and reminders, but also for non responseadjustments Branding and communication • Unifiedbrandinglikelyhelped the survey, but no actual test withoutbranding • Social media was not effective in the context of the study – Facebook campain made lots of noise • BuyingAdwordswas a good investment

  33. Website content • Keepit simple • Study objectives clearlystated • FAQ +/- • Make «Participate » buttonalwaysvisible

  34. Questionnaire • Keep it short • Think like Mad Men: push limits of survey tool • Avoid matrix questions in web version

  35. Data collection - fieldwork Invitation letters • Shouldbe official – letterhead - logos • Shouldconsider the diversity of audience • Should focus on concrete objectives • Offer simple instructions for participating Reminders • Web focus telephonereminders are effective and require minimum effort • Telephone focus reminders are effective with people not likely to complete on the web • Postal and e-mail reminders have limited impact on response rates • 48% off total respondentsparticipated in first 3 weeks of eachwaves

  36. Data collection - fieldwork Active telephone interviews • Startingweek 4 of eachwaves, main objective is to completesamplewithtelephone interviews • Possiblyalreadyin refusal conversion mode Complexpre-programmedinterviewer scripts • Multi modes • Multi language • Multi reminders • Waves • Wave 3 too short to acheive optimal response rates

  37. Global results • Did not contribute to higherresponse rates • Didcontribute to betterrepresentativity • Low item non response • Heavy validation • Good quality and comparability of main healthindicators • Costreduction: 33%

  38. Merci! Questions?

  39. What works? Getting the General Population To Go Online in a Mixed Mode Local Health Survey Louis-Robert Frigault et al. lrfrigau@santepub-mtl.qc.ca Montreal Public HealthDepartment Agence de la santé et des services sociaux de Montréal Web surveys for the general population: How, why and when? Workshop – 6-7 june 2013 University of Essex, Colchester

More Related