Fieldwork logistics and data quality control procedures

1. Fieldwork logistics and data quality control procedures Kathleen Beegle Workshop 17, Session 2 Designing and Implementing Household Surveys March 31, 2009

2. What makes a good survey? • Relevance • Answer the policy questions of import • Quality • Methodology • Accuracy and reliability • Adherence to international statistical guidelines • Timeliness • Punctuality • Production time • Comparability over time • Sustainability • Public dissemination • Documentation

3. Overview • Overview of the main actors • Quick description of broad activities of an LSMS • Options for organization of field work & quality control aspects

4. Who does what • Core team • Principal Investigators who understand the purpose of the survey • Local team • Will implement the survey and process data • For LSMS is usually the national statistics office • For smaller/specialized surveys, usually local firms • Need to identify who in the local team is the project manager • Technical assistance • Short-term consultants for specialized expertise such as sample design and data entry system development • Long-term assistance: possible project head if lacking in local team

5. Activities • Broad activities involved with an LSMS • Management/logistics • Questionnaire development • Sampling • Field staff: recruitment, training, payments • Field work • Data management • Data analysis and final documentation • Some activities are sequential (training after questionnaire finalization), but lots of overlap in timing (sampling & questionnaire development; logistics is on-going).

6. Design Data collection Analysis Management: Prepare a timeline…

7. ….and make it detailed, with deadlines!

8. Questionnaire development • Pilot/pre-test questionnaires in various settings as relevant (urban, rural, etc) • Careful review of translation (if any). Back translation is ideal. • Finalize questionnaire before any training. • Prepare Manuals with specific instructions for enumerators and supervisors, other staff, to accompany the questionnaire. This is also part of the final documentation to assist data users.

9. Field staff • Recruit & train more enumerators than you need • Enables a selection process during training – competition among candidates • Provides reserves in case field staff leave early • Training in two parts (if large field staff) • Training of Trainers (TOT): train field supervisors • Training of enumerators: main training with active participation of supervisors as facilitators

10. Field staff • Training for an LSMS – a long, multi-topic questionnaire – 3-4 weeks. Includes: • in-class study of questionnaires • in-class mock interviews/exercises • tests • field practice and feedback • more if the survey will use special instruments (anthropometric, GPS, etc.) • Common training for all team members if feasible • Payments • Usually not paid per questionnaire, instead paid daily rate • Bonuses paid at end of field work can reduce staff attrition during field work (important for longer surveys) • Performance bonuses (if they can be objectively determined)

11. Field work • There are many options for field work structure. Your selection will depend on the specifics of the survey, timeline, and the budget: • Mobile teams with integrated DE. Improves quality of data because teams get immediate feedback on inconsistencies in the questionnaire. • Mobile teams without DE. DE in office. • Enumerator in the village. DE in office. Difficult to supervise well! May not be cost effective (with exception of surveys over several moths with ~20 HHs per village). • Local office from which teams operate. Feasible for a survey focused in one region/small geographic area. • No excuse not to do concurrent DE! Don’t wait until end of field work to start entering questionnaires. This is often the reason for lags in data availability.

12. Data entry operator .maybe. Anthropo-metrist Supervisor Interviewers Field work: Composition of a mobile field team

15. Other factors of survey quality • Management Team for the survey • Composed of • Project Manager • Fieldwork Manager • Data Manger • Dedicated to the project in all three phases • Design and Preparation • Implementation • Dataset documentation and initial tabulations

16. Other factors of survey quality • Actions of the field team supervisors • Visual scrutiny of completed questionnaires • Less critical if DE is integrated with field work. Computers do it better • Visual observation (direct observation) of interviews • Some of this is needed, but not too much • Check-up visits • Critical, and can only be implemented by human supervisors • Need to be random • Supervisors must be supervised too!!

17. Other factors of survey quality • Review data files as they come in – done by Core staff (i.e. in Stata, look for inconsistencies across modules of the questionnaire which is normally not done in DE programs, compare with baseline data if a panel, etc…) • Send Addendum notes/instructions to field staff as issues arise (for things missing from the Manuals) • Keep detailed documentation, needed for analysis and relevant for panel surveys

18. Aspects for evaluation work • If the evaluation entails panel data (baseline and follow-up surveys), plan surveys accordingly. • At baseline, think about how to minimize attrition and match people correctly • get adequate address/map/phone information to locate households again • Record complete names of all household members on questionnaire and enter in DE • Other possibilities: GPS, photos of respondents • At follow-up • pre-print some baseline data (Roster: names, age, sex, roster IDs) and also baseline information relevant to the evaluation for measuring changes (assets?) • collect some minimal information on respondents/households who drop out or move (in case you are unable to find them for a re-interview). For example: did they move, when, why…