Multiple Indicator Cluster Surveys Survey Design Workshop

1. MICS Survey Design Workshop Multiple Indicator Cluster SurveysSurvey Design Workshop Data Entry and Processing

2. Content of the Presentation • Overview of the MICS data processing system • Data processing using paper questionnaires • Main characteristic of the MICS CAPI system • Creating analysis files • Data archiving

3. Content of the Presentation • Overview of the MICS data processing system

4. MICS Data Processing System: Actors and Roles • Country data processing manager and country team: • Customization of data entry programs, data entry, editing, and production of datasets • Customization of tabulation syntaxes and tabulation • Regional Office MICS Coordinator • Coordination and supervision, organization of the Data Processing workshop • Regional Office Data Processing Consultant • Technical support and review of customized programs and close work with country teams • HQ Data processing unit • Development of standard programs, templates and coordination of Data Processing workshops

5. MICS Data Processing System • The data-processing system can be divided into following phases: • Customization of MICS data entry/collection program and tabulation syntaxes, • Establishing the data entry system locally, • Primary data processing (data entry/data collection), • Secondary data processing (creating analysis files), and • Tabulation.

6. Content of the Presentation • Overview of the MICS data processing system • Data processing using paper questionnaires

7. MICS Data Processing System • Designed to deliver the first results of a survey within several weeks after the end of fieldwork • Such rapid turnaround time is possible when completed questionnaires are entered simultaneously with survey fieldwork • Data for each cluster is stored in a separate data file and is processed as soon as all the questionnaires from a cluster are returned from the field • This approach breaks data processing down into discrete segments and allows it to progress while fieldwork is ongoing • By the time the last questionnaires are finished and returned to headquarters, most of the data have already been processed

8. Customization of data entry files • Adapting MICS standard data entry program files to the country-specific questionnaire • Second Regional Workshop [Data Processing] • Adapting MICS standard tabulation syntax files • Customisation of the programs during and immediately after the workshop

9. Establishing the data entry system • The goal is to be ready to begin data entry shortly (about one to two weeks later) after the fieldwork commences • The preparation phase involves the following steps: • Obtaining computer equipment and setting up a data-processing room • Identifying and recruiting appropriate personnel so that they participate to fieldwork personnel training • Setting up a system for managing the questionnaires and data files

10. Computers • Operating system: Microsoft Windows XP, Vista, Windows 7, or Windows 8. • Computers adequate to run Windows XP or better versions • Adequate hard disk space (265 megabytes memory and at least 70 megabytes of free disk space) • Adequate memory • Network • Printer

11. Software • CSPro • Windows software tool for census and survey processing • Developed jointly by IPC-US Census Bureau, Macro International, SerPro • Public domain • Free download at • www.cspro.org or • www.census.gov/ipc/www/cspro/index.html

12. CSPro • Modules • Data entry • CAPI Data collection: laptops/tablets (PDAs v4.1) • Data editing and imputation • Frequencies and crosstabulations • Data manipulation utilities • Data dissemination • Export to statistical packages

13. CSPro • Data Entry • Interactive • Range, skip, consistency checks • Structure check • Ensures completeness of data • Verification • 100% double entry • Secondary editing • Complex consistency checks • Export to SPSS

14. Primary Data Processing • The goal of primary data processing is to produce clean, edited data files. Primary data processing involves the following steps: • Entering all questionnaires for a cluster onto a data file • Production of field check tables • Checking the structure of the data file • Entering the data a second time and then verifying the data file • Backing up the checked and verified data file • Performing secondary editing on the data file • Backing up the edited, or final, data file

15. Dataprocessingrequirements • Computers • 1 per data entry operator • Plus 1 for supervisor • Arrange day/night shifts if workload is heavy • Staff • Data processing supervisor (1 or 2) • Data entry operators (variable) • Can enter about 1 cluster per day • Data editors (1 or 2) • 6-7 hours per day including breaks

16. Flow of data entry

18. Content of the Presentation • Overview of the MICS data processing system • Data processing using paper questionnaires • Main characteristic of the MICS CAPI system

19. Mobile data collection • In the recent years we have seen development of many innovative data collection tools using handheld PocketPC personal digital assistants (PDAs), smartphones or tablet personal computers. 

20. Tablet Personal Computer • Tablet – sized computer with the key features of a full – size personal computer • Various operating systems • Computer vs. cell phone capability

21. Personal Digital Assistants • Handheld computers • Various operating systems • Windows mobile • Palm • Others (e.g. iPhone, Nokia) • Cell phone capability • Plus: good communications • Minus: security

22. Tablet vs PDA • Choosing a tablet over a PDA • Tablets have much larger screens, better resolutions, more space for on screen typing making things easier to see and more useful for data entry • Tablets and PDAs have similar battery life, though if PDAs are used for making and receiving calls battery will need more frequent charging • Better security

23. Software requirements • MICS survey requirements • Keep track of the “data path” • Multiple languages on the fly • Roster manipulation • Saving an interview in the middle • Dynamically customized questions and response categories • Powerful programming language • Able to handle complex consistency checks and skips • Ability to run the same application in PDAs, tablets or PCs • Capability to administer instruments at different times • Able to handle long/complex questionnaires

24. Hardware requirements for CSPro Tablets: • Required configuration: Full Microsoft Windows  7 or 8 (NO windows RT tablets) PDAs: • Required configuration: Windows Mobile versions 5 and 6 (note that UNICODE is not supported)

25. MICS CAPI System • All applications to collect and administer data are written in CSPro • Three systems: • Interviewers - data collection – Tablet/PDA • Supervisors – data monitoring and control –Tablet/PDA • Central Office - centralized data and monitor fieldwork - PC

26. Int 5 Int 2 Repair Utility Interviewer’s System Update Utility Supervisor’s System Screen Reports Supervisor Supervisor Reports Central Office System System Updates System Updates Central Office

27. Questionnaires • One question per screen • Question area • Answer area • (for text or numeric responses) • List of categories (where appropriate)

28. Customizable questions • Multiple languages • Language can be changed during interviewing • Questions customized to the individual • Response categories also customizable • Radio buttons for categorical variables • Check boxes for multiple response variables

29. A Typical Screen Questions Area Display/enter values Possible Responses Navigation/tools

30. Virtual Keyboard Values in ranges or textual text needs to be typed Soft Keyboard

31. Dynamically Generate Answers Colors have meanings Value Sets populated

32. Benefits • Improved data quality? • Pro – checking of data in real time during interview • Con – possibility for interviewer’s to ‘avoid’ work • Overall – positive improvements, but only with good quality controls in place • Eliminates missing information • Shortens interviews • Data ready almost immediately • Survey indicators monitored early in fieldwork

33. Equipment • Power - battery charging is an issue • Tablets/PDAs generally give close to a full day of work • May require charging or swapping batteries during the day • Spare battery needed • Separate charger • Need for extra tablets/PDAs • Needed for all trainees (not just number of interviewers/supervisors planned for fieldwork), plus IT staff • In case of problems/loss in the field • Cases/covers • Good protection needed depending on expected conditions

34. Equipment • Security issues • Theft of Tablets/PDAs is a risk • Increased risk if PDAs are also cell phones (smartphones) • Virus risk • Tablet/PDA viruses from downloaded programs • PC viruses infecting SD cards (e.g. when SD cards are used in PCs in Internet cafés) • Cell phones to communicate with field teams • Built in if PDA is smartphone

35. Data loss risk • Data loss risk • Lost/stolen PDAs/Tablets • Hardware failure • Viruses • User error • Risk management • Backup to SD card immediately after interview • Copy to supervisor at end of day • Paper questionnaires can be used in emergency

36. PDA Costs • Basic Tablet/PDAs cost approx $450 with no accessories • Plus approx $100 in needed accessories: • SD card • Spare battery • Case • Screen protector • Spare stylus • Vehicle chargers (1 per team)

37. Costs • Costs balanced out by • Printing of only 10% of questionnaires • No field editors • No data entry • No computers required for data entry • However • Technical assistance costs are higher! • Cost savings • if Tablets/PDAs are re-used • Savings on secondary processing, data editing

38. Technical assistance • Significant extra programming compared with classic data entry • Questionnaire programming is more complicated • Addition of question text • More customized response categories • More complicated flow to facilitate interviewing • Possible need for multiple languages • Data management programming much more complicated (even than questionnaires) • Three systems (interviewer’s, supervisor’s, central) compared with two (data entry, data coordinator) • Logistics of data management much more complicated

39. Preparation and Pretesting • Early design of questionnaires needed • Last minute changes to questionnaires are difficult • Programs prepared well in advance and thoroughly pretested • Full pretest required to properly test all aspects of the system • Programs must be ready and thoroughly tested before training begins • Updating applications during training is problematic • Making the change is easy, but • Preparing update for all Tablets/PDAs takes time • Updating all Tablets/PDAs is very time consuming

40. Training • Paper questionnaires still important • Discussions and version control • Documentation • Training • Facility for training (power, desks) • Good documentation • Train on paper first, then Tablet/PDA • Gives the ‘big picture’ • Paper can also be used in emergency in the field • Fieldwork testing very important • Recommended – 4 week training, • Including 4-5 separate days of field practice • Extensive practice during training is needed

41. Data transfer • Logistics of transferring data to central office • Cell phone • Only if PDA is smartphone • Wireless email • Limited Wifi locations • Internet cafés • Virus risk on PCs • Field control teams • Via bluetooth or SD cards • More reliable, but slower

42. Field Staff • Interviewers should be computer literate • Younger interviewers who are comfortable with cell phones quickly learn to use PDAs • Interviewers need an introduction to computers and to PDAs • Supervisors need to be very comfortable with applications • Balance between good interview experience and good computer knowledge

43. DP staff • Several local data processing staff needed to troubleshoot in the field • User errors in using the system • Program malfunctions – usually related to user error • Lost data problems – usually found on the backups • PDA viruses • PC viruses on SD cards • Frozen PDAs • Problems transferring data • Problems related to data inconsistencies that cannot be resolved by the team • Upgrading programs • Central office staff for secondary processing

44. Conclusions • Quicker, better results are possible • Many complications • Costs likely higher • Logistics much more complicated • Technical assistance needs increased • Don't underestimate how much work it takes to properly implement a survey, even with Tablets or PDAs. • It is very easy to conduct a poor survey

45. MICS Pilot experience:Duration of interview • Household questionnaire • 33 minutes to administer paper household questionnaire. • 19 minutes to administer PDA household questionnaire. • Women questionnaire • 15 minutes to administer paper women’s questionnaire. • 12 minutes to administer PDA women’s questionnaire. • Men questionnaire • 15 minutes to administer paper men’s questionnaire. • 9 minutes to administer PDA men’s questionnaire. • Under 5 questionnaire • 18 minutes to administer paper U5 questionnaire. • 10 minutes to administer PDA U5 questionnaire. • Important to add around 5-10 minutes needed for data transfer after completing household questionnaire

46. MICS Pilot experience:Duration of interview • Household questionnaire • 33 minutes to administer paper household questionnaire. • 19 minutes to administer PDA household questionnaire. • Women questionnaire • 15 minutes to administer paper women’s questionnaire. • 12 minutes to administer PDA women’s questionnaire. • Men questionnaire • 15 minutes to administer paper men’s questionnaire. • 9 minutes to administer PDA men’s questionnaire. • Under 5 questionnaire • 18 minutes to administer paper U5 questionnaire. • 10 minutes to administer PDA U5 questionnaire. • Important to add around 5-10 minutes needed for data transfer after completing household questionnaire.

47. Content of the Presentation • Overview of the MICS data processing system • Data processing using paper questionnaires • Main characteristic of the MICS CAPI system • Creating analysis files

48. Secondary Data Processing • The goal of secondary data processing is to produce analysis data files and to create the MICS standard tables. Secondary data processing involves the following steps: • Bringing together all cluster data files into one data file • Exporting the data to the SPSS software • Recoding some variables to be used in analysis • Calculating sample weights and adding to data files • Computing wealth index and adding to data files • Creating the tables required to analyse the data • [Archiving and distributing the data files]

49. Software • SPSS v 20 • Construction of analysis files • HH: Household • HL: Household Listing • WM: Women • MN: Men • CH: Children • Production of tabulations • Analysis of sampling errors/confidence intervals