Day 2 – Afternoon Preparatory tasks: Census forms and manuals

UNECE Workshop on Census Management in SPECA member countries StatCapCA Training Workshop Nr.1 Day 2 – AfternoonPreparatory tasks: Census forms and manuals Paolo Valente (UNECE)

Topics covered : 1. Form design 2. Form testing 3. Instructions manuals 4. Printing of forms and other documents Reference material: • Handbook on Census Management for Population and Housing Censuses (Chapter II; sections D, E, F)

Form design • A well-designed census form: • Allows to collect and capture data more efficiently • Reduce the number of errors and improve quality • Reduce the time and costs of data processing • A poorly designed census form: • Is difficult to complete by enumerator or respondent • May result in scarce cooperation by the respondent • May affect quality of data collected  These problems cannot be overcome later, during or after enumeration!

Form design • Types of forms (depending on methodology): • Individual • Household • Housing unit • Building • Communal establishments • …

Form design Issues to consider when designing census forms: 1. Respondent burden 2. Wording and format of questions 3. Form layout and design 4. Type and format of forms used

Form design 1.1 Respondent burden  Respondent burden must be MINIMIZED in order to obtain accurate information  Particularly important when self-enumeration method is adopted Factors affecting respondent burden: 1. Length of the form 2. Number and type of questions 3. How the form is easy to complete

Form design 1.2 Wording and format of questions Factors to be considered: 1. Level of accuracy and detail required (depending on users’ needs) 2. Use of language appropriate and easy to understand 3. Possible use of standard definitions or question wording 4. Data processing system 5. Order of questions 6. Space required for each answer and…

Form design 1.2 Wording and format of questions Factors to be considered: …7. Language diversity Possible strategies to deal with language diversity: • Multilingual forms • More versions of the forms, in different languages • Forms in one language, but translations available

Form design 1.3 Form layout and design  Form layout and design has a direct impact on how the form is completed, and on accuracy of output Most important factor is data-capture method: • Manual data entry • Form scanning + OCR/OMR  Requires specific format BUT  Data-capture requirements should not have a negative impact on the respondent’s perception

Form design 1.4 Type and format of forms used Individual form or Household form?  Depends on enumeration method adopted Different formats of household form: • Booklet: first questions on person 1, then person 2, 3… • Matrix: questions on left column, answer for persons in different columns Option: Use of short form and long form • Long form submitted to sample of households (15-20%)

Form design 1.4 Type and format of forms used Suggestion: Consider the experience of other countries!  UNECE in 2003 sent to all ECE countries a complete set of census forms used by all ECE countries in the 2000 round However: Be careful to possible effects of the same question wording in different languages,and to possible implications of cultural differences!

2. Form testing • Form testing should be only one of the components of a larger testing programme • The testing programme shout test effectively all the main component of the census: forms, enumeration, data processing etc. • The testing programme should focus in the first phase on testing forms. • Key element of the testing programme is the final pilot census, that should reflect as much as possible the final census conditions

2. Form testing • Form testing is particularly important for new topics or questions • Important to test also impact on other questions, already existing before • If changes to the form are considered, the form should be evaluated BEFORE and AFTER the changes, to assess if performance has improved

2. Form testing Different procedure to evaluate forms: 1. Analysis of errors 2. Cognitive testing 3. Analysis of quality and level of detail

2. Form testing2.1 Analysis of errors Analysis of errors: Counting number and type of errors occurred on a sample of forms Purposes: 1. Find out what errors occur 2. Provide information on possible changes 3. Provide benchmark to evaluate form 4. Allow comparison among different forms 5. Provide estimate of costs to repair errors

2. Form testing2.2 Cognitive testing Cognitive testing: Market research approach to form testing. Different activities, including: • Focus group research (group discussion) • Observational studies (observe respondents) Advantage: Provides in-depth understanding of how questions are interpreted by respondents Disadvantages: It’s relatively expensive, and it’s based on small samples

2. Form testing2.3 Analysis of quality and level of detail • Particularly important for open-ended questions where information has to be collected at high level of detail (occupation, industry) • The question wording influences detail, and ultimately quality of output • As part of the testing, replies are coded to evaluate if the level of detail is sufficient

3. Instruction manuals3.1 Categories of instruction manuals Generally, different manuals for different levels: 1. Enumerators’ handbook 2. Supervisors’ handbook 3. Regional mangers’ handbook  Staff at each level should be familiar also with handbooks for lower levels  Some headings may appear in all manuals, but with different content or different level of detail

3. Instruction manuals3.2 Enumerators’ handbook  The Enumerators’ handbook is one of the most important field documents! • It details responsibilities and tasks of the enumerator • It should provide the enumerator with sufficient information to work independently in the field Trade-off: Ideally, it should cover all eventualities BUT it should not be too bulky and difficult to use  Right balance: It should cover most ordinary situations in some detail, and provide guidance on how to deal with unusual situations

3. Instruction manuals3.3 Content of instruction manuals Suggested lists of contents for the three instruction manuals are presented in:  Handbook on Census Management for Population and Housing Censuses (Chapter II; section E)

4. Printing of census documents • Printing census documents is a MAJOR activity, due to the size of the operation and the amount of material to be managed • Timing, planning and coordination of the printing process are fundamental factors for the success of the census

4. Printing of census documents4.1 Planning and management of printing • Most difficult aspect:Coordination of the information needed from all census areas that require items to be printed • Information include: • Details of the items to be printed • Specifications needed for logistics (for packing etc.) • Reliable estimates of quantities • Key information for planning: • Timetable for preparation of documents from census areas • Timetable and requirements for delivery of printed material

4. Printing of census documents4.1 Planning and management of printing A system should be developed to recordfor each item to be printed: 1. all details relevant to obtain estimates of printing costs 2. quantities 3. estimated, quoted and actual costs • With this information the estimate of the total printing costs can be updated regularly, and the overall printing budget can be monitored

4. Printing of census documents4.2 Printing of census forms Printing of census forms has a CRUCIAL importance Important issues to consider: • The quality of form printing is very important  Data processing methods such us OCR/OMR require specific standards in terms of positioning of response areas, colours of the form, etc. • Estimation of the quantity of forms to be printed • The paper stock should be estimated well in advance, to make sure that paper of the required quality be available • Timing is very important, considering the long lead times

4. Printing of census documents4.3 Printing of other census material Printed itemsthat may be needed in large quantities: • Information booklets distributed by enumerators or mailed to all households in advance • Instructions for the respondents (for self-compilation) • Multilingual material or translation of questionnaires Other printed items: • Instruction manuals for enumerators, supervisors, etc. • Record books and control books and forms • Training material • Administrative items (for census staff recruitment, appointment, finance, etc…)

4. Printing of census documents4.4 Quality assurance The quality of census form is crucial for the success of the census, especially when using OCR/OMR  Census forms should be subject to intensive quality assurance checks • Checking of proofs (during development of material):At each stage of the development, proofs should be carefully checked by project leader • Production runs (during printing process):Independent checks should be conducted on samples of printed forms over the entire printing process

Day 2 – Afternoon Preparatory tasks: Census forms and manuals