Pre-printing experiences at Statistics Sweden

1. Pre-printing experiences at Statistics Sweden Anders Holmberg Department of Research & Development Statistics Sweden SE-701 89 Örebro Sweden Tel: +46 19 176905 Fax: +46 19 177084 E-mail: Anders.Holmberg@scb.se Anders Holmberg Seminar/University of Essex 2004-09

2. Outline • The pre-printing ‘culture’ at Statistics Sweden; motives for and against and some examples. • Results from two experimental studies • Conclusions from a review at Statistics Sweden • Some study areas for the future Anders Holmberg Seminar/University of Essex 2004-09

3. Pre-printing at Statistics Sweden • Historical data can be used in surveys where respondents are contacted repeatedly, (or if data of the same object e.g. a business that is collected repeatedly.) • Updating and/or verifying purposes: (Normally updating) • Dependent interviewing, Pre-printing self administered questionnaires. Anders Holmberg Seminar/University of Essex 2004-09

4. Possible advantages • Respondent support • e.g. Less response burden, Questionnaire guidance, memory support, anchoring, feed-back. • Reduction of measurement errors and improved data quality • Improved efficiency in the data collection process Anders Holmberg Seminar/University of Essex 2004-09

5. Possible disadvantages • Risk of bias due to underreporting of changes. • Loss in confidence and goodwill • Disclosure risks Anders Holmberg Seminar/University of Essex 2004-09

6. Pre-printing experiments • The LHU-survey 2000: • Annual survey of building rental units with premises used for commercial purposes. • Rented area and total monthly rental income for shops, industries and offices. • Due to the sampling design, the number of common building units in two successive samples is large. • FASTIGO’S yearly collection of wages • Individually designed Excel questionnaires to member companies (2000) • Contracts, working hours, occupation codes Anders Holmberg Seminar/University of Essex 2004-09

7. LHU experiment • Randomized block design with three treatments • No pre-print • One reference month pre-printed • Two reference months pre-printed • Response variability • Occurrence and size of outliers • Pre-printing recurrences (same value repetitions) • Effects on estimates of yearly rates of change Anders Holmberg Seminar/University of Essex 2004-09

8. Experiment plan (LHU) Anders Holmberg QDET Charleston 2002-11-17

9. Results LHU • Pre-printing clearly reduces spurious/unwanted response variability • Less and smaller outlier values with pre-printing, indicates higher data quality and less measurement errors • ‘Same-value-repetitions’ Pre-printing recurrences were more common Without pre-printing. • In this survey, pre-printing will improve the quality of the estimates of change over time. Anders Holmberg Seminar/University of Essex 2004-09

10. Treatment Type of Premises 1 2 3 Shops Minimum -87% -29% -22% Maximum 497% 77% 292% Proportion of units with a deviating value > 25%. 14.3% 4.2% 2.0% Table 1: Ratio of repeated measures for December 99. Max. and Min. values and proportion of units with a deviating value larger than 25% Anders Holmberg Seminar/University of Essex 2004-09

11. Direction of change Treatment Total number of units 1 2 3 Decreaselower value reported for Dec-00 19 29 16 198 UnchangedSame value reported 30 18 23 132 Increase higher value reported for Dec-00 51 54 61 368 Total 100 100 100 635 Table 2: Number units with office premises reporting different values of rental income in December-00 compared to December-99, by treatment and direction of change. (%) Anders Holmberg Seminar/University of Essex 2004-09

12. FASTIGO’s wages • They wanted to stop pre-printing the variable Hours worked (in % of full time), since they thought changes in the variable was not updated frequently enough. • They deliver data to Stat Sweden for some of their companies. • Time of response, as well as a dichotomous outcome variable (at least one change (update) in hours worked since last year/no changes) were studied. • Pre-printing and not pre-printing treatments were randomized within size groups and whether companies delivered data to Stat Sweden or not. Anders Holmberg Seminar/University of Essex 2004-09

13. FASTIGO’s wages • No differences were observed in response time. • Somewhat surprising to the FASTIGO staff, the updating / change frequency were higher for the pre-printing treatment! (Significantly higher in the group of Stat. Sweden companies). Anders Holmberg Seminar/University of Essex 2004-09

14. FASTIGO’s wages Proportion of companies with at least one update by treatment and group. Anders Holmberg Seminar/University of Essex 2004-09

15. Summary of the studies • Both the response burden and data quality arguments can be strong in favor of pre-printing. • Studies indicate that spurious response variability is significantly reduced with pre-printing. • In our review no cases were found were the possible negative properties of pre-printing had significant effects on macro-level statistics. • With careful (re-)design of questionnaires and proper testing of its effects on data quality, pre-printing historical data should be seriously considered. Anders Holmberg Seminar/University of Essex 2004-09

16. Future fields and some ongoing activities • Data of the collection process automatically captured/monitored in electronic questionnaires (online/offline) • Layout issues • Multidisciplinary methodological efforts especially concerning effects of pre-printing on the rate of change of categorical variables. Anders Holmberg Seminar/University of Essex 2004-09

17. Anders Holmberg Seminar/University of Essex 2004-09