1 / 30

Towards Producing and Using Response Burden Data for Establishment Surveys at Statistics Sweden

Johan Erikson, Dan Hedlin, Boris Lorenc Statistics Sweden. Towards Producing and Using Response Burden Data for Establishment Surveys at Statistics Sweden. Background.

samuel-strickland
Télécharger la présentation

Towards Producing and Using Response Burden Data for Establishment Surveys at Statistics Sweden

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Johan Erikson, Dan Hedlin, Boris Lorenc Statistics Sweden Towards Producing and Using Response Burden Data for Establishment Surveys at Statistics Sweden Q2008 - Rome Italy, July 2008

  2. Q2008 - Rome Italy, July 2008 Background • A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ • Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏

  3. Q2008 - Rome Italy, July 2008 Background • A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ • Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏

  4. Q2008 - Rome Italy, July 2008 Background • A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ • Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏

  5. Q2008 - Rome Italy, July 2008 Background • A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ • Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏ • Regulations, SCM, earlier Stats Sweden work

  6. Q2008 - Rome Italy, July 2008 Agenda • Background • Relation between Handbook and the results of the Statistics Sweden response burden (RB) project • The process of measuring response burden according to Handbook • Suggestions according to the Statistics Sweden RB project • Towards the implementation • the mini-measurement and its implementation • glance into the future • Conclusions

  7. Q2008 - Rome Italy, July 2008

  8. Q2008 - Rome Italy, July 2008 The Handbook: Intro • Basically discusses the following main topics: • Monitoring the burden (Ch. 2)‏ • 9 core questions (q’s)‏ • Evaluating analytically the burden, relating it to causal factors (Ch. 3)‏ • About 17 additional q’s • Sampling and data collection procedures (Ch. 4 & Ch. 5)‏ • Reporting (Ch. 6)‏

  9. Q2008 - Rome Italy, July 2008 The Handbook: Intro‏ • Basically discusses the following main topics: • Monitoring the burden (Ch. 2)‏ • 9 core questions (q’s)‏ • Evaluating analytically the burden, relating it to causal factors (Ch. 3)‏ • About 17 additional q’s • Sampling and data collection procedures (Ch. 4 & Ch. 5)‏ • Reporting (Ch. 6)‏

  10. Q2008 - Rome Italy, July 2008 The Handbook: Intro (cont’d)‏ • In discussing the topics I-II, identifies some key distinctions, amongst them: • actual vs. perceived response burden • survey burden vs. time spent on survey • response process steps: data collection vs. filling in the questionnaire (Q)‏

  11. Q2008 - Rome Italy, July 2008 The Handbook: Actual/Perceived Distinction • Actual burden = Perceived burden? No, that's why both actual & perceived needed! • “The aim of this handbook is to provide a tool for measuring perceived and actual response burden in business surveys that will help statistical organisations and other parties carry out their own response burden (PRB) surveys.” [p.1] • “This publication is a handbook written for statistical organisations and other institutions that carry out business surveys. Its main topic is how to monitor and evaluate perceived response burden (PRB) among business survey respondents.” [p.5]

  12. Q2008 - Rome Italy, July 2008 The Handbook: Burden/Time Distinction • Burden = Time? No, that's why both time & burden needed! • “...it is not the time use itself but the perception of time and effort that is likely to affect response quality. Time passes quickly if the topic is interesting and the respondent feels that he or she is competent to answer the questions posed.” [p.6] • “Actual burden is traditionally measured by time use. As we have mentioned earlier, we have chosen more detailed time measures than what has been traditionally used. We have designed two questions to measure time to collate information and one to measure time to fill in the questionnaire.” [p.11]

  13. Q2008 - Rome Italy, July 2008 The Handbook: Steps of the Response Process Distinction • The response process has these two steps (amongst others): • data collection (corresponds to step 5 of the Hybrid model of Willimack, Nichols & Sudman)‏ • questionnaire fill in (corresponds to step 7 of the Hybrid model)‏

  14. Q2008 - Rome Italy, July 2008 The Handbook: I. Monitoring Burden (Ch.2)‏ • 4 dimensions, 8 indicators, 9 variables:

  15. The Handbook: II. Analysing Causes of Burden (Ch.3)‏ • 4 dimensions, 6 indicators, 17 variables:

  16. Q2008 - Rome Italy, July 2008

  17. Q2008 - Rome Italy, July 2008

  18. Q2008 - Rome Italy, July 2008

  19. Q2008 - Rome Italy, July 2008 Issues Discussed in the RB-Project‏ • Given burden ≠ time, is Perceived length of time (q.1 of Monitoring) properly an indicator of Perceived burden(dim.1)? • Is Self-reported time (q.3 of Monitoring) properly an indicator of Actual burden (dim.2)? • from the premise of actual burden ≠ perceived burden • from the measurement perspective: a retrospective question‏ • Perceived time asked only regarding data collection (q.1), Perceived burden asked only regarding Q completion (q.2). Why? Why not the converse?

  20. Q2008 - Rome Italy, July 2008 Issues Discussed in the RB-Project‏ (cont’d)‏ • Is motivation to participate in surveys exhausted by tackling the utility of participation for the business and the society? • Cooperation principles (mentioned in the Handbook)‏ • Social exchange • Helpfulness, etc. • Practicalities about the questionnaire: • Length • Content (e.g. Why causes in Monitoring [its dim.3]?)‏ • Question formulation (e.g. Time to collect information - business (q.4), does that include or exclude Time to collect information - self (q.3)?)‏ • Question order (e.g. the complex branching instructions pertaining to the first question)

  21. Q2008 - Rome Italy, July 2008 Positions Taken in the RB-Project‏ • Measure respondent burden or measure respondent work? • Measure burden to begin with, but bring later on even positive aspects of survey participation into focus • Monitor or evaluate? • Monitor on a broad front, evaluate in conjunction with changes or actions for improvement. (Same as in the Handbook.)‏ • Actual or perceived burden? • Perceived, as well as an (assumed) correlate of actual time: the self-reported time (for oneself and for others)‏ it took to participate in the current survey

  22. Q2008 - Rome Italy, July 2008 Positions Taken in the RB-Project‏ (cont’d)‏ • Breakdown of burden in the evaluation Q: along questions or along steps of the response process? • Response process steps • Motivation • More than utility: left out for the moment • Explanatory variables • Number of people involved (q.3 of Evaluation)‏ • Assessment of data availability (-)‏ • Previous experience with the same survey (q.9 of Evaluation)‏ • Respondent’s role in the current data collection (q.4 of Eval.) • [Data regarding simultaneous surveys can be taken out of our own data base]

  23. Q2008 - Rome Italy, July 2008 Basic Proposal of the RB-Project‏ • Perceived burden measured on three levels: • Monitoring on a broad level (level A; contains the mentioned explanatory variables)‏ • Breakdown of burden along the steps of the response process (level B); measured in conjunction with carrying out an improvement project‏ • Identification of probable causes (level C); measured in conjunction with carrying out an improvement project‏

  24. Q2008 - Rome Italy, July 2008 RB-Project‏: Level A • Variables • Perception regarding the current participation • General level of burden • A correlate of actual burden • self-reported time it took for own participation (retrospective) • reported time it took for other’s participation (retrospective) • Explanatory variables • ‏Number of people involved • Assessment of data availability‏ • Previous experience with the same survey • Possibly, suggestions for improvement‏

  25. Q2008 - Rome Italy, July 2008 RB-Project‏: Level B • Variables • Perceptions regarding the current participation • Burden to understand the request for data • Burden to distribute and coordinate the data collection • Burden to collect data • Burden to process data • Burden to provide data in the required form • Explanatory variables • Respondent’s role in the current data collection

  26. Q2008 - Rome Italy, July 2008 RB-Project‏: Level C • Variables • Perceptions regarding the current participation • Causes of the burden to understand the request for data • Causes of the burden to distribute and coordinate the data collection • Causes of the burden to collect data • Causes of the burden to process data • Causes of the burden to provide data in the required form

  27. Q2008 - Rome Italy, July 2008 RB-Project‏: Conclusion • Level A measured systematically and continuously in all enterprise surveys • But, not each occurrence of an recurring survey: a sampling procedure needed (not treated here) • Levels B/C used in conjunction with evaluation or improvement of the data collection procedures • Provide input for simplification, questionnaire evaluation work, etc. • Possibly followed by qualitative methods on an even smaller scale

  28. Q2008 - Rome Italy, July 2008 The Mini RB-measurement • Implemented gradually since a year ago • Measures self-reported time • Consists of a single question, posed at the end of the survey • Time within the business it took to participate in the current survey • Routines to process, present and use these data are currently implemented • sampling • estimation • useful summary tables • input to other processes

  29. Q2008 - Rome Italy, July 2008 Conclusion • Do measure response burden • But, only if the collected data are used for the benefit of the data provider • Otherwise, no real justification to do this kind of survey

  30. Q2008 - Rome Italy, July 2008 Thank You Johan Erikson Dan Hedlin Boris Lorenc firstname.lastname@scb.se

More Related