The application of selective editing to the ONS Monthly Business Survey

1. The application of selective editing to the ONS Monthly Business Survey Emma Hooper Office for National Statistics emma.hooper@ons.gsi.gov.uk

2. Overview • Editing at the ONS • Monthly Business Survey (MBS) • Application of selective editing to MBS • Quality indicators • Implementation and post-implementation

3. Editing at the ONS • 2008 project reviewed editing processes for Office for National Statistics (ONS) business surveys • New selective editing methodology for ONS short-term business surveys • Mix of selective editing and traditional manual micro editing was previously used

4. Surveys using selective editing • Tested and implementing selective editing for the Retail Sales Inquiry • methodology developed with assistance from Pedro Silva (University of Southampton) • MBS selected as second survey to test and implement selective editing on • Currently investigating using Selekt for Annual Business Inquiry

5. Monthly Business Survey • Launched in January 2010, it brings together existing short-term surveys that cover different sectors of the economy • Old selective editing methodology used edit rules, those units that failed an edit rule would have a selective editing score calculated • New selective editing methodology to run on live MBS data from summer 2010

6. Editing processes for MBS 1. Edit rule checks Check for valid dates 2. Automatic editing Check £000s error and components 3. Selective editing Check records with unit score greater than threshold 4. Macro editing Check aggregated data

7. Selective editing for MBS • Target units that have significant effect on key estimates by domain (input/output group) if not edited • Calculate item score for each unit and key variable • turnover, export turnover, new orders (monthly) and total employment (quarterly) • Predictor for true value • previous edited value (else use register value for turnover or employment, or pseudo-imputed value for export turnover or new orders)

8. Item score

9. Unit score • Combine item scores into single unit score using average of item scores • Units ranked according to their unit score • if score for a unit is above threshold then that units responses are sent for manual editing • units with scores below threshold are not manually checked

10. Thresholds • Thresholds set for each key domain to reduce editing costs without impacting quality • Quality indicators used to compare thresholds • 41 periods of data used, should ensure robustness of results

11. Absolute relative bias • Absolute relative bias aims to control the residual bias left in the domain estimates after editing

12. Savings • Savings measure the change in the number of units that will be manually micro edited

13. Absolute relative bias

14. Savings

15. Quality indicators • Aimed to keep ARB below 1%, ARB levels showed large improvement compared to bias left after current micro editing • Overall savings in the number of units being edited of around 40% in non-employment months • Overall savings of 55% (MPI sectors) and 15% (MIDSS sectors) in employment months

16. Current edit rule method

17. New selective editing method

18. Implementation and limitations • Selective editing is carried out via a module in the in-house built Common Software system • The module is currently • restricted to 5 item scores • restricted to combining the item scores as a mean or maximum • restricted to only using variables already available in the system for use in calculating predicted values • not able to use current edit rules to calculate an edit-related score

19. Following implementation • Need to monitor the thresholds, ideally through editing a small sample of those that aren’t being selectively edited • This would enable us to estimate the bias left in the estimates and adjust the thresholds accordingly • Continue testing these methods for other ONS business surveys, more efficient editing will result in a better quality editing process