A Framework for the Accuracy Dimension of Data Quality for Price Statistics

1. A Framework for the Accuracy Dimension of Data Qualityfor Price Statistics Ottawa Group Ottawa, October 2007 Presenter: Geoff Neideck Australian Bureau of Statistics

2. Aim of the Presentation • Introduction • Extending the ABS Quality Framework • Comparisons with other Frameworks • Why Do Errors Occur? • Key Principles in Managing for Accuracy • Quality Gates

3. Introduction • Many NSOs have experienced errors in their prices statistics • Some major reviews of practices have been undertaken • Some serious consequences Undermining reputation and raising questions of reliability

4. Extending the ABS Data Quality Framework • Dimension of quality: • Relevance • Timeliness • Accuracy • Coherence • Interpretability • Accessibility

5. Extending the ABS Data Quality Framework

6. Comparisons with other frameworks – ILO CPI Manual Ch. 11 Taxonomy of Errors • Sampling error • Selection error • Estimation error • Non-sampling error • Observation error • Over coverage • Response error • Processing error • Non-observation error • Under coverage • Non-response

7. Comparisons with other frameworks – IMF DQAF for CPI/PPIs

8. Why Do Errors Occur? • Critical Risk Areas • Culture • Change • Education • Documentation • Engagement with stakeholders • Spreadsheets and black boxes

9. Culture • Understanding the ‘why’ • Taking an ‘end-to-end’ view • Taking a questioning approach • Taking responsibility for quality

10. Change • Indexes subject to much change • Risks increase • Routine change • Regular but less frequent change • New methods and procedures • Transfer of knowledge • Change outside existing systems

11. Prices theory • Introduction of • new concepts • and practices Best practice -choice of options Standard methods low high Complexity of knowledge & decision-making Education

12. Documentation • High on staff list of things that would enable them to do a better job • Necessary but not sufficient • Filling in the gaps/making assumptions

13. Engagement with stakeholders • Identifying key stakeholders • Flow of information • Knowledge transfer

14. Spreadsheets and black boxes • Spreadsheets • A great tool but …. • Usually outside the existing system • Cottage industries • Protocols/documentation • Black Boxes • Data in, data out … • …but what happened in between

15. Key Principles for Managing Data Quality • Developing and sustaining a quality culture • A program of capability development • Managing Change • Regular communication with stakeholders • Appropriate documentation • A program of reviews

16. Index Compilation Process and Quality Gates Source data Intermediate data Output data Q Q Q Q Data Collection Micro Editing Macro Editing Output Indexes Data Release Products/ Outlets Upper level indexes EAs Q = Quality Gate Direct Collection Other Sources Validation against independent sources Product Information Industry Information Independent Indicators Quality Gates