Jitka Prokop, Czech Statistical Office

1. Jitka Prokop, Czech Statistical Office SMS-QUALITY The project and application focused onmetadata on quality European conference on Quality in Official statistics 3-5 June 2014, Vienna

2. Introduction - aims, features, coverage, current state • Architecture - Q-attributes, hierarchical structure, design, preparation, data retrievals and inputs, functionality, stability of values and updates • Benchmarking • Challenges • Topics

3. Starting points • Horizontal way of management • Demands for quality reporting & relevant metadata ‚standardisation‘ • Standardisation of quality reports & adjustments for domain statistics Tool for managers • Semi-interactive cross-cutting overviewsabout quality of a survey (incl. assessments…) • Quality reporting • Application using web-browserenvironment • Reasons and Aims

4. Q reporting focused on a survey (of any kind) and groups of surveys • Cross-cutting info on quality of statistical process, outputdata and products • Data preferably retrieved from other source databases • Data monitoring, comparisons, aggregations, assessment, benchmarking • Flexibilityof metadata content and possibility of survey’s adjustments • Help to improve quality reporting and statistical quality itself • Encourage self-assessment, support auditing • Aims and main features

5. The application is integrated within the internal SIS and SMS systems • Meta-data values retrieved from databases or manually inputted • Design & preparation of various quality reports • Hierarchical structure (refers to ESQR, GSBPM, DESAP) • Publicvs. non-public – individual items or complete reports • Bilingual (multi-lingual) solution • Usual output formats PDF, HTML, XLS, DBF, DOC, not SDMX • User roles: admin, owner, editors, viewers (public vs. internal) • Features

6. Other SMS subsystems can provide certain knowledge on quality criteria e.g. accuracy, relevance, accessibility, clarity, timeliness, punctuality. • SMS SURVEYS: statistical processes (particular surveys) • SMS REQUIREMENTS: management of main user requirements • SMS DISSEMINATION, CATALOGUE OF PUBLICATIONS: dissemination, product quality, info service in some cases in relation to concrete surveys • Interlinks with other SMS-applications

7. Any statistical process processed or at least with its data stored in the central DWH. • Business statistics • Social and demography statistics • National accounts • Price statistics • Administrative data statistics. • Coverage

8. Type of info on quality - quantitative and qualitative: • Reference metadata • Info about process and its phases • Schedules • Quality performance indicators • Calculations • Benchmark results • Evaluation, (self-)assessments, commentaries • Textual, Numerical, Date • Q-attribute (item, meta-information, indicator)

9. Basic information (about a survey) • User requirements agenda • Methodology info • Time schedules; Timeliness; Punctuality • Statistical process phases • Data confidentiality and protection • Data sources; Frame; Sample • Outputs and dissemination • Individual quality criteria (i.e. quality dimensions) • Quality performance indicators • Categories of Q-attributes (info on…)

10. Relates to functionality (stability of values) • General • Statistical survey (key users, methodology, key statistical variables…) • Reference year • Processing (all reference periods processed or revised at one time) • Reference periods • Levels of Q-attributes

11. To provide relevant & up-to-date information • Validity for certain years, batches (i.e. processings), ref.periods • When generating data for new reference periods... • Metadata updates on each level • How to update the derived Q-Maps • Managers informed and decide via the application • Keeping history and updates • Updates of metadata structures and values

12. Structure (hierarchy): Sections, Sub-sections, Q-attributes • Q-Maps: monitoring, benchmarking General - > Specific -> Survey Q-Maps design, specifications Value Q-Maps output report • Q-Forms: also comparisons and aggregations… General QM for Q-Forms -> Q-Form -> Value Q-Form Q-Forms use (not only) ValueQ-Maps as the source of data • Q-Maps & Q-Forms

13. Comparisons, aggregations over • Statistical variables • Reference periods • Surveys • Years… Which data • Values • Benchmark results • Q-Forms - comparisons, aggregations

14. Design of a report • General Q-Map -> A type of report. General design, pre-setting of parameters. • Specific Q-Map -> A group of surveys. Selection of Q-attributes, way of benchmarking. • Survey Q-Map -> A survey Statistical variables, benchmark scales, links to data. Output report • Value Q-Map -> One reference period. Retrieval, editing, approval of values. Benchmarking. • Levels of Q-Maps - Hierarchy

15. Primarily for internal management purposes • Benchmarked values: numerical or textual • Adjustments of scales (boundaries) for particular surveys • Parameters • To benchmark or not to benchmark? • Manually (each value individually) or Automatically (pre-definitions) • Categories’ definition –number of categories, and either definition of boundaries or assignment of values from a nomenclature • Categories’ labelling –from a special nomenclature or directly in the app • Benchmarking

16. Deeper relations between subsystems • Revisions of quality attributes • Involvement of domain statisticians • Full implementation • ESS standard quality reports in SDMX • Challenges

17. Thank you for your attention. Any questions? jitka.prokop@czso.cz