Quality considerations in Statistical Surveys

1. Quality considerations inStatistical Surveys Mukesh K Srivastava FAO Statistics Division

2. What is Quality?

3. Concepts: What is Quality? • Means different things to different people • ease in use and comforts (beds sheets, clothes) • reliability (long life) • risk associated with use of product (electrical goods) • value for money (substitute perfume) • credibility (news paper) • Association of quality with “Brand names” • efforts to establish credibility (advertisements) • Giorgio Armani vs. DOS, MOA

4. Concepts: Scope of quality • Product • Characteristics • Process (computerized eye testing) • Inputs • Mozzarella di Buffala • Vino di Toscana • “Champagne” of specific region of France • Formal statements of quality need a framework • an agreed set of characteristics/variables on which information is to be provided for comparison • Total Quality Management: • INPUT- THROUGHPUT- OUTPUT

5. Quality in Official Statistics • National Quality Assessment Frameworks • http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx • A review reveals that the quality does not have the “same” meaning across the Globe, though there is broad consensus on its importance and key characteristics. • Distinguish between: • Quality of Data • Quality of Survey • Quality of the Statistical system of a Country (Country Assessments of Global Strategy)

6. Quality of Statistics: Product • How close to Reality? • Errors: measured by distance from reality • Sources of Errors: • Due to sampling and estimation procedure • Non-sampling errors (applicable to both sample survey and complete enumeration) • Coverage • Measurement • Response

7. Measures of quality of survey data • Estimates of Sampling errors: a measures of reliability of the estimate • (closeness to the true parameter) • Depends upon • Sampling design • The estimator (formula) e.g. mean, median, mode as measure of central tendency • The values in the specific sample (bad sample) • Statement about efforts to control non-sampling error • extent of coverage • quality of response (post-enumeration survey)

8. Decision to undertake survey user needs analysis survey objective definition Survey design survey concepts sample design development and testing of measurement instruments Data collection data sources training of enumerators non response Data capture and data processing data capture editing and imputation procedures Data Analysis and output quality Relevance, accuracy, timeliness, comparability, coherence, data analysis, disclosure control Documentation and dissemination metadata documentation dissemination strategies data management Improvement cycles adaptability/flexibility expertise in relevant areas quality management Process elements of survey quality

9. How to achieveQuality in Official Statistics? • UNSD • Principles governing international statistical activities • Fundamental Principles of Official Statistics http://unstats.un.org/unsd/methods/statorg/FP-English.htm • EUROSTAT • National Methodological Report on survey (model: Ref. doc.) • The European Self Assessment Checklist for Survey Managers (Ref. doc) • MEDSTAT workshop

10. شكرا جزيلا Message: • Statisticians should make an effort to develop a culture of quality in their domain by disseminating information on quality along with data. • Quality information helps in brand positioning • Quality information increases demand of the products and thus enhances revenues/resources.