1 / 23

Information Management Framework Data Quality

Information Management Framework Data Quality. What is quality. Quality is dynamic concept that is continuously changing to respond to changing customer requirements Defined in 3 ways: Conformance to specifications (DQA) Fitness for use (Surveys). Quality issues. Problems can result from:

rad
Télécharger la présentation

Information Management Framework Data Quality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Information Management Framework Data Quality

  2. What is quality • Quality is dynamic concept that is continuously changing to respond to changing customer requirements • Defined in 3 ways: • Conformance to specifications (DQA) • Fitness for use (Surveys)

  3. Quality issues • Problems can result from: • Human error • Machine error • Process error

  4. Purpose

  5. Conformance to specifications: Quality Plan Data Quality Assessments

  6. Data Quality Assessments

  7. DQ Assessment and Remediation Process Data Remediation Approval / Priority Process Part 3 Data Management Plan Data Quality Assessment Audit Recommendations Data Store Data Collection Data Access Historic data Collection Storage Access & Use Archive/Disposal Information lifecycle phases

  8. Recording quality - ANZLIC

  9. Business rules • Each business rule should have an expected outcome (benchmark) • Business rules need to align to quality ANZLIC elements

  10. Findings - DQ Processes • The processes and guidelines are good! • The Data Management Plan is important • Needs to be completed by all data sets prior to Assessment • Benchmarks for quality established with Data Managers before DQA

  11. Soil Profile • Very large and varied data set (millions of soil properties) • Where Data exists - is mostly good • Many missing values • Data Transformation Errors • Data on forms different to values in database • Missing values set to default values in load program.

  12. Data Analysis – Soil Properties • Examples of problems: • Location Accuracy - Invalid grid references for a grid zone • Mandatory Fields missing data • Nature of Exposure - 1269 records missing value • Logical Inconsistencies • If Horizon Code begins with 'B' And ACS Order is 'SO' (Sodosol)Then pH >= 5.5238 records in error.

  13. Data Analysis – Ground Water • Minimal spatial data (point locations only) • Data where present is mostly good • Many missing values

  14. Examples of problems • Invalid Key fields • Work Number of non standard format • Location Accuracy • Invalid grid references for a grid zone • Logical Inconsistencies • Jobs completed before they started • Hole depth of 36km • Mandatory Fields missing data • Work Type Code - 1503 records missing value.

  15. Region Code Region Name GW Licenses in LAS GW Licenses in GDS GW Licenses not in GDS Percentage Missing 10 Sydney - South Coast 3622 3420 202 5% 20 Hunter 2000 1280 720 36% 30 North Coast 3201 3162 39 1% 40 Murrumbidgee 1912 1807 105 5% 50 Murray 2350 911 1439 61% 60 Lower Murray / Darling 84 42 42 50% 70 Lachlan 1913 1371 542 28% 80 Macquarie - Western 2345 2002 343 14% 90 Barwon 4526 4445 81 2% Data Analysis – Ground Water • Database Issues: • No Load or creation date in database (only update date) • Impossible to apply date based business rules • GW licenses mandatory from 2001 onwards. • Logical Inconsistencies: • License Form A received and no GDS record (1000’s) • Needs investigation

  16. Data Analysis • Action Lists • Generated for each data set • Scope of Remedies • Improving data quality goes beyond the identifying, measuring and fixing the data in the IT systems. • Improve data capture • Train entry staff • Replace entry processes • Provide meaningful feedback • Change motivations to encourage quality • Add defensive checkers, Periodic DQ asssessments, Data cleansing

  17. Data Quality Reporting • Data Quality Portal • General DQ information • Statistical Reporting and Monitoring • Data Quality Exception Reporting • Management of Data Quality issues

  18. Fitness for use - User needs covered later in day

  19. Improving quality

  20. Ways of improving quality • Tackle quality at source, not downstream in the lifecycle • Training data collectors in importance on getting it right • Continual improvement with quality method

  21. Links among Process Groups in a Phase Initiating process Planning Process Controlling process (check) Executing process (do) (Arrows represent flow of information) Closing process ( PMBOK 2000 Fig 3-1 p31)

More Related