SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Data Quality Issues: ... by Example

binah avatar
binah
11 diapositives · 753 Views · Sep 09, 2014

Data Quality Issues: ... by Example. Wolfgang Lehner Dresden University of Technology Faculty of Computer Science Database Technology Group (SyA). Dagstuhl Seminar on Data Quality. … my personal background. Statistical and Scientific Databases

#data quality issues: ... by example #data quality issues: ... by example ppt #powerpoint #presentation
Data Quality Issues: ... by Example - slide 1
01 / 11Data Quality Issues: ... by Example
1 / 17

Télécharger la présentation

Data Quality Issues: ... by Example
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Data Quality Issues:... by Example • Wolfgang Lehner • Dresden University of Technology • Faculty of Computer Science • Database Technology Group (SyA) • Dagstuhl Seminar on Data Quality

  2. … my personal background • Statistical and Scientific Databases • very long tradition (1st Berkeley Workshop in 1981) • modeling perspective • come up with statistical data models • capture semantics of micro and macro data • processing perspective • "workflow": collection, preprocessing, analysis • provide database technology for efficient data analysis • a small subset of SSDB-techniques are used by • Data-Warehouse-Systems …

  3. Example 1: Processing Perspective of DQ Non-Food Tracking Retail panelbased marketing information for manufacturers and retailers in consumer technology industries GfK Group Monitoring the global markets for consumer goods • Which arethetop ranking brands of mobile phones at present? • How large is the market share of digital television sets? • How much do consumer spent on computer in average? periodical monitoring

  4. local terms per shop Clients global reporting terms Retailers Working Areas Data - IN Data - Preparation MDM IDAS Data Warehouse(Extrapolation, Reports) DWH Creating value through knowledge

  5. RawTrackingData DataRelease DataOrder Data Orders Reporting projects Base projects Identified data DWH QC Korrdat DWH QC project Target  Actual DWH Reporting „Good“ data Data & Control Flow from 10.000 feet DWH2Tools GIMWinCosSeparation WebTAS MDMitems & shops DHWSuite DWH Explorer DWH Extrapolation /Projection DWH Builder IDAS InfoSystem Preprocessing Fact Tool IDASOutput Pool DWHLoader IDAS2DWH Dataflow IDAS2MF Controlflow

  6. Data & Control Flow … more detailed

  7. Issues for Data Quality Discussion • Observation • subsequent production steps depend on the success and DQ of the result set produced by preceding steps(simplest form: count(*) > x, in most cases expert knowledge) • data context (more technical: primary key) changes(very hard to trace outliers at the end back to the incoming data object) • DQ determines production process, e.g. TODO-lists of >100 workers • DQ is key factor for Production Optimization • impact on demand-driven production • Example: need report of cell phone sales by end of next week • report quality depends on type of customer: premium customer  higher data quality  identify important data providers, prioritize single jobs/data orders/…  propagate individual deadline to participating working steps • impact on error correction • more raw data material: manual article identification (extremely expensive) • better raw data material: manually "correct" data, re-order tracking data  data lineage is a critical issues(fine granular data quality causes data explosion!!!)

  8. Example 2: Integration Perspective of DQ Perform Analysis across different Data Sources • What a similar sub-sequences of amino acid residues? • What are stable/typical conformations? Current Situation • independent (mostly non-relational) data sources • no integrity constraints (within/across) different data sources • DQ is key factor for integrated data access • problems beyond "regular" integration issues data sets are growing…

  9. protein code 1AL3, atoms 1478 to 1492 conflicting 3D coordinates for protein atoms (protein code 1AL3, atoms 1478 to 1492) Example: Protein Structure  the happy day scenario: 1:N Protein Code Atom Positions  the real world scenario:

  10. Issues for Data Quality Discussion • Observation • in theory: "nice" entities and relationshipsin practice: many exceptions due to the experimental nature of the data collection process • have to relax the schema constraints bad "data schema quality" • Data Schema Quality • impact on data analysisstatistical analysis process requires constraints as a guideline of data exploration (e.g. dimensional structures in OLAP) • need outlier management at the conceptual level schema exceptions

  11. Summary and Conclusion • Data Quality • key issue in statistically analyzing huge data sets • data analysis means:complex – often DQ driven - process of transforming micro into macro data • Solution • a) no size fits it all ... • b) need a general framework considering multiple aspects • statistical metadata • data lineage (most critical from my perspective) • users with expert knowledge (voting concept?)