1 / 9

Enhancing Data Mining Through Effective Data Preprocessing

Understand the importance of data preprocessing to improve data quality and mining process efficiency. Explore techniques like data cleaning, integration, transformation, and reduction for better pattern extraction. Handle missing values, noisy data, and inconsistencies to enhance analysis.

marlenef
Télécharger la présentation

Enhancing Data Mining Through Effective Data Preprocessing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Preprocessing

  2. The need of data preprocessing • Problems with huge real-world database • Incomplete data : missing value • Noisy • Inconsistent  Influence data mining process, especially pattern mined

  3. Techniques • Data cleaning • Data integration • Data transformation • Data reduction  Improve the quality of the pattern mined and/or the time required for the actual mining

  4. Data Cleaning – Missing values  Tuples have no recorded value for several attributes • Ignore the tuple • Fill in the missing value • Using global constant • Using ‘measured’ values : attribute mean, most probable value

  5. Data Cleaning – Noisy  Random error or variance in a measured variable • Binning  smooth a sorted data value by consulting its ‘neighborhood’  local smoothing

  6. Clustering  Detect the outliers by grouping similar values Regression  smooth data by fitting data to a function, such as regression  linear regression, multiple linier regression

  7. Data Integration • Combine data from multiple sources into coherent data store • Schema integration: entity identification problem • Redundancy: detected by correlation analysis • Detection & resolution of data value conflict: semantic heterogenity & different representation

  8. Data Transformation • Data are transformed or consolidated into forms appropriate for mining • Involve: • Smoothing • Aggregation • Generalisation • Normalisation

  9. Data Reduction • Reduce representation of data set that is much smaller in volume, while maintains the integrity of the original data. • Strategies: • Data cube aggregation • Dimension reduction • Data compression

More Related