1 / 20

Introduction to Data Mining

Introduction to Data Mining. Dr. Hany Saleeb. Why Data Mining? — Potential Applications. Direct Marketing identify which prospects should be included in a mailing list Market segmentation identify common characteristics of customers who buy same products Market Basket Analysis

Télécharger la présentation

Introduction to Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Introduction to Data Mining Dr. Hany Saleeb

  2. Why Data Mining? — Potential Applications • Direct Marketing • identify which prospects should be included in a mailing list • Market segmentation • identify common characteristics of customers who buy same products • Market Basket Analysis • Identify what products are likely to be bought together • Insurance Claims Analysis • discover patterns of fraudulent transactions • compare current transactions against those patterns

  3. What Is Data Mining? • Combination of AI and statistical analysis to discover information that is “hidden” in the data • associations (e.g. linking purchase of pizza with beer) • sequences (e.g. tying events together: marriage and purchase of furniture) • classifications (e.g. recognizing patterns such as the attributes of employees that are most likely to quit) • forecasting (e.g. predicting buying habits of customers based on past patterns) Expert systems or small ML/statistical programs

  4. What can data mining do? • Classification – Classify credit applicants as low, medium, high risk – Classify insurance claims as normal, suspicious • Estimation – Estimate the probability of a direct mailing response – Estimate the lifetime value of a customer • Prediction – Predict which customers will leave within six months – Predict the size of the balance that will be transferred by a credit card prospect

  5. What can data mining do? (cont’d) • Association – Find out items customers are likely to buy together – Find out what books to recommend to Amazon.com users • Clustering – Difference from classification: classes are unknown!

  6. Market Analysis and Management • Where are the data sources for analysis? • Credit card transactions, loyalty cards, discount coupons, customer complaint calls, plus (public) lifestyle studies • Target marketing • Find clusters of “model” customers who share the same characteristics: interest, income level, spending habits, etc. • Determine customer purchasing patterns over time • Conversion of single to a joint bank account: marriage, etc. • Cross-market analysis • Associations/co-relations between product sales • Prediction based on the association information

  7. Data Mining: Confluence of Multiple Disciplines Database Technology Statistics Data Mining Machine Learning Visualization Information Science Other Disciplines

  8. Data Mining: On What Kind of Data? • Relational databases • Data warehouses • Transactional databases • Advanced DB and information repositories • Object-oriented and object-relational databases • Spatial databases • Time-series data and temporal data • Text databases and multimedia databases • Heterogeneous and legacy databases • WWW

  9. Data Mining Process Learning Collecting relevant data Model building Understanding of business Problem identification Business strategy and evaluation Action

  10. Requirements/challenges in Data Mining • User interface • Mining methodology • Performance • Data source • Social and Security

  11. Requirements/challenges in Data Mining(2) • User interface - Data Visualization • Understandability and interpretation of results • Information representation and rendering • Screen real-estate - Interactivity • Manipulation of mined knowledge • focus and refine mining tasks • Focus and refine mining results

  12. Requirements/challenges in Data Mining(3) • Mining Methodology • Mining different kinds of knowledge in databases • Interactive mining of knowledge at multiple levels of abstraction • Incorporation of background knowledge • Query languages • Expression and visualization of results • Handling noise and incomplete data • Pattern evaluation

  13. Requirements/challenges in Data Mining (4) • Performance • Efficiency and scalability of data mining algorithms • Linear algorithms needed • Parallel and distributed methods • Incremental methods • Divide and conquer?

  14. Requirements/challenges in Data Mining(5) • Data Source • Diversity of data types • Handling complex types of data • Mining information from heterogenous data bases or information repositories • Can we expect a DM algorithm to do well on all types of data ? • Data glut • Are we collecting the right data for the right answer? • Distinguish between important and unimportant data

  15. Requirements/challenges in Data Mining(6) • Social and Security -Social Impact • Private and sensitive data is gathered and mined without individual’s knowledge and/or consent • Appropriate use and distribution of discovered knowledge - Regulations Need for privacy and DM policies

  16. Data Mining Tools

  17. DBMiner : A free tool • DBMiner: A data mining system originated in Intelligent Database Systems Lab and further developed by DBMiner Technology Inc. • OLAM (on-line analytical mining) architecture for interactive mining of multi-level knowledge in both RDBMS and data warehouses • Mining knowledge on Microsoft SQLServer 7.0 databases and/or data warehouses • Multiple mining functions: discovery-driven OLAP, association, classification and clustering

  18. Input and Output • Input: SQLServer 7.0 data cubes which are constructed from single or multiple relational tables, data warehouses or spread sheets (with OLEDB and RDBMS connections) • Multiple outputs • Summarization and discovery-driven OLAP: crosstabs and graphical outputs using MS/Excel2000 • Association: rule tables, rule planes and ball graphs • Classification: decision trees and decision tables • Clustering: maps and summarization graphs • Others: • Data and cube views • Visualization of concept hierarchies • Visualization for task management • Visualization of 2-D and 3-D boxplots

  19. Data Mining Tasks • DBMiner covers the following functions • Discovery-driven, OLAP-based multi-dimensional analysis • Association and frequent pattern analysis • Classification (decision tree analysis) • Cluster analysis • 3-D cube viewer and analyzer • Other function • OLAP service, cube exploration, statistical analysis • Sequential pattern analysis • Visual classification

  20. Summary • The benefits of knowing one’s business is critical; technologies are coming together to support data mining. • Data mining is the process and result of knowledge production, knowledge discovery and knowledge management.

More Related