1 / 29
Télécharger la présentation

Three Challenges in Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Three Challenges in Data Mining Anne Denton Department of Computer Science NDSU

  2. Why Data Mining? • Parkinson’s Law of Data Data expands to fill the space available for storage • Disk-storage version of Moore’s law Capacity  2 t / 18 months • Available data grows exponentially!

  3. Outline • Motivation of 3 challenges • More records (rows) • More attributes (columns) • More subject domains • Some answers to the challenges • Thesis work • Generalized P-Tree structure • Kernel-based semi-naïve Bayes classification • KDD-cup 02/03 and with Csci 366 students • Data with graph relationship • Outlook: Data with time dependence

  4. Examples • More records • Many stores save each transaction • Data warehouses keep historic data • Monitoring network traffic • Micro sensors / sensor networks • More attributes • Items in a shopping cart • Keywords in text • Properties of a protein (multi-valued categorical) • More subject domains • Data mining hype increases audience

  5. Algorithmic Perspective • More records • Standard scaling problem • More attributes • Different algorithms needed for 1000 vs. 10 attributes • More subject domains • New techniques needed • Joining of separate fields • Algorithms should be domain-independent • Need for experts does not scale well • Twice as many data sets • Twice as many domain experts?? • Ignore domain knowledge? • No! Formulate it systematically

  6. Some Answers to Challenges • Large data quantity (Thesis) • Many records • P-Tree concept and its generalization to non-spatial data • Many attributes • Algorithm that defies curse of dimensionality • New techniques / Joining separate fields • Mining data on a graph • Outlook: Mining data with time dependence

  7. Challenge 1: Many Records • Typical question • How many records satisfy given conditions on attributes? • Typical answer • In record-oriented database systems • Database scan: O(N) • Sorting / indexes? • Unsuitable for most problems • P-Trees • Compressed bit-column-wise storage • Bit-wise AND replaces database scan

  8. P-Trees: Compression Aspect

  9. P-Trees: Ordering Aspect • Compression relies on long sequences of 0 or 1 • Images • Neighboring pixels are probably similar • Peano-ordering • Other data? • Peano-ordering can be generalized • Peano-order sorting

  10. Peano-Order Sorting

  11. Impact of Peano-Order Sorting • Speed improvement especially for large data sets • Less than O(N) scaling for all algorithms

  12. So Far • Answer to challenge 1: Many records • P-Tree concept allows scaling better than O(N) for AND (equivalent to database scan) • Introduced effective generalization to non-spatial data (thesis) • Challenge 2: Many attributes • Focus: Classification • Curse of dimensionality • Some algorithms suffer more than others

  13. Curse of Dimensionality • Many standard classification algorithms • E.g., decision trees, rule-based classification • For each attribute 2 halves: relevant  irrelevant • How often can we divide by 2 before small size of “relevant” part makes results insignificant? • Inverse of • Double number of rice grains for each square of the chess board • Many domains have hundreds of attributes • Occurrence of terms in text mining • Properties of genes

  14. Possible Solution • Additive models • Each attribute contributes to a sum • Techniques exist (statistics) • Computationally intensive • Simplest: Naïve Bayes • x(k) is value of kth attribute • Considered additive model • Logarithm of probability additive

  15. Semi-Naïve Bayes Classifier • Correlated attributes are joined • Has been done for categorical data • Kononenko ’91, Pazzani ’96 • Previously: Continuous data discretized • New (thesis) • Kernel-based evaluation of correlation

  16. Results • Error decrease in units of standard deviation for different parameter sets • Improvement for wide range of correlation thresholds: 0.05 (white) to 1 (blue)

  17. So Far • Answer to challenge 1: More records • Generalized P-tree structure • Answer to challenge 2: More attributes • Additive algorithms • Example: Kernel-based semi-naïve Bayes • Challenge 3: More subject domains • Data on a graph • Outlook: Data with time dependence

  18. Standard Approach to Data Mining • Conversion to a relation (table) • Domain knowledge goes into table creation • Standard table can be mined with standard tools • Does that solve the problem? • To some degree, yes • But we can do better

  19. “Everything should be made as simple as possible, but not simpler” Albert Einstein

  20. Claim: Representation as single relation is not rich enough • Example: Contribution of a graph structure to standard mining problems • Genomics • Protein-protein interactions • WWW • Link structure • Scientific publications • Citations Scientific American 05/03

  21. Data on a Graph: Old Hat? • Common Topics • Analyze edge structure • Google • Biological Networks • Sub-graph matching • Chemistry • Visualization • Focus on graph structure • Our work • Focus on mining node data • Graph structure provides connectivity

  22. Protein-Protein Interactions • Protein data • From Munich Information Center for Protein Sequences (also KDD-cup 02) • Hierarchical attributes • Function • Localization • Pathways • Gene-related properties • Interactions • From experiments • Undirected graph

  23. Questions • Prediction of a property (KDD-cup 02: AHR*) • Which properties in neighbors are relevant? • How should we integrate neighbor knowledge? • What are interesting patterns? • Which properties say more about neighboring nodes than about the node itself? But not: *AHR: Aryl Hydrocarbon Receptor Signaling Pathway

  24. Possible Representations • OR-based • At least one neighbor has property • Example: Neighbor essential true • AND-based • All neighbors have property • Example: Neighbor essential false • Path-based (depends on maximum hops) • One record for each path • Classification: weighting? • Association Rule Mining: Record base changes AHR essential AHR essential AHR not essential

  25. Association Rule Mining • OR-based representation • Conditions • Association rule involves AHR • Support across a link greater than within a node • Conditions on minimum confidence and support • Top 3 with respect to support: (Results by Christopher Besemann, project CSci 366)

  26. Classification Results • Problem (especially path-based representation) • Varying amount of information per record • Many algorithms unsuitable in principle • E.g., algorithms that divide domain space • KDD-cup 02 • Very simple additive model • Based on visually identifying relationship • Number of interacting essential genes adds to probability of predicting protein as AHR

  27. KDD-Cup 02: Honorable Mention NDSU Team

  28. Outlook: Time-Dependent Data • KDD-cup 03 • Prediction of citations of scientific papers • Old: Time-series prediction • New: Combination with similarity-based prediction

  29. Conclusions and Outlook • Many exciting problems in data mining • Various challenges • Scaling of existing algorithms (more records) • Different properties in algorithms become relevant (more attributes) • Identifying and solving new domain-independent challenges (more subject areas) • Examples of general structural components that apply to many domains • Graph-structure • Time-dependence • Relationships between attributes

More Related