SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

CIS671-Knowledge Discovery and Data Mining

delta avatar
delta
33 diapositives · 490 Views · Sep 05, 2014

CIS671-Knowledge Discovery and Data Mining. AI reminders. Vasileios Megalooikonomou Dept. of Computer and Information Sciences Temple University. (based on notes by Christos Faloutsos). Agenda. Statistics AI - decision trees Problem Approach Conclusions DB. Decision Trees.

CIS671-Knowledge Discovery and Data Mining - slide 1
01 / 33CIS671-Knowledge Discovery and Data Mining
1 / 17

Télécharger la présentation

CIS671-Knowledge Discovery and Data Mining
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. CIS671-Knowledge Discovery and Data Mining AI reminders Vasileios Megalooikonomou Dept. of Computer and Information Sciences Temple University (based on notes by Christos Faloutsos)

  2. Agenda • Statistics • AI - decision trees • Problem • Approach • Conclusions • DB

  3. Decision Trees • Problem: Classification - Ie., • given a training set (N tuples, with M attributes, plus a label attribute) • find rules, to predict the label for newcomers Pictorially:

  4. Decision trees ??

  5. Decision trees • Issues: • missing values • noise • ‘rare’ events

  6. Decision trees • types of attributes • numerical (= continuous) - eg: ‘salary’ • ordinal (= integer) - eg.: ‘# of children’ • nominal (= categorical) - eg.: ‘car-type’

  7. num. attr#2 (eg., chol-level) - - + + + - + - + - + - + num. attr#1 (eg., ‘age’) Decision trees • Pictorially, we have

  8. Decision trees • and we want to label ‘?’ ? num. attr#2 (eg., chol-level) - - + + + - + - + - + - + num. attr#1 (eg., ‘age’)

  9. Decision trees • so we build a decision tree: ? num. attr#2 (eg., chol-level) - - + + + 40 - + - + - + - + 50 num. attr#1 (eg., ‘age’)

  10. Decision trees • so we build a decision tree: age<50 N Y chol. <40 + Y N - ...

  11. Agenda • Statistics • AI - decision trees • Problem • Approach • Conclusions • DB

  12. Decision trees • Typically, two steps: • tree building • tree pruning (for over-training/over-fitting)

  13. num. attr#2 (eg.,chol-level) - - + + + - + - + - + - + num. attr#1 (eg., ‘age’) Tree building • How?

  14. Tree building • How? • A: Partition, recursively - pseudocode: Partition ( dataset S) if all points in S have same label then return evaluate splits along each attribute A pick best split, to divide S into S1 and S2 Partition(S1); Partition(S2) • Q1: how to introduce splits along attribute Ai • Q2: how to evaluate a split?

  15. Tree building • Q1: how to introduce splits along attribute Ai • A1: • for numerical attributes: • binary split, or • multiple split • for categorical attributes: • compute all subsets (expensive!), or • use a greedy approach

  16. Tree building • Q1: how to introduce splits along attribute Ai • Q2: how to evaluate a split?

  17. Tree building • Q1: how to introduce splits along attribute Ai • Q2: how to evaluate a split? • A: by how close to uniform each subset is - ie., we need a measure of uniformity:

  18. Tree building entropy: H(p+, p-) Any other measure? p+: relative frequency of class + in S 1 0 0.5 0 1 p+

  19. 1 0 0.5 0 1 p+ Tree building entropy: H(p+, p-) ‘gini’ index: 1-p+2 - p-2 p+: relative frequency of class + in S 1 0 0.5 0 1 p+

  20. Tree building entropy: H(p+, p-) ‘gini’ index: 1-p+2 - p-2 (How about multiple labels?)

  21. Tree building Intuition: • entropy: #bits to encode the class label • gini: classification error, if we randomly guess ‘+’ with prob. p+

  22. num. attr#2 (eg., chol-level) - - + + + - + - + - + - + num. attr#1 (eg., ‘age’) Tree building Thus, we choose the split that reduces entropy/classification-error the most: Eg.:

  23. Tree building • Before split: we need (n+ + n-) * H( p+, p-) = (7+6) * H(7/13, 6/13) bits total, to encode all the class labels • After the split we need: 0 bits for the first half and (2+6) * H(2/8, 6/8) bits for the second half

  24. Agenda • Statistics • AI - decision trees • Problem • Approach • tree building • tree pruning • Conclusions • DB

  25. num. attr#2 (eg., chol-level) - - + + + - + - ... + - + - + num. attr#1 (eg., ‘age’) Tree pruning • What for?

  26. num. attr#2 (eg., chol-level) - - + + + - + - ... + - + - + num. attr#1 (eg., ‘age’) Tree pruning • Q: How to do it?

  27. Tree pruning • Q: How to do it? • A1: use a ‘training’ and a ‘testing’ set - prune nodes that improve classification in the ‘testing’ set. (Drawbacks?)

  28. Tree pruning • Q: How to do it? • A1: use a ‘training’ and a ‘testing’ set - prune nodes that improve classification in the ‘testing’ set. (Drawbacks?) • A2: or, rely on MDL (= Minimum Description Language) - in detail:

  29. Tree pruning • envision the problem as compression (of what?)

  30. Tree pruning • envision the problem as compression (of what?) • and try to minimize the # bits to compress (a) the class labels AND (b) the representation of the decision tree

  31. (MDL) • a brilliant idea - eg.: best n-degree polynomial to compress these points: • the one that minimizes (sum of errors + n)

  32. Conclusions • Classification through trees • Building phase - splitting policies • Pruning phase (to avoid over-fitting) • Observation: classification is subtly related to compression

  33. Reference • M. Mehta, R. Agrawal and J. Rissanen, `SLIQ: A Fast Scalable Classifier for Data Mining', Proc. of the Fifth Int'l Conference on Extending Database Technology, Avignon, France, March 1996