1 / 197

ECOM 6349 Selected Topics in AI: Data Clustering

ECOM 6349 Selected Topics in AI: Data Clustering Most of these slides are given from internet – Authors: Tan, Steinbach, Kumar : Jiawei Han. Syllabus. Syllabus. What is Cluster Analysis?.

dstrom
Télécharger la présentation

ECOM 6349 Selected Topics in AI: Data Clustering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ECOM 6349 Selected Topics in AI: Data Clustering Most of these slides are given from internet – Authors: Tan, Steinbach, Kumar : Jiawei Han

  2. Syllabus

  3. Syllabus

  4. What is Cluster Analysis? Clustering is unsupervised learning: no predefined classes Cluster: a collection of data objects Objects are similar to objects in same cluster Objects are dissimilar to objects in other clusters Cluster analysis Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from (or unrelated to) the objects in other groups

  5. Inter-cluster distances are maximized Intra-cluster distances are minimized What is Cluster Analysis?

  6. How many clusters? Six Clusters Two Clusters Four Clusters What is Cluster Analysis?

  7. What is Cluster Analysis? • Exclusive versus non-exclusive • In non-exclusive clustering, points may belong to multiple clusters • Partial versus complete • In some cases, we only want to cluster some of the data • Heterogeneous versus homogeneous • Cluster of widely different sizes, shapes, and densities

  8. What is Cluster Analysis? • Hard Clustering versus Fuzzy Clustering • In fuzzy clustering, a point belongs to every cluster with some weight between 0 and 1 • Weights must sum to 1 • Probabilistic clustering has similar characteristics

  9. What Is Good Clustering? • A good clustering method will produce high quality clusters with • high intra-class similarity • low inter-class similarity • The quality of a clustering result depends on the similarity measure used by the method. • The quality of a clustering method is also measured by its ability to discover some or all of the hidden patterns.

  10. Vocabulary of Clustering • Records, data points, samples, items, objects, patterns… • Attributes, features, variables… • Similarity, dissimilarity, distances. • Centre, Centroid, Prototype. • Hard Clustering (Crisp Clustering)

  11. Requirements of Clustering • Scalability • Ability to deal with different types of attributes • Discovery of clusters with arbitrary shape • Minimal requirements for domain knowledge to determine input parameters • Able to deal with noise and outliers • Insensitive to order of input records • Insensitive to the initial conditions • High dimensionality

  12. Clustering Algorithms

  13. Clustering Algorithms

  14. Data Representation • Data matrix (two mode) • N objects with p attributes • Dissimilarity matrix (one mode) • d(i,j) : dissimilarity between i and j • with p attributes

  15. How to deal with missing values?

  16. Types of Clusters: Well-Separated • Well-separated clusters • A cluster is a set of points such that any point in a cluster is closer (or more similar) to every other point in the cluster than to any point not in the cluster 3 well-separated clusters

  17. Types of Clusters: Center-Based • Center-based • A cluster is a set of objects such that an object in a cluster is closer (more similar) to the “center” of a cluster, than to the center of any other cluster • The center of a cluster is often a centroid, the average of all the points in the cluster, or a medoid, the most “representative” point of a cluster 4 center-based clusters

  18. Types of Clusters: Contiguity-Based • Contiguous Cluster (Nearest neighbor or Transitive) • A cluster is a set of points such that a point in a cluster is closer (or more similar) to one or more other points in the cluster than to any point not in the cluster. 8 contiguous clusters

  19. Types of Clusters: Density-Based • Density-based • A cluster is a dense region of points, which is separated by low-density regions, from other regions of high density. • Used when the clusters are irregular or intertwined, and when noise and outliers are present. 6 density-based clusters

  20. Types of Clusters: Conceptual Clusters • Shared Property or Conceptual Clusters • Finds clusters that share some common property or represent a particular concept. 2 Overlapping Circles

  21. Types of Clusters: Objective Function • Clusters Defined by an Objective Function • Finds clusters that minimize or maximize an objective function. • Enumerate all possible ways of dividing the points into clusters and evaluate the `goodness' of each potential set of clusters by using the given objective function.

  22. Type of data in clustering analysis

  23. Symbol Table

  24. Symbol Table

  25. Frequency Table

  26. Frequency Table January 4, 2020 26

  27. Frequency Table January 4, 2020 27

  28. Frequency Table January 4, 2020 28

  29. Type of data in clustering analysis • Binary variables • Nominal variables • Ordinal variables • Interval-scaled variables • Ratio variables • Variables of mixed types

  30. Binary variables The binary variable is symmetric (Simple match coefficient) The binary variable is asymmetric (Jaccard coefficient) Object j Object i January 4, 2020 30

  31. Binary variables January 4, 2020 31

  32. Dissimilarity between Binary Variables Example gender is a symmetric attribute the remaining attributes are asymmetric binary let the values Y and P be set to 1, and the value N be set to 0 January 4, 2020 32

  33. Nominal Variables A generalization of the binary variable in that it can take more than 2 states, e.g., red, yellow, blue, green Method 1: Simple matching m: # of matches, p: total # of variables Method 2: use a large number of binary variables creating a new binary variable for each of the M nominal states January 4, 2020 33

  34. Nominal Variables Examples Eye Color Days of the week Religion Seasons Job title January 4, 2020 34

  35. Nominal Variables • Find the Proximity Matrix? January 4, 2020 35

  36. Ordinal Variables Order is important, e.g., rank Can be treated like interval-scaled replacing xif by their rank map the range of each variable onto [0, 1] by replacing i-th object in the f-th variable by compute the dissimilarity using methods for interval-scaled variables January 4, 2020 36

  37. Ordinal Variables • Find the Proximity Matrix? January 4, 2020 37

  38. Interval-valued variables Examples Temperature Weight Time Age Length January 4, 2020 38

  39. Interval-valued variables Standardize data Calculate the mean absolute deviation: where Calculate the standardized measurement (z-score) Using mean absolute deviation is more robust than using standard deviation January 4, 2020 39

  40. Ratio-Scaled Variables Ratio-scaled variable: a positive measurement on a nonlinear scale, approximately at exponential scale, such as AeBt or Ae-Bt Methods: treat them like interval-scaled variables — not a good choice! (why?) apply logarithmic transformation yif = log(xif) treat them as continuous ordinal data treat their rank as interval-scaled. January 4, 2020 40

  41. Ratio-Scaled Variables • Find the Proximity Matrix? January 4, 2020 41

  42. Variables of Mixed Types • A database may contain all the six types of variables • symmetric binary, asymmetric binary, nominal, ordinal, interval and ratio. • One may use a weighted formula to combine their effects. • f is binary or nominal: dij(f) = 0 if xif = xjf , or dij(f) = 1 o.w. • f is interval-based: use the normalized distance • f is ordinal or ratio-scaled • compute ranks rif and • and treat zif as interval-scaled

  43. Variables of Mixed Types • Find the Proximity Matrix? January 4, 2020 43

  44. Scale Conversion - Interval to Ordinal (a) No substantive change necessary. (b) Substitution. (c) Equal length categories. (d) Equal membership categories. (g) One-dimensional hierarchical linkage methods. (h) Ward’s hierarchical clustering method. (i) Iterative improvement of a partition.

  45. Scale Conversion - Interval to Nominal - Ordinal to Nominal - Nominal to Ordinal: 1. Correlation with an interval variable Let X be a nominal variable with g classes and Y be an interval variable. Where is the number of observations in the class and is the observation in the class.

  46. Scale Conversion - Nominal to Ordinal (Continue): 2. Rank correlation and mean ranks Obviously, maximizing the rank correlation r is equivalent to minimizing

  47. Scale Conversion - Ordinal to Interval 1- Mapping (Normalization) 2- Class ranks 3- Correlation

  48. Categorization of Numerical Data - Direct Categorization

  49. Categorization of Numerical Data - Direct Categorization (continue) (3.2)

  50. Categorization of Numerical Data - Direct Categorization (continue)

More Related