mmcconnell
Uploaded by
94 SLIDES
1038 VUES
950LIKES

Quantitative Cluster Validation Methods in Data Analytics

DESCRIPTION

Learn about cluster validation for clustering algorithms, comparing clusters, measuring clustering validity, and determining the number of clusters in data sets. Explore internal indexes like F-ratio, Davies-Bouldin index, and Silhouette Coefficient for assessing clustering quality. Discover how to validate clustering results quantitatively and objectively.

1 / 94

Télécharger la présentation

Quantitative Cluster Validation Methods in Data Analytics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Speech and Image Processing UnitSchool of Computing University of Eastern Finland Cluster validation Clustering methods: Part 3 Pasi Fränti 15.4.2014

  2. Part I:Introduction

  3. P Cluster validation Precision = 5/5 = 100% Recall = 5/7 = 71% Supervised classification: • Class labels known for ground truth • Accuracy, precision, recall Cluster analysis • No class labels Validation need to: • Compare clustering algorithms • Solve number of clusters • Avoid finding patterns in noise Oranges: Apples: Precision = 3/5 = 60% Recall = 3/3 = 100%

  4. Internal Index: Validate without external info With different number of clusters Solve the number of clusters External Index Validate against ground truth Compare two clusters:(how similar) Measuring clustering validity ? ? ? ?

  5. Clustering of random data Random Points DBSCAN K-means Complete Link

  6. Cluster validation process • Distinguishing whether non-random structure actually exists in the data (one cluster). • Comparing the results of a cluster analysis to externally known results, e.g., to externally given class labels. • Evaluating how well the results of a cluster analysis fit the data without reference to external information. • Comparing the results of two different sets of cluster analyses to determine which is better. • Determining the number of clusters.

  7. m* Partitions P Codebook C INPUT: DataSet(X) Clustering Algorithm Validity Index Different number of clustersm Cluster validation process • Cluster validation refers to procedures that evaluate the results of clustering in a quantitative and objective fashion. [Jain & Dubes, 1988] • How to be “quantitative”: To employ the measures. • How to be “objective”: To validate the measures!

  8. Part II:Internal indexes

  9. Internal indexes • Ground truth is rarely available but unsupervised validation must be done. • Minimizes (or maximizes) internal index: • Variances of within cluster and between clusters • Rate-distortion method • F-ratio • Davies-Bouldin index (DBI) • Bayesian Information Criterion (BIC) • Silhouette Coefficient • Minimum description principle (MDL) • Stochastic complexity (SC)

  10. The more clusters the smaller the MSE. Small knee-point near the correct value. But how to detect? Knee-point between 14 and 15 clusters. Mean square error (MSE)

  11. 5 clusters 10 clusters Mean square error (MSE)

  12. Inter-cluster variance is maximized Intra-cluster variance is minimized From MSE to cluster validity • Minimize within cluster variance (MSE) • Maximize between cluster variance

  13. Biggest jump on 15 clusters. Jump point of MSE(rate-distortion approach) First derivative of powered MSE values:

  14. Sum-of-squares based indexes SSW / k ---- Ball and Hall (1965) k2|W| ---- Marriot (1971) ---- Calinski & Harabasz (1974) log(SSB/SSW) ---- Hartigan (1975) ---- Xu (1997) (d is the dimension of data; N is the size of data; k is the number of clusters) • SSW = Sum of squares within the clusters (=MSE) • SSB = Sum of squares between the clusters

  15. Variances Within cluster: Between clusters: Total Variance of data set: SSB SSW

  16. F-ratio variance test • Variance-ratio F-test • Measures ratio of between-groups variance against the within-groups variance (original f-test) • F-ratio (WB-index): SSB

  17. Calculation of F-ratio

  18. F-ratio for dataset S1

  19. F-ratio for dataset S2

  20. F-ratio for dataset S3

  21. F-ratio for dataset S4

  22. Extension of the F-ratio for S3

  23. Sum-of-square based index SSW / SSB & MSE SSW / m log(SSB/SSW) m* SSW/SSB

  24. Davies-Bouldin index (DBI) • Minimize intra cluster variance • Maximize the distance between clusters • Cost function weighted sum of the two:

  25. Davies-Bouldin index (DBI)

  26. Measured values for S2

  27. cohesion separation Silhouette coefficient[Kaufman&Rousseeuw, 1990] • Cohesion: measures how closely related are objects in a cluster • Separation: measure how distinct or well-separated a cluster is from other clusters

  28. Silhouette coefficient • Cohesion a(x): average distance of x to all other vectors in the same cluster. • Separation b(x): average distance of x to the vectors in other clusters. Find the minimum among the clusters. • silhouettes(x): • s(x) = [-1, +1]: -1=bad, 0=indifferent, 1=good • Silhouette coefficient (SC):

  29. x x cohesion separation Silhouette coefficient a(x): average distance in the cluster b(x): average distances to others clusters, find minimal

  30. Performance of Silhouette coefficient

  31. BIC= Bayesian Information Criterion L(θ) -- log-likelihood function of all models; n -- size of data set; m -- number of clusters Under spherical Gaussian assumption, we get : Formula of BIC in partitioning-based clustering d -- dimension of the data set ni -- size of the ith cluster ∑ i -- covariance of ith cluster Bayesian information criterion (BIC)

  32. Knee Point Detection on BIC Original BIC = F(m) SD(m) = F(m-1) + F(m+1) – 2∙F(m)

  33. Internal indexes

  34. Internal indexes Soft partitions

  35. Comparison of the indexesK-means

  36. Comparison of the indexesRandom Swap

  37. Part III: Stochastic complexity for binary data

  38. Stochastic complexity • Principle of minimum description length (MDL): find clustering C that can be used for describing the data with minimum information. • Data = Clustering + description of data. • Clustering defined by the centroids. • Data defined by: • which cluster (partition index) • where in cluster (difference from centroid)

  39. Solution for binary data where This can be simplified to:

  40. Number of clusters by stochastic complexity (SC)

  41. Part IV:External indexes

  42. Measure the number of pairs that are in: Same class both in P and G. Same class in P but different in G. Different classes in P but same in G. Different classes both in P and G. Pair-counting measures G P a a b b d d c c

  43. Rand and Adjusted Rand index [Rand, 1971] [Hubert and Arabie, 1985] G P Agreement: a, d Disagreement: b, c a a b b d d c c

  44. External indexes If true class labels (ground truth) are known, the validity of a clustering can be verified by comparing the class labels and clustering labels. nij= number of objects in class i and cluster j

  45. Rand statisticsVisual example

  46. Pointwise measures

  47. Rand index(example) Rand index = (20+72) / (20+24+20+72) = 92/136 = 0.68 Adjusted Rand = (to be calculated) = 0.xx

  48. Pair counting Information theoretic Set matching External indexes

More Related