Context-Aware Clustering

1. Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University

2. Contextual pattern and co-occurrences ? Spatial contexts provide useful cues for clustering

3. K-means revisit EM Update Binary label indicator Assumption: data samples are independent Limitation: contextual information of spatial dependency is not considered in clustering data samples

4. Clustering higher-level patterns • Regularized k-means Not a smooth term! Distortion in hamming space charactering contextual patterns Distortion in original feature space Same as traditional K-means clustering Regularization term due to contextual patterns

5. Chicken and Egg Problem • Hamming distance in clustering contextual patterns • Matrix form J1 is coupled with J2 • Cannot minimize J1 and J2 separately !

6. Decoupling Fix Update Fix Update

7. Nested-EM solution Nested E-step M-step Update and separately Theorem of convergence the nested-EM algorithm can converge in finite steps.

8. Simulation results (feature space)

9. Simulation results (spatial space)

11. K-means Initialization

12. 1st round Final Phrases

16. Multiple-feature clustering • Dataset: handwritten numerical (‘0’-‘9’) from UCI data set • Each digit has three different types of features • Contextual pattern corresponds to compositional feature • Different types of features serve as contexts of each other • Clustering each type of features into 10 “words” • Clustering 10 “phrases” based on a word-lexicon of size 3x10

17. Conclusion • A context-aware clustering formulation proposed • Targets on higher-level compositional patterns in terms of co-occurrences • Discovered contextual patterns can feed back to improve the primitive feature clustering • An efficient nested-EM solution which is guaranteed to converge in finite steps • Successful applications in image pattern discovery and multiple-feature clustering • Can be applied to other general clustering problems