1 / 56

Unsupervised Learning of Hierarchical Spatial Structures

Unsupervised Learning of Hierarchical Spatial Structures. Devi Parikh , Larry Zitnick and Tsuhan Chen. Our visual world…. Intro Approach Results Conclusion. What is an object?. What is context?. … hierarchical spatial patterns. Goal. Intro Approach Results Conclusion.

edward
Télécharger la présentation

Unsupervised Learning of Hierarchical Spatial Structures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unsupervised Learning of Hierarchical Spatial Structures Devi Parikh, Larry Zitnick and Tsuhan Chen

  2. Our visual world… Intro Approach Results Conclusion What is an object? What is context? … hierarchical spatial patterns

  3. Goal Intro Approach Results Conclusion Unsupervised!

  4. Related work Intro Approach Results Conclusion [Todorovic 2008] [Zhu 2008] [Fidler 2007] Fully unsupervised Structure and parameters learnt From features to multiple objects [Sivic 2008]

  5. Model Rule based Intro Approach Results Conclusion c2 c3 0.1 r1 0.6 0.9 c4 0.6 0.7 c2 c1

  6. Model Rule based Intro Approach Results Conclusion c2 c3 0.1 r1 0.6 0.9 r2 0.6 0.7 c2 c1

  7. Model Hierarchical rule-based Intro Approach Results Conclusion c2 c3 0.1 r1 0.6 0.9 r2 0.6 0.7 c2 c1

  8. Model Intro Approach Results Conclusion • Rules R • Image-parts V • Codewords C • Features F

  9. Model • Notation Intro Approach Results Conclusion V = {v} instantiated image-parts rv rule corresponding to instantiated part v Ch(rv) = {x} children of rule rv  includes instantiated children Ch(v)  and un-instantiated children

  10. Model Intro Approach Results Conclusion

  11. Inference Intro Approach Results Conclusion

  12. Inference Intro Approach Results Conclusion

  13. Inference Intro Approach Results Conclusion

  14. Inference Intro Approach Results Conclusion

  15. Inference Intro Approach Results Conclusion

  16. Inference Intro Approach Results Conclusion

  17. Inference Intro Approach Results Conclusion

  18. Inference Intro Approach Results Conclusion

  19. Inference Intro Approach Results Conclusion

  20. Inference Intro Approach Results Conclusion

  21. Inference Intro Approach Results Conclusion Minimum Cost Steiner Tree Charikar1998

  22. Inference Intro Approach Results Conclusion

  23. Inference Intro Approach Results Conclusion Generalized distance transform Felzenszwalb et al. 2001

  24. Learning • EM style • Initialize rules • Infer rules • Update parameters • Modify rules Intro Approach Results Conclusion

  25. Learning • Initialize rules Intro Approach Results Conclusion …

  26. Learning • Inference Intro Approach Results Conclusion …

  27. Learning • Inference Intro Approach Results Conclusion …

  28. Learning • Add children Intro Approach Results Conclusion …

  29. Learning • Add children • Update parameters • Pruning children • Removing rules Intro Approach Results Conclusion …

  30. Learning • Adding rules Randomly add rules Intro Approach Results Conclusion … …

  31. Behavior • Competition among rules • Competition with root (noise) Intro Approach Results Conclusion

  32. Behavior • Competition among rules • Competition with root (noise) • Dropping children and rules • Number of children • Structure of DAG and tree • # rules, parameters, structure learnt automatically • Multiple instantiations of rules • Multiple children with same appearance Intro Approach Results Conclusion

  33. Intro Approach Results Conclusion Experiment 1: Faces & Motorbikes

  34. Faces & Motorbikes • Faces and Motorbikes • SIFT (200 words) • Learnt 15 L1 rules, 2 L2 rules • Each L1 rule  average ~7 children • Each L2 rule  average ~4 children Intro Approach Results Conclusion

  35. Example rules Intro Approach Results Conclusion

  36. Patches Intro Approach Results Conclusion

  37. Localization behavior Intro Approach Results Conclusion

  38. Categorization behavior Intro Approach Results Conclusion code-words first level rules second level rules occurrence Faces Faces Faces Motorbikes Motorbikes Motorbikes

  39. Categorization behavior Intro Approach Results Conclusion Kmeans PLSA SVM Words Rules Tree Words: 94 % Tree: 100%

  40. Edge features Intro Approach Results Conclusion Words: 55 % Tree: 82%

  41. Intro Approach Results Conclusion Experiment 2: Six categories

  42. Six categories Intro Approach Results Conclusion Words: 87 % Tree: 95 % 61 L1 rules (~9 children) 12 L2 rules (~3 children) Kim 2008: 95 %

  43. Intro Approach Results Conclusion Experiment 3: Scene categories

  44. Scene categories Intro Approach Results Conclusion Image Segmentation Codeword Mean color

  45. Outdoor scenes Intro Approach Results Conclusion rules images

  46. Intro Approach Results Conclusion Experiment 4: Structured street scenes

  47. Windows Intro Approach Results Conclusion

  48. Object categories Intro Approach Results Conclusion

  49. Object categories Intro Approach Results Conclusion

  50. Object categories Intro Approach Results Conclusion

More Related