Grouping and Segmentation

1. Grouping and Segmentation • Previously: model-basedgrouping (e.g., straight line, figure 8…) • Now: general ``bottom-up” image organization (Like detecting-specific-brightness-patterns vs “interesting” patterns/interest points/corners)

2. Grouping • Grouping is the process of associating similar image features together

3. Grouping • Grouping is the process of associating similar image features together • The Gestalt School: • Proximity: nearby image elements tend to be grouped. • Similarity: similar image elements ... • Common fate: image elements with similar motion ... • Common region: image elements in same closed region... • Parallelism: parallel curves or other parallel image elements ... • Closure: image elements forming closed curves ... • Symmetry: symmetrically positioned image elements... • Good continuation: image elements that join up nicely ... • Familiar Conguration: image elements giving a familiar object...

8. Good Continuation

9. Good continuation

11. Common Form: (includes color and texture)

12. Connectivity

13. Symmetry

14. Symmetry

15. Convexity (stronger than symmetry?)

16. Good continuation also stronger than symmetry?

17. Closureclosed curves are much easier to pick out than open ones

18. Grouping and Segmentation • Segmentation is the process of dividing an image into regions of “related content” Courtesy Uni Bonn

19. Grouping and Segmentation • Segmentation is the process of dividing an image into regions of “related content” Courtesy Uni Bonn

20. Grouping and Segmentation • Both are ill defined problems “Related” or “similar” are “high level” concepts. Hard to apply directly to image data • How are they used? Are they “early” process that precedes recognizing objects? • A lot of research, some pretty good algorithms. No single best approach.

21. Boundaries

22. User can help by clicking on endpoints Problem: Find best path between two boundary points.

23. How do we decide how good a path is? Which of two paths is better?

24. Discrete Grid • Curve is good if it follows object boundary • So it should have: • Strong gradients along curve (on average) • Curve directions roughly perpendicular to gradients • (since these is usually true for boundaries) • Curve should be smooth, not jaggedSince: • Object boundaries typically smooth • Curves that follow “noise edges” typically jagged • So good curve has: • Low curvature (on average) along curve • Slow change in gradient direction along curve

25. Discrete Grid • How to find the best curve? • Approach • Define cost function on curves (Given any curve, rule for computing its cost) • Define so good curves have lower cost(non-wiggly curves passing mostly through edgels have low cost) • Search for best curve with lowest cost (minimize cost function over all curves)

26. Discrete Grid • How to find the best curve? • Approach • Define cost function on curves (Given any curve, rule for computing its cost) • Define so good curves have lower cost(non-wiggly curves passing mostly through edgels have low cost) • Search for best curve with lowest cost (minimize cost function over all curves) • How to compute cost? • - At every pixel in image, define costs for “curve fragments” connecting it to adjoining pixels • - Compute curve cost by summing up costs for fragments.

27. Defining cost: smoothness • Path: series of pixels , , … , , • Good curve has small curvature at each pixel • Small direction changes. • small on average (summing along curve) • Good curve has small changes in gradient direction One possible term in cost definition: Fragment Fragment Cost

28. Defining cost: smoothness • Path: series of pixels , , … , , • Good curve has small curvature at each pixel • Small direction changes. • small on average (summing along curve) • Good curve has small changes in gradient direction One possible term in cost definition: small when gradients at both nearly perpendicular to fragment direction Fragment Fragment Cost

29. Defining cost: smoothness • Path: series of pixels , , … , , • Good curve has small curvature at each pixel • Small direction changes. • small on average (summing along curve) • Good curve has small changes in gradient direction One possible term in cost definition: small when gradients at both nearly perpendicular to fragment direction Fragment Fragment Cost If this small at both i, i+1, then small at i. Low average value of this quantity  low average value (good curve)

30. Path Cost Function(better path has smaller cost) • Path: , , , , • Total cost: sum cost for each pixel over whole path One possible cost definition:

31. Path Cost Function(better path has smaller cost) • Path: , , , , • Total cost: sum cost for each pixel over whole path One possible cost definition: Small for high gradient, small direction change

32. How do we find the best Path? Computer Science… Remember: Curve is path through grid. Cost sums over each step of path. We want to minimize cost.

33. Map problem to Graph Each pixel is a node Edges connect pixels Edge weight = cost of fragment connecting the two pixels

34. Map problem to Graph Example Cost: Edge weight Edge weight = cost of fragment connecting the two pixels

35. Map problem to Graph Example Cost: Note: this is just an example of a cost function. The cost function actually used differs Edge weight Edge weight = cost of fragment connecting the two pixels

36. Algorithm: basic idea • Two steps: • Compute least costs for best paths from start pixel to all other pixels. • Given a specific end pixel, use the cost computation to get best path between start and end pixels.

37. Dijkstra’s shortest path algorithm 9 link cost 4 5 1 0 3 3 3 2 • Algorithm: compute least costs from start node to all other nodes • Iterateoutward from start node, adding one node at a time. • Cost estimates start high, nodes gradually “learn” their true cost • Always choosenext node so can calculate its correct cost.

38. Dijkstra’s shortest path algorithm 4 9 5 9 L 4 5 1 1 0 3 3 3 3 2 3 2 3 • Algorithm: compute least costs from start node to all other nodes • Iterateoutward from start node, adding one node at a time. • Cost estimates start high, gradually improve • Always choosenext node so can calculate its correct cost. • Example: choose left node L with cost 1. Min cost toL equals 1 • sinceany other path to L gets cost>1 on first step from start node

39. Dijkstra’s shortest path algorithm • Computes minimum costs from seed to every pixel • Running time (N pixels): O(N log N) using active priority queue (heap) fraction of a second for a typical (640x480) image • Then find best path from seed to any point in O(N).  Real time!

40. Intelligent Scissors

41. Results

42. Voting again • Recall Hough Transform • Idea: data votes for best model(eg, best line passing through data points) • Implementation • Choose type of model in advance (eg, straight lines) • Each data point votes on parameters of model (eg, location + orientation of line)

43. Another type of voting • Popularity contest • Each data point votes for others that it “likes,” that it feels “most related to” • “Most popular” points form a distinct group • (Like identifying social network from cluster of web links)

44. B “feels related” to A and votes for it. A B

45. C also “feels related” to A and votes for it. A C

46. So do D… A D

47. So do D and E A E

48. “Popular” points form a group, with “friends” reinforcing each other

49. feels unconnected to , doesn’t vote for it (or votes against)

50. also feels unconnected and doesn’t vote for