Understanding Gestalt Principles in Image Segmentation and Boundary Detection

1. Segmentation and Grouping • Gestalt approach • Problem - We don’t perceive local events in an image - we perceive more global figures • Elucidate principles which determine grouping of local “things” in an image into figures • Proximity • Similarity • Pragnanz • good continuation • symmetry

2. Proximity

3. Similarity

4. Colinearity

6. Real world problems to which we can apply gestalt principles • Segmentation • determining where objects are in an image and what their boundaries are. • Grouping • grouping together stuff as part of the same object; for example, across occluders.

7. Boundary detection(Local)

8. Boundary detection(Local) • Luminance edges

9. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields

15. Finding meaningful contours in image • Local edge detection • Problems - false targets, misses

16. Finding meaningful contours in image • Local edge detection • Problems - false targets, misses • Solution 1: use other cues • Texture • Motion • Disparity

17. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields • Texture edges

18. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields • Texture edges • Sharp changes in orientation, scale of textures

22. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields • Texture edges • Sharp changes in orientation, scale of textures • Disparity edges Left eye Right eye

23. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields • Texture edges • Sharp changes in orientation, scale of textures • Disparity edges 000000000000000000000 000000000000000000000 000000222222222000000 000000222222222000000 000000222222222000000 000000222222222000000 000000000000000000000 000000000000000000000 = - Left eye Right eye

24. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields • Texture edges • Sharp changes in orientation, scale of textures • Disparity edges • Motion edges

25. Boundary detection (Local) • Luminance edges • Use center-surround receptive fields • Texture edges • Sharp changes in orientation, scale of textures • Disparity edges • Motion edges

30. Paradigm • Look for textures which “pop-out” to observers. • Characterize texture properties which support texture pop-out - fill in the blank: • A figure pops-out from the background if its __________ (property of texture) differs from that of the background. • Logic: • Pop-out is the result of automatic, low-level segmentation processes.

31. Texture properties which the visual system uses to do segmentation • Brightness • Contrast • Scale • Orientation

32. Segmentation by Contrast

33. Segmentation by scale

34. Segmentation by orientation

38. Real-world Justification for these properties • Most objects in a scene will differ in at least one, and probably more of these properties. • When an object’s texture doesn’t differ from that of it’s background it is camouflaged. • But why only these and not others?

41. Mechanisms for texture segmentation • Texture is a semi-local property of an image • Texture is the “micro-pattern” in an image • An individual point in an image cannot have texture, but a small region can. • Complex cells are good coders of texture properties • have local receptive fields, but aren’t sensitive to position of a pattern within the receptive fields • Signals how much oriented “stuff” falls within their receptive fields

42. Cortical images • Treat a set of cortical cells with the same receptive field properties as an image. The activity of the cell whose receptive field is centered at a given position of the visual field is the “intensity” of the cortical image. • Have cortical images for each combination of orientation and scale preferences.

43. Complex cell images • A cortical image made by looking at the firing rates of complex cells with the same orientation and scale preferences. • Example: • Fine-scale, vertical complex cell image - • firing rates of complex cells with small, vertical receptive fields. • An image of the fine-scale vertical “stuff” in an image.

47. Problem - • How does visual system resolve ambiguities in local measures of image intensity changes to decide what is part of a contour and what isn’t? • How does the visual system integrate local edge information into global figures? • Phenomenal window into visual processing of contours: • illusory contours and amodal completion

48. Illusory contours and amodal completion are flip sides of the same coin • Amodal completion - Filling in boundaries of objects behind occluders. • Illusory contours - Filling in boundaries OF occluders. • The appearance of illusory contours usually coincides with the appearance of amodally completed boundaries.