Computational Vision CSCI 363, Fall 2018 Lecture 23 Perceptual Organization

1. Computational VisionCSCI 363, Fall 2018Lecture 23Perceptual Organization

2. Organizing the Scene Once you have detected oriented lines and determined their depth, color and motion, you must determine which parts of the image go with which other parts. Stages of perceptual organization: Image Segmentation: Divide the image into parts. Grouping: Group together parts that belong together. Parsing: Locate distinct pieces of the segmented regions.

3. Figure/Ground Once we have segmented the image into distinct regions, we must determine which regions are objects, or figures, and which regions are background, or ground. How do we determine which is which?

4. Study with Ambiguous figures Our interpretation of this image changes as we alternate our perception of which is figure and which is ground. Note that we do not see both images as figure at the same time.

5. Properties of Figure and Ground Figure Ground Thinglike Not thinglike Closer to the observer Further from the observer Bounded by contour Extends behind contour Shape defined by contour No shape at contour

6. Principles of Figure/Ground organization Surroundedness: Surrounded region perceived as figure. 2. Size: Smaller regions perceived as figure. 3. Orientation: Vertical and Horizontal regions perceived as figure. 4. Contrast: Region with greatest contrast with surround is figure. Symmetry 6. Convexity 7. Parallelism Note that these are not all or nothing effects. It is not known what the relative strengths of these effects are. They are qualitative results.

7. Meaningfulness If one region can be interpreted as a familiar object, it will tend to be seen as the figure. Problem: It is thought that figure/ground distinction must occur before object recognition can happen. The above observation suggests that object recognition can occur prior to figure/ground distinctions.

8. Physiology Lamme and colleagues have shown that some cells in V1 respond more strongly to an oriented edge or grating if it is part of the figure than if it is part of the surround. This is true even when the edges determining the figure/ground separation are far from the cell's receptive field.

9. Visual Interpolation • The visual system is very good at filling in gaps in the visual information available. This filling-in is known as interpolation. • Evidence: • Blind spot: We are unaware of our blind spot, because the gap in our visual field is filled in. • Stroke patients: Some stroke patients lose vision in large portions of the visual field, but are unaware of this loss. The extent of the gap can be determined with careful clinical tests. • Peripheral vision: We only have finely detailed vision in the center of our visual field. The peripheral image is quite blurry. Our visual system "fills in" details so we think we have a detailed view of the entire scene.

10. Occlusion In a typical real scene, many objects are at least partially blocked, or occluded, by other objects. We have no problem perceiving the entire object. Our visual system automatically fills in the missing part. This is known as visual completion or amodal completion. Example: We perceive this as a circle that is partially occluded by a square.

11. Rules for completion How does the visual system know what the completed shapes will look like? Familiarity: Familiarity is not required, but it helps.

12. Other Possible Rules Gestaltists believed that a figure could have a measure of "goodness" or "pragnanz". Problem: How do you measure goodness? Example: The more axes of symmetry, the better the figure. Counter example: Don't see hexagon in figure below. Kellman and Shipley developed a set of rules for relating pairs of discontinuous edges to one another for use in visual completion. The theory is a bit more quantitative than the others, but it still has difficulties explaining all the details of visual completion.

13. Illusory Contours The Kanisza Triangle Certain configurations cause us to see lines that are not actually present in the image.

14. Relation to Visual Completion We only see illusory contours when some pieces of the image appear to be occluded. The visual system "completes" the occluded regions.

15. Physiology Von der Heyt found cells in V2 that respond to illusory contours.

16. Responses of cells

17. Model for V2 response to illusory contours. This model makes use of end-stopped cells lined up along the contour.

18. Fun with Size Constancy The Ponzo illusion The Shepard illusion

19. Organizing the Scene Once you have detected oriented lines and determined their depth, color and motion, you must determine which parts of the image go with which other parts. Stages of perceptual organization: Image Segmentation: Divide the image into parts. Grouping: Group together parts that belong together. Parsing: Locate distinct pieces of the segmented regions.

20. 2 regions 3 regions 3 regions Segmentation A region = A bounded 2-D area that is a spatial subset of the image. How do we define these: Uniform connectedness: Find regions of uniform image properties (luminance, color, texture, etc.) • Boundary Based Segmentation • Use an edge detection algorithm to find closed contours. • When the edges form a closed contour, this divides the image into 2 regions: a bounded inside and an outside region. • Marr-Hildreth edge detection (zero crossings): Guaranteed to form closed contours. • Other methods may or may not generate closed contour.

21. Problems with edges One problem with edge-based methods is that there may be many spurious edges in the image from shadows and highlights. There needs to be a way to find connected surfaces in spite of lighting conditions. A second problem with edge based methods occurs when one object partially occludes another. We still tend to perceive the whole occluded object.

22. Region based approaches Region based approaches identify regions by finding sets of pixels that are most similar within a region and most dissimilar between regions. Note that the outline of the regions found is reasonably true to what we perceive. The edge detection algorithm (Canny) does not do as well. Edges Regions

23. Evidence for Edges The human visual system probably makes use of edges for image segmentation. Evidence: If you stabilize an edge on the retina, it disappears. Image segmentation also disappears. Experiment: Red circle in green annulus. Stabilize inner edge, but not outer edge. Red circle disappears!