Attention and Selection: Part 2

1. Attention and Selection:Part 2

2. The increasingly important role played by objects in studies of visual attention • Miller’s ‘Magic Number 7’ has continued to haunt us even beyond studies of short-term memory (STM). • There is a limitation in visual information processing that is beyond the limitation of acuity and of channel capacity: The perceptual system is limited in what it can individuate and how many of these individuals it can deal with at one time. • The capacity to individuate is different from memory capacity and discrimination capacity. • This notion of individuating and of individuals may be related to Miller’s “chunks”, but it has a special role in vision which we will explore later. A chunk is a relatively ill-defined notion in general whereas the units of visual attention are better thought of as objects

3. Experimental evidence for attentional selection of objects • Single Object Advantage: pairs of judgments are faster when both apply to the same perceived object • Entire objects acquire enhanced sensitivity from focal attention to a part of the object • Single-Object advantage occurs even with generalized “objects” defined in feature space • Simultanagnosia and hemispatial neglect show object-based effect • Attention moves with Moving Objects • IOR • Object Files • MOT

4. Single-object superiority even when the shapes are controlled

5. More controls for the Baylis study… (Baylis, 1994) Controls for separability, convexity, area…

7. There is also evidence from neuropsychology that is consistent with the object-based view • Neglect • Balint and simultanagnosic patients

8. Visual neglect syndrome is object-based When a right neglect patient is shown a dumbbell that rotates, the patient continues to neglect the object that had been on the right, even though It is now on the left (Behrmann & Tipper, 1999).

9. Simultanagnosic (Balint Syndrome) patients only attend to one object at a time Simultanagnosic patients cannot judge the relative length of two lines, but they can tell that a figure made by connecting the ends of the lines is not a rectangle but a trapezoid (Holmes & Horax, 1919).

10. Balint patients can only attend to one object at a time even if they are overlapping Luria, 1959

11. Objecthood endures over time Picking out objects is an example of the parsing of a scene into things that are likely to be physical objects. But the same must occur in time – temporal parsing entails solving the correspondence problem Several studies have shown that what counts as an object (as the same object) endures over time and over changes in location; Certain forms of changes in location as well as disappearances in time preserve objecthood. This gives what we have been calling a “visual object” a real physical-object character and partly justifies our calling it an “object”.

12. The “Ternus Configuration” to demonstrate the early visual effect of objecthood Short time delays result in “element motion” in which the middle object persists as the “same object” and does not appear to move so the end objects appear to move

13. Long time delays results in “group motion” in which the middle object does not persist but is perceived as a new object each time it reappears

14. Inhibition of return appears to be object-based(as well as to some extent location-based) • The original study used static objects. Then (Tipper, Driver & Weaver, 1991) showed that IOR moves with the inhibited object.

15. IOR appears to be object-based (it travels with the object that was attended)

16. Objects endure despite changes in location; and they carry their history with them! Object File Theory of Kahneman & Treisman Letters are faster to read if they appear in the same box where they appeared initially. Priming travels with the object. According to the theory, when an object first appears, a file is created for it and the properties of the object are encoded and subsequently accessed through this object-file.

17. Demo of Object File Experiment Object File 1 Object File 2

18. The limitation of individuation and selection • There are obviously limitations on the input side of vision that depend on the acuity of the sensors and the range of physical properties to which they respond. • But there is a limitation beyond that of acuity: The perceptual system is limited in what it can individuate and how many of these individuals it can deal with at one time. The capacity to individuate is different from the capacity to discriminate. • There is reason to think that individuating is a separate and distinct process in early vision

19. Picking out is different from discriminating:Pick out the third contour from the left

20. Individuating as a distinct process • Individuating has its own psychometric function: The minimum distance for individuating is much larger than for discriminating. • It may be that in vision our attention is limited in the number of things we can individuate and simultaneously access (more on this later). But how do you determine what counts as a “thing”? • Individuating is a prerequisite for recognition of patterns and other properties defined among a number of individual parts • An example of how we can easily detect patterns if they are defined over a small enough number of parts is subitizing • Another area where the concept of an individual has become important is in cognitive development, where it is clear that babies are sensitive to the numerosity of individual things in a way that is distinct from their perceptual abilities but is limited in its capacity

21. Pick out 3 dots and keep track of them • In a field of identical elements you can select a number of them and move your attention among them (e.g., “move one up” or Move 2 right” etc) so long as at no time do you have to hold on to more than 4 dots

22. Pick out 3 dots I will cue and keep track of them • After you pick out the 3 cued dots, I’ll ask you move your attention from the center one. Describe the new relation among the three dots. • In a field of identical elements you can select several of them and move your attention among them (e.g., “move one up” or Move 2 right” etc) so long as at no time do you have to hold on to more than 4 dots (Intriligator & Cavanagh, 2001)

23. Individuals and patterns • Vision does not recognize patterns by applying templates since the size, shape, retinal location, orientation, and other properties must be abstracted away, • A pattern is encoded over time (and often over saccades), therefore the visual system must keep track of the individual parts and merge descriptions of the same part at different times and stages of encoding • Therefore in order to recognize a pattern, the visual system must pick out individual parts and bind them to the representation being constructed • Examples include what Ullman called “visual routines”

24. Are there collinear items (n>3)?

25. Several objects must be picked out at once in making relational judgments • The same is true for other relational judgments like inside or on-the-same-contour… etc. We must pick out the relevant individual objects first. Respond: Inside-same contour? On-same contour?

26. When items cannot be individuated, predicates over them cannot be evaluated Do these figures contain one or two distinct curves? Individuating these curves requires a “curve tracing” operation, so Number_of_curves (C1, C2, …) takes time proportional to the length of the shortest curve.

27. The figure on the left is one continuous curve, the one on the right is two distinct curves – as shown in color.

28. Signature subitizing phenomena only appear when objects are automatically individuated and indexed Counting slope subitizing slope Trick, L. M., & Pylyshyn, Z. W. (1994). Why are small and large numbers enumerated differently? A limited capacity preattentive stage in vision. Psychological Review, 101(1), 80-102.

29. A different view of the role of attention • What’s the equivalent of “chunks” in vision (Visual Chunks?) • Attention as the “glue” that allows properties that occur together to be represented as conjoined • Experiments showing the special difficulty that vision has in detecting conjunctions of several properties have provided a basis for understanding an important problem in in visual analysis

30. Encoding conjunctions of properties • Experiments showing the special difficulty that vision has in detecting conjunctions of several properties have provided a basis for understanding an important problem in in visual analysis

31. How are conjunctions of features detected? Read the vertical line of digits in the following display Under these conditions Conjunction Errors are very frequent

32. Rapid visual search (Treisman) Find the following simple figure in the next slide:

33. This case is easy – and the time is independent of how many nontargets there are – because there is only one red item. This is called a ‘popout’ search

34. This case is also easy – and the time is independent of how many nontargets there are – because there is only one right-leaning item. This is also a ‘popout’ search.

35. Rapid visual search (conjunction) Find the following simple figure in the next slide:

37. Find the unique item in this slide

38. Serial vs parallel search? • Finding an element that differs from all others in a scene by a single feature – called a single-feature search – is fast, error-free and almost independent of how many nontargets there are; • Finding an object that differs from all others by a conjunction of two or more features (and that shares at least one feature with each object in the scene) – called a conjunction search – is usually slow, error-prone, and is worse the more nontargets there are in the scene*. • These results suggest that in order to find a conjunction, which requires solving the binding problem, attention has to be scanned serially to all objects. * This way of putting is simplifies things. Under certain conditions the serial-parallel distinction breaks down

39. Single-Feature vs Conjunction-feature search

40. Treisman’s Attention as Glue Hypothesis Another answer to what attention is for • The purpose of visual attention is to Bindproperties together in order to recognize objects • This is called the “binding problem” or the “many properties problem” and it is of considerable interest to philosophers as well as vision scientists • We can recognize not only the presence of “squareness” and “redness” in our field of view, but we can also distinguish between different ways they may be conjoined

41. Break 

42. The “Binding Problem” “Pandemonium” This sort of model of early vision does not work: it does not provide information that would allow the “cognitive demons” to solve the binding problem. Information about grouping (conjunction) of properties is lost.

43. How is the binding problem solved? • Here is the most common view. To determine whether properties P and Q are conjoined, you detect P, encode its location, then check whether Q is found at that location. • A more detailed model is Treisman’s Feature Integration Theory. It postulates separate maps for each feature type, as well as a master map that allows attention to direct the search for features.

44. The role of attention to location in Treisman’s Feature Integration Theory

45. Problems with solving the binding problem in terms of co-location of features • In order for the conjunction-by-location to work one has to have determined the location of each property • A punctate location will not do since properties have extension and the extension is relevant (cf encode INSIDE) • A region will not do unless one knows the boundaries of the region • In either case one has to have identified the relevant object before one can use its location • It’s the object that has the properties in question that determines whether the properties are conjoined • Properties are conjoined just in case they are properties of the same object

46. But in encoding properties, early vision can’t just bind them together according to their spatial co-occurrence – even their co-occurrence within the same region. That’s because the relevant region depends on the object. So the selection and binding must be according to the objects that have those properties

47. The problem is even worse when the relation between items is “Inside”

48. If co-location of properties will not give us a way of solving the binding problem, what will? • It’s not being at the same location that binds properties together, it’s being properties of the same object • This is why we need object-based selection and why the object-based attention literature is relevant …

49. An alternative view of how we solve the binding problem • If we assume that only properties of selected objects are encoded and that these are stored in object files associated with each object, then properties that belong to the same object are stored in the same object file, which is why they get bound together • This automatically solves the binding problem! • This is the view exemplified by both FINST Theory (1989) and Object File Theory (1992) to be described next • The assumption that only properties of selected objects are encoded raises the question of what happens to properties of the other objects or properties in a display (more on this later) • The logical answer is that they are not encoded and therefore not available to conceptualization and cognition • But this is counter-intuitive!

50. The FINST theory • Why do we need Indexes? • Some background on nonconceptual selection