Vision after complete blindness: Mike May

1. Vision after complete blindness: Mike May Fine et al., Nature Neuroscience 2003; Robert Kurzon, ‘Crashing Through’

2. We tend to take normal perception for granted, while finding bizarre experiences of exceptional individuals intriguing… But understanding normal perception is the toughest and most important challenge for psychology Occasionally, though, an exceptional individual’s experience can advance our understanding of normal perception… For instance, the case of Mike May helps us not to take perception for granted

3. To what extent does visual processing rely on visual experience?

4. Ione Fine (UCSD USC)

5. Ione Fine (UCSD USC) Geoff Boynton (UCSD/Salk): fMRI+

6. Ione Fine (UCSD USC) Brian Wandell (Stanford) and Alex Wade, Alissa Brewer (Stanford) Geoff Boynton (UCSD/Salk): fMRI+

7. Ione Fine (UCSD USC) Brian Wandell (Stanford) Alex Wade, Alissa Brewer (Stanford) Geoff Boynton (UCSD/Salk): fMRI+ Stuart Anstis (UCSD)

8. Subject Mike May • Blinded by a chemical accident at age 3. • Light sensitive (no form vision) between ages of 3-43

9. Subject Mike May • Sight restored by a new procedure - • Corneal epithelial stem cell replacement

11. Resolution • 2 & 3d Form • Motion • Object/Face Recognition

12. Contrast-Sensitivity Function (CSF) Sensitivity Spatial frequency (cycles/degree) Campbell & Robson (1968) Resolution limit: 50cpd

13. Resolution limit < 2 cpd, despite good optics 2 1.5 Log sensitivity 1 MM (Post-operatively) Normal 0.5 0 0 1 2 3 Spatial frequency (c/deg)

14. No improvement over time 2 MM + 5 months MM +11 months MM +17 months MM +21 months CONTROL 1.5 Log sensitivity 1 0.5 0 0 1 2 3 Spatial frequency (c/deg)

15. 2D FORM MM could identify simple shapes

16. 2D FORM …but not shapes defined by illusory contours.

17. 2D FORM Mike can identify simple 2d forms (100% correct) Letters recognizable But “constructive” 2d perception is harder MM = 80%; controls=100%, 90%, 95%) MM = 73%; controls = 80%, 85%, 100% guessing

18. 3D FORM Sensitive (100% correct) to occlusion …

19. 3D FORM Shading gave no automatic impression of depth: The circle was seen as a flat disc, with non-uniform surface lightness

20. 3D FORM Fails with: Shape from Shading: Perspective: “A square with lines attached”

21. MOTION Could NOT recognize a stationary cube from any angle - “square with lines”

22. Couldn’t identify STATIC cube…but with a ROTATING one, “It’s a cube! …going in …going out”

23. MOTION Could NOT recognize a stationary cube from any angle - “square with lines”

24. MOTION Could NOT recognize a stationary cube from any angle - “square with lines” YET … Can exploit motion to construct 3D structure- “it’s a cube! …moving in, moving out”

25. MOTION Could make sense of… Point-light walker Rotational Glass patterns Structure from motion (100% correct) MM = 90%; controls=95%, 80%, 85%).

27. Sophisticated processing of MOTION: • Can see form from motion (KDE cube) • Saw depth in face masks by rocking his head • Could see Johansson’s walking man • Can play catch • Skiing: vision now helps!

28. Motion SB (Ackroyd et al) “His only signs of appreciation were to moving objects, particularly the pigeons in Trafalgar square… He clearly enjoyed … watching … the movement of other cars on the road …He spotted a speeder coming up very fast behind us” Virgil (Sacks) “when [the gorilla] finally came into the open he thought that, though it moved differently, it looked just like a large man”

29. Poor object & face identification MM 25% correct control 100% Gender MM 70% correctcontrol 100% Expression (happy/sad/neutral) MM 61% correct control 100%

30. Clinton&Gore By Sinha &Poggio

32. These dissociations between form and motion tasks were consistent with the size and activation of visual areas measured using fMRI V1 and (especially) extrastriate areas in the temporal stream, thought to be responsible for form processing, were small and showed low activity levels. The Medial Temporal complex, thought to be responsible for motion processing, was normal in both size and activation

33. control observers MM Size of V1 and MT+ 35 30 25 20 Surface area (cm2) 15 10 5 0 V1 (L) V1 (R) MT+ (L) MT+ (R) Cortical area

34. “The eye of the artist” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

35. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

36. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

37. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

38. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

39. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

40. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

41. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

42. “flat world” Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion: Tables have the SAME projective shape, and to MM they look the same

43. Mike correctly sees the diamonds as similar in lightness, and responds photometrically to illumination and shadow in pictures, seeing shadows as dark things. http://psylux.psych.tu-dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/html/shadow.html Images from Ted Adelson

44. Phenomenal Regression to the Real Object • Normally sighted subjects cannot retrieve any aspect of experience that is a function of retinal illuminance or projected size. But MM has these (and nothing else) available to undirected introspection. In this sense he is free (unfortunately), of the good ‘illusions’ on which normal vision is founded. • One example of resulting difficulties: Shadows at the edges of sidewalks appeared to him as black ridges that could present a potential hazard in walking

45. Phenomenal Regression to the Real Object • Why can’t we see and judge what’s present at the sensory input, as MM can? • William James wrote: “Pure sensations can only be realised in the earliest days of life. They are all but impossible to adults with memories and stores of association acquired." • For MM (though NOT necessarily for a newborn: Granrud), James may be right, perhaps because the irrepressible interpretative processes of the normally sighted brain are not involved. • For the normally sighted, interpretation is not an integument that can be peeled away to reveal sensory bedrock: it penetrates all our consciousness, presumably thanks to the continuously bidirectional flow of information through the visual system. • So we have no ‘pure sensations’…but those ares all that MM has.

46. The visual process as a causal chain Parietal (action) temporal (perception) V1 LGN

47. The visual process as a feedback system Parietal (action) temporal (perception) V1 LGN In the normal visual system, each neural representation depends on the later ones.

48. MM was not sensitive to perspective cues Yet he WAS susceptible to the Muller-Lyer and related illusions DEPTH

49. Richard Gregory

50. “Dumbbell” variant of the Muller-Lyer illusion