Cognitive Computing 2012

1. Cognitive Computing 2012 Embodiment and Enactivism A SENSORIMOTOR ACCOUNT OF VISUAL CONSCIOUSNESS (O’REGAN & NOE) Professor Mark Bishop - Images from J. K. O’Regan, 2001.

2. The ‘explanatory gap’ • Traditional ‘representational theories of vision’ leads to a fundamental problem, the explanatory gap: • They must either ‘employ a homunculus’ or ‘account for the gap between the physics of a conscious system and the raw feel of experience’. • But just how could any neural structure be responsible for [give rise to; generate] first person lived experience? (c) Bishop: Consciousness and Cognition

3. The phenomenology of different sensory modalities • Why is the experience of red not like the sound of a bell? • Is it because there is something characteristic about the nerve pathways from vision to brain contra ear to brain? • Or is it because there is something characteristic about the visual cortex of the brain different from the auditory cortex? (c) Bishop: Consciousness and Cognition

4. The problem of representation:change blindness (1) • Kevin O’Regan’s seminal work on ‘human change blindness’ research suggests that our‘internal representations’of the world are very sparse; • they contain little or no detail. (c) Bishop: Consciousness and Cognition

5. Change blindness (2) • Consider the retinal blindspot where there are no photoreceptors yet we perceive a continuous - and rich - visual field. • Similarly colour vision is weak outside the ‘fovea centralis’. (c) Bishop: Consciousness and Cognition

6. Change blindness (3) • Even if we stare at the area of change in an image, the probability of detecting it is less than 60%. (c) Bishop: Consciousness and Cognition

7. Change blindness (4) • The further away a change takes place from the area that we are attending, the less likely we are to notice it • even if it quite large.. (c) Bishop: Consciousness and Cognition

8. Change blindness (5) (c) Bishop: Consciousness and Cognition

9. Change blindness (6) (c) Bishop: Consciousness and Cognition

10. Change blindness (7) • Even if we stare at the area of change in an image we can continue to miss the change. • ‘Inattentional Blindness’ • Changes can be slow. • < http://www.newscientist.com/blogs/nstv/2011/06/friday-illusion-can-you-spot-the-change.html > (c) Bishop: Consciousness and Cognition

11. A ‘sensorimotor account of vision’ • Experience is not something we passively feel but something we actively do; ‘seeing’ is a verb! • Analogy is of driving a car. • This automatically accounts for the difference between the different sensory modalities as each employ different sensorimotor skills. • Contrast this sensorimotor account of the ‘phenomenology of perception’ with: • Mueller’s ‘specific nerve energies’ (sensory pathways); • Cortical areas of activation; • In either case why should the phenomenology of perception [of the senses; of the different sensory modalities] be different? (c) Bishop: Consciousness and Cognition

12. The need for high fidelity internal representations in vision? • No high fidelity internal representations need be created for functional vision; • [Pace Brookes] In this view there is no need for ‘detailed internal representations’ as the brain can uses the outside world as its own ‘external memory’. • I.e. ‘Knowledge of the world’ becomes accessible via our sensorimotor co-ordinated interactions with it. • Consider the ‘harmonica in the bag’ experiment. (c) Bishop: Consciousness and Cognition

13. ‘Harmonica in the bag’ experiment • Children’s game in which someone puts a household object like a cork, or a potato, or a harmonica into a bag, • Subject puts hand into the bag and tries to figure out what the object is.. (c) Bishop: Consciousness and Cognition

14. ‘Feeling’ the harmonica • At first you have no idea what the object is, but then suddenly you have a kind of “aha!” experience. • Suddenly you feel you’re no longer touching bits of texture on the ends of your fingers, but you're holding a whole object: it's a harmonica. • And it’s ALL there at once, even though you’re in fact only touching a few parts of it. • It’s not just that you know it’s a whole harmonica, you actually feel it’s a whole harmonica . • The reason you have the feeling of touching the whole harmonica is that you have mastery of the appropriate sensorimotor contingencies: • You KNOW that IF you were to move your fingers this way, then you would get THIS feeling, and if you moved them that way, you would get THAT feeling. (c) Bishop: Consciousness and Cognition

15. Exploring the world • No need for a rich internal representation of the world, we just need to learn algorithms for sensorimotor coordination to explore it. • For example, in vision these algorithms would control the eye saccades as we attend visual features of the world. • This process results in the grand illusion that we see the whole visual field albeit we actually only attend to a small part of it at any one time. (c) Bishop: Consciousness and Cognition

16. On really ‘seeing’ • Consider our perception of the image opposite: • Instead of regularly exploring the image to build up a representation, we simply check to see it confirms our belief that it depicts a man and woman eating. • We believe we maintain a rich visual representation of the world, because each time we enquire of some visual feature, our eye immediately saccades to attend to it. (c) Bishop: Consciousness and Cognition

17. ‘Seeing’ with our eye muscles • Seeing only occurs when exorcising our sensorimotor mastery over a scene • Image manipulation by moving our eyes. • Only the aspects attended to by our eyes are actually perceived. • In a recent flight simulator experiment two airline pilots out of eight continued to attempt to land their aircraft even though the runway had a small light aircraft parked across it. (c) Bishop: Consciousness and Cognition

18. A problem • If seeing is the process of accessing external memory, why doesn't the phenomenology of seeing correspond to the phenomenology of memory? • We visually perceive dreams even though there are no sensorimotor contingencies at play however the dream sight is different to a consistent, rich, real vision of the world. • Similarly, seeing my mother is different [and feels different] to recalling a memory of her. • This is because (in the 2001 work) O’Regan and Noe suggest that perception in the real world (cf. memories) also has the twin dimensions of: • Bodiliness. • Grabiness. (c) Bishop: Consciousness and Cognition

19. ‘Bodiliness’ and ‘Grabiness’ • Bodiliness • In visual experience the process of exploring the environment is closely tied to a multitude of minute bodily actions which together form a direct linkage between the outside world and the body. • Bodiliness is not present in even the most vivid dreams or hallucination. • Grabiness • Our visual system is hard-wired such that sudden events in the world grab our attention, forcing us to examine them. • Grabiness gives us the illusion we have ‘tabs’ on what is going on. (c) Bishop: Consciousness and Cognition

20. O’Regan’s 2011 version of SMT • In O’Regan’s latest version of SMT (cf. “Why red doesn’t sound like a bell”) he introduces two additional dimensions to characterise the quality of experience. • These new dimensions are: • (i) The RICHNESS of the real world and • (ii) The [partial] insubordinateness of the real world. • Together with an additional requirement for cognitive agent awareness [presumably to rule out conscious ‘perception’ in relatively ‘dumb’ industrial style robots]: • (a) the agent meta-awareness of ‘having a perception’ and • (b) the agent self: the agent/world distinction. (c) Bishop: Consciousness and Cognition

21. Memory and vision • Memory is the process of accessing information stored in the brain. • Vision is the process of accessing information, (stored), in the world. • E.g. A list of Latin verbs scores low on ‘grabiness’ or ‘bodiliness’ hence we have a different phenomenal experience of them than something visual. • Bodiliness Grabiness Raw Feel • Memory - - . • Wealthiness + - + • Driving ++ - ++ • Seeing +++ +++ +++ • NB. The act of driving per-se has no grabiness, however driving involves seeing (which clearly does).. (c) Bishop: Consciousness and Cognition

22. The hard problem of qualia • What is it about a red bit of paper that makes us experience red? • Redness is defined by the processes that are carried out by the sensorimotor system when the changes of incoming reflectance spectra are typical of a ‘red’ thing. • Hence redness is not just the spectra of a point on a red surface; but the relative changes we observe when looking across the scene. (c) Bishop: Consciousness and Cognition

23. The qualities of sensory modalities • Johannes Muller • Different pathways have different, ‘nerve energies’. • O'Regan & Noe • Car driving is different from bike riding, is different from juggling, is different from swimming etc. as they all involve mastery of different sensorimotor skills. (c) Bishop: Consciousness and Cognition

24. Two views of vision • Representation Vs Action The ‘explanatory gap’ (c) Bishop: Consciousness and Cognition

25. Conclusion (1) • 'Enactive' cognitive science was defined (by Varela, Thompson and Rosch, in The Embodied Mind, 1991) as a new theoretical perspective, influenced by the Husserlian phenomenology, Heidegger and Gibsonian ecological approaches to perception. • The ‘sensorimotor’ approach of O’Regan [and Noe] is one enactive approach to cognition … • … but one whereby ‘seeing’ is not simply an on-going coupled process between organism and environment, but a particular kind of ‘enacted knowledge’ • where the environment is an external memory • and where particular ‘bodiliness’ and ‘grabiness’ interactions ‘colour’ the seeing.. (c) Bishop: Consciousness and Cognition

26. Conclusion (2) • The sensorimotor theory attempts to explain the ‘feltness’, of mental states. • The different sensory modalities; the intermodal gap. • The different shades of perception - the ineffable red of a rose rather than the ineffable pink; the intramodal gap. • However in additional commentary on sensorimotor theory Hurley and Noe – ‘Neural Plasticity and consciousness’, (2003) – seem to set aside the question of the ‘absolute [explanatory] gap’ • … ‘dividing [the problem] in the hope of conquering’. (c) Bishop: Consciousness and Cognition