Connectionism and LOTH

1. Connectionism Connectionism and LOTH & Language of Thought

2. How is connectionism an alternative to LOTH? • LOT usually represented as implemented by “classical AI.” (Also known as GOFAI: “good, old-fashioned AI”.) • Semantic symbols and syntactic rules are easy to represent in classic AI architecture. • Connectionism does not require symbols, but representations can be symbolic.

3. Types of Connectionist Representations 1) Local representation. Meows Fur Pointed ears Whiskers Output: “it’s a cat” This node is a local representation of “cat”.

4. Characterizing local representations: • Individual nodes are symbols, and can be components of a language of thought. • Not typical of connectionist networks. • One neuron per symbol does not seem biologically plausible. Cell assemblies havethus been proposed for neuro-symbols:http://publik.tuwien.ac.at/files/PubDat_166316.pdf

5. 2) Distributed representations. E.g.: Cat Tiger Leopard Lion See also: David L. Anderson. Computer Types: Classical vs. Non-classical http://www.mind.ilstu.edu/curriculum/nature_of_computers/computer_types.php (cf. SmartKitchen: http://smartkitchen.ict.tuwien.ac.at/project/project.html)

6. Characterizing distributed representations: • Connectionist networks are typically distributed representations. • Distributed representations are not necessarily symbolic. • Distributed representations are more robust to damage than local representations.

7. 3) No representation. More controversially, connectionist networks might have no representational properties. Note: • Output of connectionist network may be recognition of a concept, e.g. Cat, Tiger, Man, etc. but… • Output of connectionist network may also be action, e.g. moving through space, reading aloud • Rather than representing content, networks can just act.

8. Comparison What goes on in your mind when you decide to drink a glass of water that is in front of you? LOTH: the action is the conclusion of a practical syllogism conducted through symbol manipulation Connectionism: the action is output of a neural net responding to a certain set of inputs

9. LOTH approach: I am thirsty. There is a cup of water in front of me. I believe that drinking the water will relieve my thirst. (There is no reason not to drink the water) Conclusion: I drink the water. The conclusion is reached after manipulating the semantic symbols representing beliefs and desires in accordance with syntactic laws. Beliefs and desires give rise to action.

10. Connectionist approach: Inputs from body Inputs from environment Output: I drink the water. There are no symbols involved.

11. Connectionism makes eliminativism possible. Note: in the connectionist/eliminativist approach, the mind concocts the belief-desire explanation, “I drank the water because I was thirsty” to explain its behavior. But the desire (thirst) and beliefs (“the water is in front of me”, “the water is safe to drink”, “the water will relieve my thirst”) are not literally part of the process whereby the mind decides to drink. In other words, the mind only uses symbolic representation when translating/explaining its thoughts in language (talking to oneself or talking to others).

12. But how can “thirst” not play a role in deciding to drink? Isn’t it part of the input from the body? “Thirst” is a feeling. What plays the functional role of “thirst” may be a mechanism to detect that the body is low on water, or is somewhat overheated, but this may not be recognized by you as a desire, until you try to explain your own behavior. Note: imagine reaching unconsciously for a glass of water, and when someone asks, “why are you drinking that?”, you say, “I guess I was thirsty.” The explanation could be rather different than the cause (cf. Freud’s concept of rationalization).

13. Advantages of Connectionism 1) Biological plausibility Connectionist networks are deliberately analogous to neural processes in the brain Units ~ neurons Connections ~ synapses Activations ~ neural signals Neuron Connectionist unit

14. 2) Fast processing via parallelism • “100 Step” argument. • Neurons change state very slowly compared with computer computations. Neurons can only process 100 steps a second (whereas computers can process a million). But the brain can solve many complex problems in less than 1 second, e.g. face recognition  for these, it can use maximally 100 steps. • Conventional computer programs do mostly serial processing and usually require considerably more than 100 processing steps for problems where brains need less than a second.So, such computers cannot provide a good model of cognition. • Connectionist computations are done by parallel processing, thus much more can be achieved in 100 steps. • Cf.: http://www.ucs.louisiana.edu/~isb9112/dept/phil341/myths/myths.html

15. 3) Performance of connectionist networks resembles performance of human brains Connectionist networks are good at: • Pattern recognition: networks can learn through examples • Content-addressable memory: items can be retrieved based on their meanings or properties • Generalizations: networks can generalize connections between characteristics or properties

16. Connectionist networks exhibit: Graceful degradation When a connectionist network has some incorrect input -- “noisy input” -- or is itself partially damaged, it still performs, more poorly, but doesn’t completely break down.

17. 4) Connectionism provides a naturalistic mechanism for creating concepts. No need to posit inborn concepts. Concepts can precede language without being inborn. Fodor once claimed that mentalese was“the only game in town”. Connectionism is a new game!

18. Criticisms of connectionism The advantages of connectionism revisited: • Biological plausibility • 100-steps argument • Pattern recognition and concept formation: yes, but can be slow

19. Biological plausibility Networks aren’t really like neurons. • No reverse connections (necessary for backward propagation) in the brain. • Neurons only fire or not: they cannot be both inhibitory and excitatory. • Connectionist units are too fast, neurons are quite slow.

20. Biological plausibility (cont.) • There are many different types of neurons in the brain, but connectionist units are meant to represent all neurons. • In addition, role of neurotransmitters and hormones in thinking is ignored in connectionist models. Note: most people admit that connectionist networks are still more biologically plausible than classical AI architectures. Different types of neurons

21. 2) The 100 step argument Problem: what is a step? Is, recognizing a color one step? Or does it break down into numerous steps? The 100 step argument only works if each unit of a connectionist network corresponds to one neuron. If one unit corresponds to several neurons working together, the 100 step constraint may be greatly exceeded. Also, the 100 step argument assumes only connectionist architectures are parallel processors, while all non- connectionist architectures are serial. But it is possible to build parallel non-connectionist architectures. Cf. review:http://www.icsr.agh.edu.pl/publications/html/ppam97prof/ppam97prof.html

22. 3) Network learning can be slow Many connectionist networks need a large amount of explicit feedback to learn. Others, e.g. self-organizing maps, use unsupervised learning :http://www.willamette.edu/~gorr/classes/cs449/Unsupervised/SOM.html The brain often seems to learn a new concept or pattern in one shot. One-shot learning is especially easy when information is gathered through language. Example: think of teaching an intelligent chimp vs. a five-year-old child, to push the red button for food.

23. Another weakness of Connectionism Systematicity and productivity: very difficult (impossible?) to implement in connectionist architecture. Connectionist responses: • Deny systematicity and productivity of the mind: Is human thinking really systematic/productive? Do animals think systematically/productively? • Maintain the ability of connectionist nets to generate systematicity and productivity

24. The Relationship between Connectionism and LOTH Three possibilities: • Connectionism implements LOTH • Connectionism replaces LOTH • Hybrid theory. Some mental processes are connectionist, others are conducted through LOT.

25. Connectionism implements LOT Connectionist nets can be regarded as a lower-level implementation of LOT. Neural nets can represent semantic symbols which are then manipulated in accordance with language-like laws (also implemented by neural nets). Criticism: if connectionist nets only implement LOT, many of the advantages of connectionism are lost.

26. 2) Connectionism replaces LOT Consequence: all the advantages (e.g. systematicity and productivity) of LOT are lost. Can we do without them?

27. Hybrid theory • Some mental processes are connectionist, others are conducted through LOT. • E.g.: • Perception and motor control handled by connectionist nets. • Reasoning and language handled by LOT, and implemented by connectionist nets.

28. Class-level logic of hybrid theory presented as a graph Mental Process subClassOf Peripheral Process Central Process Reasoning Language Perception Motor Control handle handle LOT Connectionist Net implement

29. Rule composing the implement and handlerelations Z handle Y realize X implement

30. Class-level logic extended by inferred relation Mental Process subClassOf Peripheral Process Central Process Reasoning Language Perception Motor Control handle handle LOT realize Connectionist Net implement

31. Connectionism and Modularity Connectionist networks can do simple, small tasks. In more complicated tasks, they are overwhelmed by the complexity (because the connections increase exponentially). Mind must be organized into simple units, connected up in an efficient way. “Connectoplasm”: the mind an unorganized mess of connections. Not a viable idea. Mental modules: some connections preset, others learned.A way to contain the complexity (maybe even recursively: modules of modules).

32. Readings for next week Focus: Thomas Nagel (1974), “What is it like to be a bat?”, The Philosophical Review, LXXXIII, 4 (October 1974), 435-50 http://www.clarku.edu/students/philosophyclub/docs/nagel.pdf Block (2002), “Some Concepts of Consciousness”, in David Chalmers (Ed.). Philosophy of Mind: Classical and Contemporary Readings Oxford University Press http://www.nyu.edu/gsas/dept/philo/faculty/block/papers/Abridged%20BBS.htm Extra: Gallup, Jr., Povinelli (1998). Can Animals Empathize? Yes. Scientific American - Exploring Intelligence (a debate).http://www.sciamdigital.com/index.cfm?fa=Products.ViewIssuePreview&ARTICLEID_CHAR=9123A7A5-59B3-4355-8946-C0E31A72A09