What is Cognitive Science?

1. Zenon Pylyshyn, Rutgers Center for Cognitive Science What is Cognitive Science? What’s in the mind that we may know it? http://ruccs.rutgers.edu/faculty/pylyshyn.html

2. Cognitive science is a delicate mixture of the obvious and the incredible Granny was almost right: Behavior really is governed by what we know and what we want (together with the mechanisms for representing and for drawing inferences from these)

3. It’s emic, not etic properties that matterKenneth Pike • What determines our behavior is not how the world is, but how we represent it • As Chomsky pointed out in his review of Skinner, if we describe behavior in relation to the objective properties of the world, we would have to conclude that behavior is essentially stimulus-independent • Every behavioral regularity (other than physical ones like falling) is cognitively penetrable

4. It’s emic states that matter!

5. The central role of representation presents some serious problems for a natural science What representations are aboutis what matters But how can the fact that a belief is about some particular thing have a lawful, observable consequence? e.g. How can the presence of “holy grail” in a belief determine behavior when the holy grail does not exist? In a natural science if “X causes Y” then X must exist and be causally connected to Y! It’s even worse than that; even when X exists, it is not X’s physical properties that are relevant! e.g., the North Star & navigation

6. This dilemma is sometimes referred to as the problem of intentionality orBrentano’s problem What determines our actions is what our mental states are about, but aboutness is not a category of natural science. That is why Brentano concluded that psychology was beyond the grasp of natural science.

7. There are other properties that are special to cognitively determined behavior • The Semantic determinants of most cognitive behavior. To capture regularities in cognitively-caused behavior we must use semantic terms – terms referring to what things mean. Same-meaning stimuli are equivalent for many generalizations of cognitive science. • The Cognitive Penetrability of most cognitive processes. Almost any regularity can be systematically altered in a quasi-rational way by imparting new information.

8. Properties that are special about cognition… • The productivity and systematicity of systems of mental representation. Systems of mental representation are structured so that if they are capable of representing certain situations then they are also capable of representing an unbounded number of other related situations. This leads to the requirements that representations be compositional, and that they have constituent structure. • The critical role of "Cognitive Capacity". Because of an organism's ecological or social niche, only a small fraction of its behavioral repertoire is ever actually observed. Nonetheless an adequate cognitive theory must account for the behavioral repertoire that is compatible with the organism's structure, which we call its cognitive capacity. • That’s why “variance accounted for” is a poor measure of a theory’s explanatory value.

9. Is it hopeless to think we can have a natural science of cognition? Along comes The computational theory of mind “the only straw afloat”

10. The major historical milestones • Brentano’s recognition of the problem of intentionality • The formalist movement in the foundations of mathematics: Hilbert, Gödel, Russell & Whitehead, Turing, Church, … • Representational/Computational theory of mind: Newell & Simon, Chomsky, Fodor

11. How to make a purely mechanical system reason about things it does not understand or know about? The discovery of symbolic logic. (1) Married(John, Mary)orMarried(John, Susan) and the equation or “statement”, (2)not[Married(John, Susan)]. from these two statements you can conclude, (3) Married(John, Mary) But notice that (3) follows from (1) and (2) regardless of what is in the parts of the equation not occupied by the terms or or not so that you could write down the equations without mentioning marriage or John or Mary or, for that matter, anything having to do with the world. Try replacing these expressions with the meaningless letters P and Q. The inference still holds: (1') PorQ (2') notQ therefore, (3') P

12. Intelligent systems behave the way they do because of what they represent How do we avoid dualism? But they are not causally connected to what they represent! • To be a physical system, the relevant properties of the system (those that make it a representation of a particular thing) must fall under physical laws • In order to fall under physical laws, representations themselves must be instantiated as physical properties • To encode knowledge in physical properties you must first encode it in symbolic form (Proof Theory tells us how) and then instantiating those symbolic codes physically (computer science tells us how)

13. The foundation of Cognitive Science rests on the Tri-Level Hypothesis Intelligent systems are organized at three (or more) distinct levels: The physical or biological level The symbolic or syntactic level The knowledge or semantic level This means that different regularities may require appeal to different levels

14. Calculator example • Why is the calculator’s printing faint and irregular? Why are parts of numbers missing in the LED display? • Why does it take longer to multiply large numbers than small ones, whereas it takes the same length of time to add large numbers as small numbers? • Why does it take longer to calculate trigonometrical functions than sums? • Why is it especially fast at calculating the logarithm of 1? • Why is it that when one of the keys (labeled ) is pressed after a number is entered, the calculator prints what appears to be the square root of that number? Will it always do so? • When the answer to an arithmetic problem is too long to fit in the display window, why are some of the digits left off?

15. How can we find out? • Given these serious problems in understanding cognition, is it even possible in principal to find out how the mind works? • Is there even a fact of the matter about what process is responsible for certain behaviors? • Is the only road to understanding cognition through neuroscience? • How can we discover how mental processes work?

16. Weak vs Strong Equivalence • Is cognitive science concerned only with developing models that generate the same Input-Outputbehavior as people exhibit in carrying out certain tasks? • A theory that correctly predicts I-O (or S-R) behavior is said to be weakly equivalentto the psychological process it is supposed to explain. It is what some people mean by “simulating behavior”. • Everyone in Cognitive Science is interested in strong equivalence– we want to explain not only the observed behavior, but also how it is generated. The how will usually takes the form of an algorithm.

17. Simulating the Input-Output function Black Box Input Output • Can we do any better than I-O simulation without looking inside the black box? • If all you have is observed behavior, how can you go beyond I-O simulation?

18. Simulating the Input-Output function • Think about this for a few minutes: • Is there any way to find out HOW a person does a simple problem such as adding two 4 digit numbers? • What are possible sources of evidence that may be relevant to this question?

19. Here is a simple example of an artifact (a simple calculator) • How can we explain different aspects of the calculator’s behavior?

20. Calculator example • Why is the calculator’s printing faint and irregular? Why are parts of numbers missing in the LED display? • Why does it take longer to multiply large numbers than small ones, whereas it takes the same length of time to add large numbers as small numbers? • Why does it take longer to calculate trigonometrical functions than sums? • Why is it especially fast at calculating the logarithm of 1? • Why is it that when one of the keys (labeled ) is pressed after a number is entered, the calculator prints what appears to be the square root of that number? Will it always do so? • When the answer to an arithmetic problem is too long to fit in the display window, why are some of the digits left off?

21. Modeling the Cognitive Process (or the algorithm used in a particular task) Black Box Input Output Index of process If all you have is observed behavior, how can you go beyond I-O mimicry? • Answer: Not all observations are Inputs or Outputs: some are meta-behavior or indexes of processes.

22. Does intentionality (and the trilevel hypothesis) only apply to high-level processes such as reasoning? • Examples from color vision. “Red light and yellow light mix to produce orange light” This remains true for any way of getting red light and yellow light: e.g. yellow may be light of 580 nanometer wavelength, or it may be a mixture of light of 530 nm and 650 nm wavelengths. So long as one light looks yellow and the other looks red the “law” will hold.

23. Does intentionality (and the trilevel hypothesis) only apply to high-level processes such as reasoning? • Examples from vision.

24. Architecture and explanation • Weak vs Strong equivalence – and what it is • Strong equivalence models explain the process – i.e. how it’s done • Methodologies • Example of the Sternberg task • Form vs function – a core distinction • The final distinction: Cognitive Penetrability • The role of neuroscience evidence

25. Example of the Sternberg memory search • The initial input consists of the instructions and the presentation of the memory set (n items). • On each trial the particular input to the black box consists of the presentation of a target letter. • The output consists of a binary response (present or absent). The time taken to respond is also recorded. That is called the “Reaction Time”. • The reaction time is not part of the output but is interpreted as an index of the process (e.g., an indication of how many steps were performed).

26. Example of the input-output of a computational model of the Sternberg task • Inputs: Memory set is (e.g.) C, D, H, N • Inputs: Probe (e.g., C or F) • Output: Pairs of Responses and Reaction Times (e.g. output is something like “Yes, 460 msecs”) • Does it matter how the Output is derived? • It doesn’t if all you care about is I-O behavior • It does if you care about Strong Equivalence (i.e., HOW it works)

27. Example of the input-output of a computational model of the Sternberg task • Inputs are: (1) Memory set = C,D,H,N (2) Target probe = C (or R) • Input-Output prediction using a table: Is this model weakly- or strongly-equivalent to a person?

28. Example of a weakly equivalent model of the Sternberg task • Store memory set as a list L. Call the list size = n • Read target item, call it  (If there is no , then quit) • Check if  is one of the letters in the list L • If found in list, assign =“yes” otherwise  =“no”(That provides the answer, but what about the time ?) • If  =“yes”, set  = 500 + K * n  Rand(20  x  50) • If  =“no”, set  = 800 + K * n  Rand(20  x  50) • Print , Print  • Go to 2 Is this the way people do it? How do you know?

29. What reasons do you have for doubting that people do it this way? Because in this case time should not be one of the computed outputs, but a measure of how many steps it took. The same is true of intermediate states(e.g., evidence includes what subjects say, error rates, eye tracking, judgmentsabout the output, and so on.) Reaction time is one of the main sources of evidence in cog sci. Question: Is time always a valid index of processing complexity?

30. Results of the Sternberg memory search task What do they tell us about how people do it? Is this Input-Output equivalent or is it strongly equivalent to human performance? Self-terminating search Exhaustive search