1 / 27

Psych 85-419: Introduction to Parallel Distributed Processing

Psych 85-419: Introduction to Parallel Distributed Processing. Michael Harm, Professor Anthony Cate, TA. Course Objectives. Solid background in the philosophical and computational underpinnings of modern connectionist (PDP) research Experience with the construction and analysis of pdp models

kyrene
Télécharger la présentation

Psych 85-419: Introduction to Parallel Distributed Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Psych 85-419: Introduction to Parallel Distributed Processing Michael Harm, Professor Anthony Cate, TA

  2. Course Objectives • Solid background in the philosophical and computational underpinnings of modern connectionist (PDP) research • Experience with the construction and analysis of pdp models • Appreciation of the benefits (and limitations!) of PDP approaches to psychological research

  3. By May, You All Should Be Able To: • Recognize when a PDP model may be useful to your research, • Build a model of a phenomena that interests you • Understand the contributions of models you see in the literature • … and/or critique them!

  4. Course Will Be Geared Towards Two Communities • Modelers who plan to use these techniques in their work • Researchers who want to better understand these models and their implications, even if they don’t want to be a modeler • straw poll: which group do you fall into?

  5. Grading • Four homeworks, each of which count for 10% of your final grade • One exam, worth 15% of your grade • A project proposal, worth 5% • A final project worth 30% of your grade • Class participation, worth 10% of your grade • No final exam

  6. Class Web Page www.cnbc.cmu.edu/~mharm/courses/pdp_spring2001/ Watch for updates

  7. What is Expected of You • Readings assigned for each class. Read them! • Come prepared with thoughtful questions • Participate in class discussions • Complete assignments on time • Come to us if you need help! Don’t wait until the last minute!

  8. Overview of Class • What is PDP, anyway? (That’s next) • Processing and Constraint Satisfaction • Simple learning and distributed representations • Learning internal representations • Unsupervised learning • Psychological phenomena • Language, vision, higher level cognition

  9. So What is PDP, Anyway? • Start by describing more traditional approaches • Why would one want a different approach? • PDP defined • A case study • History of the approach

  10. Traditional Approach to Studying Cognition • The mind is like a computer • There are rules, facts and propositions • There is a logic engine that operates over these rules and propositions • Generates new propositions, new facts, new rules • The Name of the Game: Identify the rules and propositions for a given phenomena

  11. Who Uses This Method (Implicitly or Otherwise)? • Traditional AI, e.g. unification • if (not (married X)) -> (bachelor X) • (not (married JOHN)) implies JOHN is a bachelor • Traditional linguistics (Chomsky, etc.) • Philosophy of Mind (Fodor, etc.) • Psychologists (some, at least)

  12. Why Would One Question This Approach? • Descriptive versus explanatory • An equation for an ellipse describes planetary motion. • But planets do not compute the equation for an ellipse to decide where to go! • Has an air of Greek Mythology about it • Creating theories to account for data, with no external validation

  13. Why Would One Question This Approach (More) • Doesn’t seem to be how the mind actually works • Robust to damage • Graded degradation in performance • Doesn’t seem to be a single “logic engine” shared across all domains

  14. Why Would One Question This Approach (Yet More) • No obvious link to neuroscience • Single cell recordings, systems neuroscience • Impairments that have different effects on cells • Method is typically grounded in symbolic rules • What about phenomena that aren’t rule governed?

  15. So, Fine. Now Will You Tell Us What PDP Is? • The idea that cognition can arise through the interactions of simple processing units • Blind to the global task at hand • Output activity based on state and summed input • … kind of like neurons • … and that this may be a good way to study cognition

  16. The Name of the Game • Construct a model consisting of processing units and connections between them • Guided by theory, observation, hypothesis • Explore the behavior of the model. Relate to behavioral data • Use model to gain insights into causes of behavioral data

  17. A Case Study: Frequency by Regularity in Reading • Regular words are words whose spelling to sound correspondences are predictable from other words. Like gave, save, wave, pave. • Exception words are ones that violate the normal rules of pronunciation, like have, yacht, sergeant • Word frequency is how often it is seen. Words like the versus yacht

  18. Frequency by Regularity • Exception words affected by frequency • Regular words not (more or less)

  19. Traditional Account (Coltheart and colleagues) • One cognitive module is responsible for reading exception words. It is frequency sensitive • Another module can only read regular items. It is rule governed, frequency insensitive.

  20. An Alternative Account, Part I: The Existence Proof • Seidenberg & McClelland ‘89 constructed large scale connectionist model of reading • Mapped spelling patterns onto pronunciation • Observed same frequency by regularity interaction • Therefore, data does not necessitate separate systems for rules and exceptions

  21. An Alternative Account, Part II: Analysis • Plaut et al ‘96 analyzed a network that exhibited frequency by regularity interaction • Accounted for effect through mathematical analysis of network • This is a different kind of theorizing. • Rooted in computational principles • Discovered, rather than designed

  22. History I: The Age of Discovery • McCulloch & Pitts (1943) • Networks of simple logic gates can compute any finite logic proposition • Hebb (1949) • Clear definition of a learning rule for neurons • Selfridge (1958) • Intelligent behavior from interactions of many agents • And many others...

  23. History II: The Cold Years • Minsky & Papert ‘69: Simple associators cannot compute problems that are not linearly separable • The XOR problem • Many problems aren’t linearly separable • Led to scarcity of funding for such research. Golden years of artificial intelligence.

  24. History III:Renaissance of the Mid ‘80s • Discovery of training algorithms that are more powerful than simple associators • Could compute problems that are not linearly separable • Resurgence in interest in use of these models for theory construction

  25. History IV:The Counter Attack • Pinker & Prince ‘88 launched attack on PDP account of inflectional morphology • Fodor & Pylyshyn ‘88 attacked connectionist enterprise as a whole • Besner et al., Coltheart et al. attacked findings of Seidenberg & McClelland ‘89 model • McCloskey: Networks are not theories!

  26. Where We Are Today High Level Reasoning, Creativity Parsing Sentences Morphology Reading Classical Conditioning, Priming Low Level

  27. For Next Class • Read PDP1, Chapter 2 • Optional: Read PDP1, Chapter 1

More Related