1 / 29

CMSC 723 / LING 723: Computational Linguistics I

CMSC 723 / LING 723: Computational Linguistics I. September 3, 2008: Bonnie Dorr Overview, History, Goals, Problems, (J&M 1). CL vs NLP. Why “Computational Linguistics” (CL) rather than “Natural Language Processing” (NLP)?

creda
Télécharger la présentation

CMSC 723 / LING 723: Computational Linguistics I

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMSC 723 / LING 723: Computational Linguistics I September 3, 2008: Bonnie Dorr Overview, History, Goals, Problems, (J&M 1)

  2. CL vs NLP Why “Computational Linguistics” (CL) rather than “Natural Language Processing” (NLP)? • Computational Linguistics: Computers dealing with language, modeling what people do • Natural Language: Applications on the computer side • Why “natural”? Refers to the language spoken by people, e.g. English, Japanese, Swahili, as opposed to artificial languages, like C++, Java, etc.

  3. Relation of CL to Other Disciplines Electrical Engineering (EE) (Optical Character Recognition) Artificial Intelligence (AI) (notions of rep, search, etc.) Linguistics (Syntax, Semantics, etc.) Machine Learning (particularly, probabilistic or statistic ML techniques) Psychology CL Philosophy of Language, Formal Logic Human Computer Interaction (HCI) Information Retrieval Theory of Computation

  4. Where does it fit in the CS taxonomy? Computers Artificial Intelligence Alg/Thy/NA Sys/networks Databases SWE/HCI Robotics ML Search Logic Natural Language Processing Information Retrieval Machine Translation Language Analysis Semantics Parsing Adapted from Rada Mihalcea (2007)

  5. A Sampling of “Other Disciplines” • Linguistics: formal grammars, abstract characterization of what is to be learned. • Computer Science: algorithms for efficient learning or online deployment of these systems in automata. • Engineering: stochastic techniques for characterizing regular patterns for learning and ambiguity resolution. • Psychology: Insights into what linguistic constructions are easy or difficult for people to learn or to use

  6. History: 1940-1950’s • Development of formal language theory (Chomsky, Kleene, Backus). • Formal characterization of classes of grammar (context-free, regular) • Association with relevant automata • Probability theory: language understanding as decoding through noisy channel (Shannon) • Use of information theoretic concepts like entropy to measure success of language models.

  7. 1957-1983 Symbolic vs. Stochastic • Symbolic • Use of formal grammars as basis for natural language processing and learning systems. (Chomsky, Harris) • Use of logic and logic based programming for characterizing syntactic or semantic inference (Kaplan, Kay, Pereira) • First toy natural language understanding and generation systems (Woods, Minsky, Schank, Winograd, Colmerauer) • Discourse Processing: Role of Intention, Focus (Grosz, Sidner, Hobbs) • Stochastic Modeling • Probabilistic methods for early speech recognition, OCR (Bledsoe and Browning, Jelinek, Black, Mercer)

  8. 1983-1993: Return of Empiricism • Use of stochastic techniques for part of speech tagging, parsing, word sense disambiguation, etc. • Comparison of stochastic, symbolic, more or less powerful models for language understanding and learning tasks.

  9. 1993-1999 • Advances in software and hardware create NLP needs for information retrieval (web), machine translation, spelling and grammar checking, speech recognition and synthesis. • Stochastic and symbolic methods combine for real world applications.

  10. The Rise of Machine Learning: 2000-2007 • Large amounts of spoken & written material now widely available: LDC, etc. • Increased focus on learning has led to more serious interplay with statistical ML community. • Unsupervised learning techniques on the rise—in part brought about by difficulty of producing reliably annotated corpora.

  11. Language and Intelligence: Turing Test • Turing test: • machine, human, and human judge • Judge asks questions of computer and human. • Machine’s job is to act like a human, human’s job is to convince judge that he’s not the machine. • Machine judged “intelligent” if it can fool judge. • Judgement of “intelligence” linked to appropriate answers to questions from the system.

  12. ELIZA • Remarkably simple “Rogerian Psychologist” • Uses Pattern Matching to carry on limited form of conversation. • Seems to “Pass the Turing Test!” (McCorduck, 1979, pp. 225-226) • Eliza Demo: http://www.lpa.co.uk/pws_dem4.htm

  13. What’s involved in an “intelligent” Answer? Analysis: Decomposition of the signal (spoken or written) eventually into meaningful units. This involves …

  14. Speech/Character Recognition • Decomposition into words, segmentation of words into appropriate phones or letters • Requires knowledge of phonological patterns: • I’m enormously proud. • I mean to make you proud.

  15. Morphological Analysis • Inflectional • duck + s = [N duck] + [plural s] • duck + s = [V duck] + [3rd person s] • Derivational • kind, kindness • Spelling changes • drop, dropping • hide, hiding

  16. S NP VP I V NP OR: watch Subject Object I terrapin Det the watched det N the terrapin Syntactic Analysis • Associate constituent structure with string • Prepare for semantic interpretation

  17. Semantics • A way of representing meaning • Abstracts away from syntactic structure • Example: • First-Order Logic: watch(I,terrapin) • Can be: “I watched the terrapin” or “The terrapin was watched by me” • Real language is complex: • Who did I watch?

  18. Lexical Semantics The Terrapin, is who I watched. Watch the Terrapin is what I do best. *Terrapin is what I watched the Predicate: “watch” Watcher: “I” Watchee: “Terrapin”

  19. Proposition Experiencer Predicate: Be (perc) I (1st pers, sg) pred patient saw the Terrapin Compositional Semantics • Association of parts of a proposition with semantic roles • Scoping: Every man loves a woman

  20. Word-Governed Semantics • Any verb can add “able” to form an adjective. • I taught the class . The class is teachable • I rejected the idea. The idea is rejectable. • Association of particular words with specific semantic forms. • John (masculine) • The boys ( masculine, plural, human)

  21. Pragmatics • Real world knowledge, speaker intention, goal of utterance. • Related to sociology. • Example 1: • Could you turn in your assignments now (command) • Could you finish the homework? (question, command) • Example 2: • I couldn’t decide how to catch the crook. Then I decided to spy on the crook with binoculars. • To my surprise, I found out he had them too. Then I knew to just follow the crook with binoculars. [ the crook [with binoculars]] [ the crook] [ with binoculars]

  22. Discourse Analysis • Discourse: How propositions fit together in a conversation—multi-sentence processing. • Pronoun reference: The professor told the student to finish the assignment. He was pretty aggravated at how long it was taking to pass it in. • Multiple reference to same entity:George W. Bush, president of the U.S. • Relation between sentences:John hit the man. He had stolen his bicycle

  23. NLP Pipeline speech text Phonetic Analysis OCR/Tokenization Morphological analysis Syntactic analysis Semantic Interpretation Discourse Processing

  24. Relation to Machine Translation analysis input generation output Morphological analysis Morphological synthesis Syntactic analysis Syntactic realization Semantic Interpretation Lexical selection Interlingua

  25. Ambiguity I made her duck I cooked waterfowl for her I cooked waterfowl belonging to her I created the (plaster?) duck she owns I forced her to lower her head By magic, I changed her into waterfowl

  26. Syntactic Disambiguation S S NP VP NP VP I V NP VP I V NP made her V made det N duck her duck • Structural ambiguity:

  27. Part of Speech Tagging and Word Sense Disambiguation • [verb Duck ] ! [noun Duck] is delicious for dinner • I went to the bank to deposit my check. I went to the bank to look out at the river. I went to the bank of windows and chose the one dealing with last names beginning with “d”.

  28. Resources forNLP Systems • Dictionary • Morphology and Spelling Rules • Grammar Rules • Semantic Interpretation Rules • Discourse Interpretation Natural Language processing involves: (1) learning or fashioning the rules for each component, (2) embedding the rules in the relevant automaton (3) using the automaton to efficiently process the input

  29. Some NLP Applications • Machine Translation—Babelfish (Alta Vista): • Question Answering—Ask Jeeves (Ask Jeeves): • Language Summarization—MEAD (U. Michigan): • Spoken Language Recognition— EduSpeak (SRI): • Automatic Essay evaluation—E-Rater (ETS): • Information Retrieval and Extraction—NetOwl (SRA): http://babelfish.altavista.com/translate.dyn http://www.ask.com/ http://www.summarization.com/mead http://www.eduspeak.com/ http://www.ets.org/research/erater.html http://www.netowl.com/extractor_summary.html

More Related