Natural Language Processing Lecture 1 : Introduction to NLP

1. Natural Language ProcessingLecture 1: Introduction to NLP Winter 2014-15 Lecturer: Prof. Roi Reichart

2. Course Information • Lecture: Roi Reichart. TA: Ira Leviant • Class: Thursday 11:30 - 14:30, TA: Thursday 10:30 - 11:30

3. Course Prerequisites • Basic Linear Algebra, Probability, Algorithms • Machine learning will help (but some of it will be reviewed in class and TA sessions) • Programming skills

4. Course Grading • Class and TA attendance – 5% (mandatory, contact me if you cannot attend) • Student lecture – 20% (last hour every week) • 40% - four home work assignments (one programming question plus a few theoretical ones) • 35% - final project (?)

5. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

6. Natural Language Processing (NLP)

7. Natural Language Processing (NLP)

8. Natural Language Processing (NLP) - Interdisciplinary

9. Natural Language Processing (NLP) - Technology

10. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

11. Modeling Language learning and Processing The four components of scientific description of a phenomenon • Theory • Model • Parameter estimation, a.k.a learning • Prediction, a.k.a inference

12. Modeling Language Learning and Processing The four components of scientific description of a phenomenon • Theory – many layers of language analysis are hidden (i.e. does not exist in the world) • Model • Parameter estimation, a.k.a learning • Prediction, a.k.a inference

13. Modeling Language Learning and Processing

14. Modeling Language Learning and Processing The four components of scientific description of a phenomenon • Theory – many layers of language analysis are hidden (i.e. does not exist in the world) • Model – since many of the layers are hidden there can be strong disagreement on their modeling • Parameter estimation, a.k.a learning • Prediction, a.k.a inference

15. Modeling Language Learning and Processing The four components of scientific description of a phenomenon • Theory – many layers of language analysis are hidden (i.e. does not exist in the world) • Model – since many of the layers are hidden there can be strong disagreement on their modeling • Parameter estimation, a.k.a learning – as models contain hidden layers this can be complicated • Prediction, a.k.a inference – as models contain hidden layers this can be complicated

16. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

17. What is in NLP ? • Machine Learning and Optimization (especially structured prediction)

18. What is in NLP ? • Machine Learning and optimization (especially structured prediction) • Statistical Modeling (parameter estimation, inference)

19. What is in NLP ? • Machine Learning and optimization (especially structured prediction) • Statistical Modeling (parameter estimation, inference) • Linguistics

20. What is in NLP ? • Machine Learning and optimization (especially structured prediction) • Statistical Modeling (parameter estimation, inference) • Linguistics • Cognitive Science

21. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

22. Language Processing

23. Basic NLP Tasks • Basic input analysis (word segmentation):

24. Basic NLP Tasks • Basic input analysis (word segmentation):

25. Basic NLP Tasks • Word level analysis (morphological segmentation):

26. Basic NLP Tasks • Sentence level analysis: Syntax, Part-of-Speech (POS) tagging: I gave the smaller balls to Itamar PRP V DT JJ N IN N PRP – Personal Pronoun JJ - Adjective V – Verb N – Noun DT – Determiner IN - Preposition

27. Basic NLP Tasks • Sentence level analysis: Syntax, syntactic parsing: I gave the ball to Itamar PRP VBD DT NN TO NNP

28. Basic NLP Tasks • Sentence level analysis: lexical semantics:

29. Basic NLP Tasks • Word meaning (semantics):

30. Basic NLP Tasks • Multilingual Processing:

31. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

32. Text Processing A powerful car bomb exploded today in Baghdad inside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came only two days after another car bomb exploded in a crowdedstreet in Mosul in the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Together with the shooting in Najaf three weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq …

33. Text Processing – Field Labeling A powerful car bomb exploded today in Baghdadinside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came only two days after another car bomb exploded in a crowdedstreet in Mosul in the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Together with the shooting in Najaf three weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq ….

34. Text Processing - Event Segmentation Event 1 • A powerful car bomb exploded today in Baghdadinside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came … Event 2 • only two days after another car bomb exploded in a crowdedstreet in Mosulin the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Event 3 • Together with the shooting in Najafthree weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq …. Event 1

35. Text Processing – Joint Task

36. Machine Translation

37. T1  H1 T1 T1 “At the end of the year, all solid companies pay dividends.” “At the end of the year, all solid companies pay dividends.” H1 H2 “At the end of the year, all solid insurance companies pay dividends.” “At the end of the year, all solid companies pay cash dividends.” T1  H2 Textual Entailment

38. Textual Entailment • Things can be much more challenging ….. Context Sensitive Paraphrasing: • Can speak replace command? • The general commanded his troops. • The general spoke to his troops. • The soloist commandedattention. • The soloist spoke to attention.

39. Text Summarization

40. Dialog Systems

41. Another Branch - Computational Models in Psycholinguistics

42. This Course • This course will focus on the basic NLP tasks: • Language modeling • Tagging tasks (sequence learning) • Syntax (as the prototype of complex structure learning) • Semantics (with an emphasize on lexical semantics) • We will take a data-driven, machine learning based approach

43. This Course • Why do we take this approach: • Allows us to develop a principled model based approach to NLP • Allows us to develop the fundamental machine learning tools of NLP • Particularly, allows us to learn about predicting structure • Focuses on the core of NLP – the models we will develop make NLP applications possible

44. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

45. Ambiguity in Language

46. Ambiguity in Language

47. Ambiguity in Language

48. Ambiguity in Language

49. Ambiguity in Language

50. Ambiguity in Language