1 / 16

Understanding Probabilistic Parsing: N-Gram and PCFG Techniques in Natural Language Processing

This lecture explores probabilistic parsing methods in natural language processing (NLP), focusing on computing the probability of sentences using N-grams and Probabilistic Context-Free Grammars (PCFG). Key concepts include probability formulations for sentences, rule probabilities in parse trees, and comparisons between unigram, bigram, and PCFG approaches. The lecture illustrates how statistics and linguistic structures contribute to understanding sentence probabilities and parsing through practical examples, elucidating the importance of grammar in NLP applications.

clem
Télécharger la présentation

Understanding Probabilistic Parsing: N-Gram and PCFG Techniques in Natural Language Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS626-449: Speech, NLP and the Web/Topics in AI Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture-16: Probabilistic parsing; computing probability of a sentence

  2. N-gram vs. PCFG

  3. Chain Rule • P(w1,w2,…,wn) = P(w1) * P(w2/w1) * P(w3/w1-2) *…* P(wn/w1-(n-1) ) where, P(wn/w1-(n-1) ) = #< w1,w2,…,wn > / #< w1,w2,…,wn-1 > # denotes “number of”

  4. Unigram & Bigram Probability • Unigram • P(w1) = #< w1> / #words • Bigram • P(w2/w1) = #< w1,w2 > / #< w1>

  5. PCFG PCFG Prob (P) (Corpus) CFG (human)

  6. N-Gram v/s PCFG

  7. Compare • P(w1,m) = P(w1) * i=1πi=m P(wi/w1, n-i ) Statistics (Speech) • P(w1,m) = ∑t ϵ all parses P(t)Statistics + Linguistics • w1,m = yield(s)linguistics

  8. Example of Sentence labeling: Parsing [S1[S[S[VP[VBCome][NP[NNPJuly]]]] [,,] [CC and] [S [NP [DT the] [JJ UJF] [NN campus]] [VP [AUX is] [ADJP [JJ abuzz] [PP[IN with] [NP[ADJP [JJ new] [CC and] [ VBG returning]] [NNS students]]]]]] [..]]]

  9. Rule Probabilities • Rule probabilities are such that E.g., P( NP  DT NN) = 0.2 P( NP  NN) = 0.5 P( NP  NP PP) = 0.3 • P( NP  DT NN) = 0.2 • Means 20 % of the training data parses use the rule NP  DT NN

  10. Probability of a sentence • Notation : • wab – subsequence wa….wb • Nj dominates wa….wb or yield(Nj) = wa….wb Nj NP wa……………..wb the..sweet..teddy ..bear • Probability of a sentence = P(w1m) Where t is a parse tree of the sentence If t is a parse tree for the sentence w1m, this will be 1 !!

  11. Assumptions of the PCFG model • Place invariance : P(NP  DT NN) is same in locations 1 and 2 • Context-free : P(NP  DT NN | anything outside “The child”) = P(NP  DT NN) • Ancestor free : At 2, P(NP  DT NN|its ancestor is VP) = P(NP DT NN) S NP VP 1 The child NP 2 The child

  12. Probability of a parse tree Domination :We say Nj dominates from k to l, symbolized as , if Wk,l is derived from Nj P (tree |sentence) = P (tree | S1,l ) where S1,l means that the start symbol S dominates the word sequence W1,l P (t |s) approximately equals joint probability of constituent non-terminals dominating the sentence fragments (next slide)

  13. Probability of a parse tree (cont.) P ( t|s ) = P (t | S1,l ) = P ( NP1,2, DT1,1 , w1, N2,2, w2, VP3,l, V3,3 , w3, PP4,l, P4,4 , w4, NP5,l, w5…l |S1,l) = P ( NP1,2 , VP3,l | S1,l) * P ( DT1,1 , N2,2 | NP1,2) * D(w1 | DT1,1) * P (w2 | N2,2) * P (V3,3, PP4,l | VP3,l) * P(w3 | V3,3) * P( P4,4, NP5,l | PP4,l ) * P(w4|P4,4) * P (w5…l | NP5,l) (Using Chain Rule, Context Freeness and Ancestor Freeness ) S1,l NP1,2 VP3,l V3,3 PP4,l DT1 N2,2 w1 w2 P4,4 NP5,l w3 w4 w5 wl

  14. Example PCFG Rules & Probabilities • S  NP VP 1.0 • NP  DT NN 0.5 • NP  NNS 0.3 • NP  NP PP 0.2 • PP  P NP 1.0 • VP  VP PP 0.6 • VP  VBD NP 0.4 • DT  the 1.0 • NN  gunman 0.5 • NN  building 0.5 • VBD  sprayed 1.0 • NNS  bullets 1.0

  15. Example Parse t1` • The gunman sprayed the building with bullets. S1.0 P (t1) = 1.0 * 0.5 * 1.0 * 0.5 * 0.6 * 0.4 * 1.0 * 0.5 * 1.0 * 0.5 * 1.0 * 1.0 * 0.3 * 1.0 = 0.00225 NP0.5 VP0.6 NN0.5 DT1.0 PP1.0 VP0.4 P1.0 NP0.3 NP0.5 VBD1.0 The gunman DT1.0 NN0.5 with NNS1.0 sprayed the building bullets

  16. Another Parse t2 • The gunman sprayed the building with bullets. S1.0 P (t2) = 1.0 * 0.5 * 1.0 * 0.5 * 0.4 * 1.0 * 0.2 * 0.5 * 1.0 * 0.5 * 1.0 * 1.0 * 0.3 * 1.0 = 0.0015 NP0.5 VP0.4 NN0.5 DT1.0 VBD1.0 NP0.2 The gunman sprayed NP0.5 PP1.0 DT1.0 NN0.5 P1.0 NP0.3 NNS1.0 the building with bullets

More Related