Stat. inference: n-gram models over sparce data

Stat. inference: n-gram models over sparce data

Stat nlp function • Taking some data(generated in accordance with some unknown probability distribution) and then making some inferences about this distribution. Ex • We might look at lots of prepositional phrase attachments in a corpus and use them to try to predict prepositional phrase attachments for English in general.

We will examine the classic task of language modelling (aka Shannon game) where the problem is to predict the next word given the previous words. • Importance: • Speech or optical recognition, SMT, spelling correction, and handwriting recognition.

Uses • Word sense disambiguation • Probabilistic parsing

Building n-gram models

http://svr-www.eng.cam.ac.uk/~prc14/toolkit.html • Preprocess the corpus using ASCII files • Check this • http://books.google.com/ngrams

MLE • Come across • 10 times of come across • 8 of which were followed by as • Once by more and once by a • PMLE (wn|w1 ….. Wn-1) = C(w1……….wn • ________________ • C(w1……. W-1

Stat. inference: n-gram models over sparce data

Stat. inference: n-gram models over sparce data

Presentation Transcript

Data Models

Chapter 6: Statistical Inference: n-gram Models over Sparse Data

Statistical Inference and Regression Analysis: Stat-GB.3302.30, Stat-UB.0015.01

Seeking Interpretable Models for High Dimensional Data

Statistical Inference and Regression Analysis: Stat-GB.3302.30, Stat-UB.0015.01

Mean Field Inference in Dependency Networks: An Empirical Study

STAT 101: Data Analysis and Statistical Inference

Gram Stain

Gram Staining

Simple Linear Regression

Inference Using Formulas

STAT 101: Data Analysis and Statistical Inference

Simple Linear Regression

Variational Methods for Graphical Models

Stat-Xplore – DWP’s new way of accessing data

Bayesian Inference

Topic 25: Inference for Two-Way ANOVA

Modelling data

Statistical NLP: Lecture 8 Statistical Inference: n-gram Models over Sparse Data (Ch 6)

Stat 501