SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Conditional Random Fields

selima avatar
selima
102 diapositives · 1.4K Views · Jul 23, 2014

Conditional Random Fields. Advanced Statistical Methods in NLP Ling 572 February 9, 2012. Roadmap. Graphical Models Modeling independence Models revisited Generative & discriminative models Conditional random fields Linear chain models Skip chain models. Preview.

Conditional Random Fields - slide 1
01 / 102Conditional Random Fields
1 / 17

Télécharger la présentation

Conditional Random Fields
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Conditional Random Fields Advanced Statistical Methods in NLP Ling 572 February 9, 2012

  2. Roadmap • Graphical Models • Modeling independence • Models revisited • Generative & discriminative models • Conditional random fields • Linear chain models • Skip chain models

  3. Preview • Conditional random fields • Undirected graphical model • Due to Lafferty, McCallum, and Pereira, 2001

  4. Preview • Conditional random fields • Undirected graphical model • Due to Lafferty, McCallum, and Pereira, 2001 • Discriminative model • Supports integration of rich feature sets

  5. Preview • Conditional random fields • Undirected graphical model • Due to Lafferty, McCallum, and Pereira, 2001 • Discriminative model • Supports integration of rich feature sets • Allows range of dependency structures • Linear-chain, skip-chain, general • Can encode long-distance dependencies

  6. Preview • Conditional random fields • Undirected graphical model • Due to Lafferty, McCallum, and Pereira, 2001 • Discriminative model • Supports integration of rich feature sets • Allows range of dependency structures • Linear-chain, skip-chain, general • Can encode long-distance dependencies • Used diverse NLP sequence labeling tasks: • Named entity recognition, coreference resolution, etc

  7. Graphical Models

  8. Graphical Models • Graphical model • Simple, graphical notation for conditional independence • Probabilistic model where: • Graph structure denotes conditional independence b/t random variables

  9. Graphical Models • Graphical model • Simple, graphical notation for conditional independence • Probabilistic model where: • Graph structure denotes conditional independence b/t random variables • Nodes: random variables

  10. Graphical Models • Graphical model • Simple, graphical notation for conditional independence • Probabilistic model where: • Graph structure denotes conditional independence b/t random variables • Nodes: random variables • Edges: dependency relation between random variables

  11. Graphical Models • Graphical model • Simple, graphical notation for conditional independence • Probabilistic model where: • Graph structure denotes conditional independence b/t random variables • Nodes: random variables • Edges: dependency relation between random variables • Model types: • Bayesian Networks • Markov Random Fields

  12. Modeling (In)dependence • Bayesian network

  13. Modeling (In)dependence • Bayesian network • Directed acyclic graph (DAG)

  14. Modeling (In)dependence • Bayesian network • Directed acyclic graph (DAG) • Nodes = Random Variables • Arc ~ directly influences, conditional dependency

  15. Modeling (In)dependence • Bayesian network • Directed acyclic graph (DAG) • Nodes = Random Variables • Arc ~ directly influences, conditional dependency • Arcs = Child depends on parent(s) • No arcs = independent (0 incoming: only a priori) • Parents of X = • For each X need

  16. Example I Russel & Norvig, AIMA

  17. Example I Russel & Norvig, AIMA

  18. Example I Russel & Norvig, AIMA

  19. A B C D E Simple Bayesian Network • MCBN1 Need: Truth table A B depends on C depends on D depends on E depends on

  20. A B C D E Simple Bayesian Network • MCBN1 Need: P(A) Truth table 2 A = only a priori B depends on C depends on D depends on E depends on

  21. A B C D E Simple Bayesian Network • MCBN1 Need: P(A) P(B|A) Truth table 2 2*2 A = only a priori B depends on A C depends on D depends on E depends on

  22. A B C D E Simple Bayesian Network • MCBN1 Need: P(A) P(B|A) P(C|A) Truth table 2 2*2 2*2 A = only a priori B depends on A C depends on A D depends on E depends on

  23. A B C D E Simple Bayesian Network • MCBN1 Need: P(A) P(B|A) P(C|A) P(D|B,C) P(E|C) Truth table 2 2*2 2*2 2*2*2 2*2 A = only a priori B depends on A C depends on A D depends on B,C E depends on C

  24. Holmes Example (Pearl) Holmes is worried that his house will be burgled. For the time period of interest, there is a 10^-4 a priori chance of this happening, and Holmes has installed a burglar alarm to try to forestall this event. The alarm is 95% reliable in sounding when a burglary happens, but also has a false positive rate of 1%. Holmes’ neighbor, Watson, is 90% sure to call Holmes at his office if the alarm sounds, but he is also a bit of a practical joker and, knowing Holmes’ concern, might (30%) call even if the alarm is silent. Holmes’ other neighbor Mrs. Gibbons is a well-known lush and often befuddled, but Holmes believes that she is four times more likely to call him if there is an alarm than not.

  25. Holmes Example: Model There a four binary random variables:

  26. W B A G Holmes Example: Model There a four binary random variables: B: whether Holmes’ house has been burgled A: whether his alarm sounded W: whether Watson called G: whether Gibbons called

  27. W B A G Holmes Example: Model There a four binary random variables: B: whether Holmes’ house has been burgled A: whether his alarm sounded W: whether Watson called G: whether Gibbons called

  28. W B A G Holmes Example: Model There a four binary random variables: B: whether Holmes’ house has been burgled A: whether his alarm sounded W: whether Watson called G: whether Gibbons called

  29. W B A G Holmes Example: Model There a four binary random variables: B: whether Holmes’ house has been burgled A: whether his alarm sounded W: whether Watson called G: whether Gibbons called

  30. Holmes Example: Tables B = #t B=#f A #t #f W=#t W=#f 0.0001 0.9999 0.90 0.10 0.30 0.70 A=#t A=#f B #t #f A #t #f G=#t G=#f 0.95 0.05 0.01 0.99 0.40 0.60 0.10 0.90

  31. Bayes’ Nets: Markov Property • Bayes’s Nets: • Satisfy the local Markov property • Variables: conditionally independent of non-descendents given their parents

  32. Bayes’ Nets: Markov Property • Bayes’s Nets: • Satisfy the local Markov property • Variables: conditionally independent of non-descendents given their parents

  33. Bayes’ Nets: Markov Property • Bayes’s Nets: • Satisfy the local Markov property • Variables: conditionally independent of non-descendents given their parents

  34. A B C D E Simple Bayesian Network • MCBN1 A = only a priori B depends on A C depends on A D depends on B,C E depends on C P(A,B,C,D,E)=

  35. A B C D E Simple Bayesian Network • MCBN1 A = only a priori B depends on A C depends on A D depends on B,C E depends on C P(A,B,C,D,E)=P(A)

  36. A B C D E Simple Bayesian Network • MCBN1 A = only a priori B depends on A C depends on A D depends on B,C E depends on C P(A,B,C,D,E)=P(A)P(B|A)

  37. A B C D E Simple Bayesian Network • MCBN1 A = only a priori B depends on A C depends on A D depends on B,C E depends on C P(A,B,C,D,E)=P(A)P(B|A)P(C|A)

  38. A B C D E Simple Bayesian Network • MCBN1 A = only a priori B depends on A C depends on A D depends on B,C E depends on C P(A,B,C,D,E)=P(A)P(B|A)P(C|A)P(D|B,C)P(E|C) There exist algorithms for training, inference on BNs

  39. Naïve Bayes Model • Bayes’ Net: • Conditional independence of features given class Y f1 f2 f3 fk

  40. Naïve Bayes Model • Bayes’ Net: • Conditional independence of features given class Y f1 f2 f3 fk

  41. Naïve Bayes Model • Bayes’ Net: • Conditional independence of features given class Y f1 f2 f3 fk

  42. Hidden Markov Model • Bayesian Network where: • yt depends on

  43. Hidden Markov Model • Bayesian Network where: • yt depends on yt-1 • xt

  44. Hidden Markov Model • Bayesian Network where: • yt depends on yt-1 • xt depends on yt y1 y2 y3 yk x1 x2 x3xk

  45. Hidden Markov Model • Bayesian Network where: • yt depends on yt-1 • xt depends on yt y1 y2 y3 yk x1 x2 x3xk

  46. Hidden Markov Model • Bayesian Network where: • yt depends on yt-1 • xt depends on yt y1 y2 y3 yk x1 x2 x3xk

  47. Hidden Markov Model • Bayesian Network where: • yt depends on yt-1 • xt depends on yt y1 y2 y3 yk x1 x2 x3xk

  48. Generative Models • Both Naïve Bayes and HMMs are generative models

  49. Generative Models • Both Naïve Bayes and HMMs are generative models • We use the term generative model to refer to a directed graphical model in which the outputs topologically precede the inputs, that is, no x in X can be a parent of an output y in Y. • (Sutton & McCallum, 2006) • State y generates an observation (instance) x

  50. Generative Models • Both Naïve Bayes and HMMs are generative models • We use the term generative model to refer to a directed graphical model in which the outputs topologically precede the inputs, that is, no x in X can be a parent of an output y in Y. • (Sutton & McCallum, 2006) • State y generates an observation (instance) x • Maximum Entropy and linear-chain Conditional Random Fields (CRFs) are, respectively, their discriminative model counterparts