1 / 48

Markov Logic Networks

Markov Logic Networks. Speaker: Benedict Fehringer Seminar: Probabilistic Models for Information Extraction by Dr. Martin Theobald and Maximilian Dylla Based on Richards, M., and Domingos , P. (2006). Outline. Part 1: Why we need Markov Logic Networks (MLN’s) Markov Networks

paco
Télécharger la présentation

Markov Logic Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markov Logic Networks Speaker: Benedict Fehringer Seminar: Probabilistic Models for Information Extraction by Dr. Martin Theobald and Maximilian Dylla Based on Richards, M., and Domingos, P. (2006) Markov Logic Networks

  2. Outline • Part 1: Why we need Markov Logic Networks (MLN’s) • Markov Networks • First-Order Logic • Conclusion and Motivation • Part 2: How the MLN’s work? • Part 3: Are they better than other methods? Markov Logic Networks

  3. Outline • Part 1: Why we need Markov Logic Networks (MLN’s) • Markov Networks • First-Order Logic • Conclusion and Motivation • Part 2: How the MLN’s work? • Part 3: Are they better than other methods? Markov Logic Networks

  4. Markov Networks • Set of variables: • The distribution is given by: with as normalization factor and as potential function Markov Logic Networks

  5. Markov Networks • Representation as log-linear model: • In our case there will be only binary features: • Each feature corresponds to each possible state • The weight is equal to the log of the potential: Markov Logic Networks

  6. Little example scenario There is a company witch has a Playstation and each company’s employee has the right to play with it. If two employees are friends then the probability is high (ω=3) that both play all day long or both do not. If someone plays all day long then the chance for her/him is high (ω=2) to get fired. Markov Logic Networks

  7. Little example scenario There is a company witch has a Playstation and each company’s employee has the right to play with it. If two employees are friends then the probability is high (ω=3) that both play all day long or both do not. If someone plays all day long then the chance for her/him is high (ω=2) to get fired. Markov Logic Networks

  8. Markov Networks Has playing Friend Plays Fired One possibility… Markov Logic Networks

  9. Markov Networks Is Friend with Some playing Employee Plays Fired And another one. Markov Logic Networks

  10. Little example scenario There is a company witch has a Playstation and each company’s employee has the right to play with it. If two employees are friends then the probability is high (ω=3) that both play all day long or both do not. If someone plays all day long then the chance for her/him is high (ω=2) to get fired. If an employee A can convince another employee B to play, depends on the lability of B. For a high lability of B there is a higher probability (ω=4) than for a low lability (ω=2) MarkovLogic Networks

  11. Markov Networks Is labile Is Friend with Playing Person Plays Fired Could be. Markov Logic Networks

  12. Markov Networks • Advantages: • Efficiently handling uncertainty • Tolerant against imperfection and contradictory knowledge • Disadvantages: • Very complex networks for a wide variety of knowledge • Difficult to incorporate a wide range of domain knowledge Markov Logic Networks

  13. Outline • Part 1: Why we need Markov Logic Networks (MLN’s) • Markov Networks • First-Order Logic • Conclusion and Motivation • Part 2: How the MLN’s work? • Part 3: Are they better than other methods? Markov Logic Networks

  14. First-Order Logic • Four types of symbols: Constants: concrete object in the domain (e.g., people: Anna, Bob) Variables: range over the objects in the domain Functions: Mapping from tuples of objects to objects (e.g., GrandpaOf) Predicates: relations among objects in the domain (e.g., Friends) or attributes of objects (e.g. Fired) • Term: Any expression representing an object in the domain. Consisting of a constant, a variable, or a function applied to a tuple of terms. • Atomic formula or atom: predicate applied to a tuple of terms • Logical connectives and quantifier: Markov Logic Networks

  15. Translation in First-Order Logic If two employees are friends then the probability is high that both play all day long or both do not. In Clausal Form: If someone plays all day long then the chance for her/him is high to get fired. In Clausal Form: Markov Logic Networks

  16. First-Order Logic • Advantages: • Compact representation a wide variety of knowledge • Flexible and modularly incorporate a wide range of domain • knowledge • Disadvantages: • No possibility to handle uncertainty • No handling of imperfection and contradictory knowledge Markov Logic Networks

  17. Outline • Part 1: Why we need Markov Logic Networks (MLN’s) • Markov Networks • First-Order Logic • Conclusion and Motivation • Part 2: How the MLN’s work? • Part 3: Are they better than other methods? Markov Logic Networks

  18. Conclusion and Motivation Markov Networks First-Order Logic Efficiently handling uncertainty Tolerant against imperfection and contradictory knowledge Compact representation a wide variety of knowledge Flexible and modularly incorporate a wide range of domain knowledge → Combination of Markov Networks and First-Order Logic to use the advantages of both Markov Logic Networks

  19. Outline • Part 1: Why we need Markov Logic Networks (MLN’s) • Markov Networks • First-Order Logic • Conclusion and Motivation • Part 2: How the MLN’s work? • Part 3: Are they better than other methods? Markov Logic Networks

  20. Markov Logic Network Description of the problem Translation in First-Order Logic Construction of a MLN-”Template” Compute whatever you want Derive a concrete MLN for a given Set of Constants Markov Logic Networks

  21. Markov Logic Network Description of the problem Translation in First-Order Logic Construction of a MLN-”Template” Compute whatever you want Derive a concrete MLN for a given Set of Constants Markov Logic Networks

  22. Markov Logic Network- Translation in First-Order Logic - If two employees are friends then the probability is high that both play all day long or both do not. In Clausal Form: If someone plays all day long then the chance for her/him is high to get fired. In Clausal Form: Markov Logic Networks

  23. Markov Logic Network Description of the problem Translation in First-Order Logic Construction of a MLN-”Template” Compute whatever you want Derive a concrete MLN for a given Set of Constants Markov Logic Networks

  24. Markov Logic Network • Each formula matches one clique • Each formula owns a weight that reflects the importance of this formula • If a world violates one formula then it is less probable but not impossible • Concrete: The weight of this formula will be ignored (that means the weight is 0) Markov Logic Networks

  25. Markov Logic Network Formula to compare: Three Assumptions: • Unique Names • Domain closure • Known functions Markov Logic Networks

  26. Markov Logic Network Description of the problem Translation in First-Order Logic Construction of a MLN-”Template” Compute whatever you want Derive a concrete MLN for a given Set of Constants Markov Logic Networks

  27. Markov Logic Network Grounding: (with Constants c1 and c2) => => => Elimination of the existential quantifier Elimination of the universalquantifier Elimination of the functions MarkovLogic Networks

  28. Markov Networks Friends(A,B) Friends(A,A) Plays(A) Plays(B) Friends(B,B) Friends(B,A) Constants: Alice (A) and Bob (B) Markov Logic Networks

  29. Markov Logic Network Friends(A,B) Friends(A,A) Plays(A) Plays(A) Plays(B) Plays(B) Friends(B,B) Friends(B,A) Fired(A) Fired(B) Constants: Alice (A) and Bob (B) Markov Logic Networks

  30. Markov Logic Network Friends(A,B) Friends(A,A) Plays(A) Plays(B) Friends(B,B) Friends(B,A) Fired(A) Fired(B) Markov Logic Networks

  31. Markov Logic Network Description of the problem Translation in First-Order Logic Construction of a MLN-”Template” Compute whatever you want Derive a concrete MLN for a given Set of Constants Markov Logic Networks

  32. Markov Logic Network What is the probability that Alice and Bob are friends, both play playstation all day long but both are not getting fired? Friends(A,B)=1 Friends(A,A)=1 Plays(A)=1 Plays(B)=1 Friends(B,B)=1 Friends(B,A)=1 Fired(A)=0 Fired(B)=0 Markov Logic Networks

  33. Markov Logic Network What happens if limω→∞? If all formulas fulfilled: If not all formulas fulfilled: => Markov Logic Networks

  34. Markov Logic Network What happens if limω→∞? If all formulas fulfilled: Markov Logic Networks

  35. Markov Logic Network What happens if limω→∞? If not all formulas fulfilled: Markov Logic Networks

  36. Markov Logic Network What is the probability that a formula F1 holds given that formula F2 does? Markov Logic Networks

  37. Markov Logic Network Learning the weights: It is #P-complete to count the number of true groundings => approximation is necessary => using the pseudo-likelihood: Markov Logic Networks

  38. Outline • Part 1: Why we need Markov Logic Networks (MLN’s) • Markov Networks • First-Order Logic • Conclusion and Motivation • Part 2: How the MLN’s work? • Part 3: Are they better than other methods? Markov Logic Networks

  39. Experiment I Setting: • Using a database describing the Department of Computer Science and Engineering at the University of Washington • 12 predicates (e.g. Professor, Student, Area, AdvisedBy, …) • 2707 constants • 96 formulas (Knowledge base was provided by four volunteers who not know the database but were member of the department) • The whole database was divided into five subsets for each area (AI, graphics, programming languages, systems, theory) => in the end 521 true ground atoms of possible 58,457 Markov Logic Networks

  40. Experiment II Testing: • leave-one-out over the areas • Prediction of AdvisedBy(x,y) • Either with all or only partial (except Student(x) and Professor(x)) information • Drawing the precision/recall curves • Computation of the area under the curve (AUC) Markov Logic Networks

  41. Experiment III MLN was compared with: • Logic (only logical KB without probability) • Probability (only probability relations without special knowledge representations) • Naïve Bayes (NB) and Bayesian Network (BN) • Inductive logic programming (automatically development of the KB) • CLAUDIEN (clausal discovery engine) Markov Logic Networks

  42. Results Markov Logic Networks

  43. Results I all Areas AI Area Markov Logic Networks

  44. Results II graphics area prog. Language Area MarkovLogic Networks

  45. Results III systems area theory Area Markov Logic Networks

  46. Sample applications • Link Prediction • Link-Based Clustering • Social Network Modeling • … Markov Logic Networks

  47. Conclusion • MLN’s are a simple way to combine first-order logic and probability • They can be seen as a template for construction ordinary Markov Networks • Clauses can be learned by CLAUDIEN • Empirical tests with real-world data and knowledge are promising for the use of the MLN’s Markov Logic Networks

  48. Literature Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine Learning Journal, 62, 107-136. Markov Logic Networks

More Related