Download
a metric based framework for automatic taxonomy induction n.
Skip this Video
Loading SlideShow in 5 Seconds..
A Metric-based Framework for Automatic Taxonomy Induction PowerPoint Presentation
Download Presentation
A Metric-based Framework for Automatic Taxonomy Induction

A Metric-based Framework for Automatic Taxonomy Induction

433 Vues Download Presentation
Télécharger la présentation

A Metric-based Framework for Automatic Taxonomy Induction

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Hui Yang and Jamie Callan Language Technologies InstituteCarnegie Mellon UniversityACL2009, Singapore A Metric-based Framework for Automatic Taxonomy Induction

  2. Roadmap • Introduction • Related Work • Metric-Based Taxonomy Induction Framework • The Features • Experimental Results • Conclusions

  3. Introduction • Semantic taxonomies, such as WordNet, play an important role in solving knowledge-rich problems • Limitations of Manually-created Taxonomies • Rarely complete • Difficult to include new terms from emerging/changing domains • Time-consuming to create; May make it unfeasible for specialized domains and personalized tasks

  4. Introduction • Automatic Taxonomy Induction is a solution to • Augment existing resources • Quickly produce new taxonomies for specialized domains and personalized tasks • Subtasks in Automatic Taxonomy Induction • Term extraction • Relation formation • This paper focuses on Relation Formation

  5. Related Work • Clustering-based Approaches • Hierarchically cluster terms based on similarities of their meanings usually represented by a feature vector • Have only been applied to extract is-a and sibling relations • Strength: Allowing discovery of relations which do not explicitly appear in text; higher recall • Weaknesses: Generally fail to produce coherent cluster for small corpora [Pantel and Pennacchiotti 2006]; Hard to label non-leaf nodes Pattern-based Approaches Define lexical-syntactic patterns for relations, and use these patterns to discover instances Have been applied to extract Is-a, part-of, sibling, synonym, causal, etc, relations Strength: Highly accurate Weakness: Sparse coverage of patterns

  6. A unified solution Metric-based Taxonomy Induction • Combine strengths of both approaches in a unified framework • Flexibly incorporate heterogeneous features • Use lexical-syntactic patterns as one types of features in a clustering framework

  7. THE FRAMEWORK • A novel framework, which • Incrementally clusters terms • Transforms taxonomy induction into a multi-criteria optimization • Using heterogeneous features • Optimization based on two criteria • Minimization of taxonomy structures  Minimum Evolution Assumption • Modeling of term abstractness  Abstractness Assumption

  8. Let’s Begin with Some Important Definitions • A Taxonomy is a data model T = (C,R | D) Concept Set Relationship Set Domain

  9. More Definitions A Full Taxonomy: Game Equipment AssignedTermSet={game equipment, ball, table, basketball, volleyball, soccer, table-tennis table, snooker table} UnassignedTermSet={} ball table

  10. More Definitions A Partial Taxonomy Game Equipment AssignedTermSet={game equipment, ball, table, basketball, volleyball} UnassignedTermSet={soccer, table-tennis table, snooker table} ball table

  11. More Definitions Ontology Metric d( , ) = 2 distance = 1.5 distance = 2 ball distance =1 d( , ) = 1 distance =1 d( , ) = 4.5 table

  12. Assumptions Minimum Evolution Assumption: The Optimal Ontology is One that Introduces Least Information Changes!

  13. Illustration Minimum Evolution Assumption

  14. Illustration Minimum Evolution Assumption

  15. Illustration Minimum Evolution Assumption ball

  16. Illustration Minimum Evolution Assumption ball table

  17. Illustration Minimum Evolution Assumption Game Equipment ball table

  18. Illustration Minimum Evolution Assumption Game Equipment ball table

  19. Illustration Minimum Evolution Assumption Game Equipment ball table

  20. Assumptions Abstractness Assumption: Each abstraction level has its own Information function

  21. Assumptions Abstractness Assumption Info3(.) Game Equipment ball Info2(.) table Info1(.)

  22. Multiple Criterion Optimization Minimum Evolution objective function Abstractness objective function Scalarization variable

  23. Estimating Ontology Metric • Assume ontology metric is a linear interpolation of some underlying feature functions • Ridge Regression to estimate and predict the ontology metric

  24. THE FEATURES • Our framework allows a wide range of features to be used • Input for the Feature Functions: Two terms • Output: A numeric score to measure semantic distance between these two terms • We can use the following types of feature functions, but not restricted to only these: • Contextual Features • Term Co-occurrence • Lexical-Syntactic Patterns • Syntactic Dependency Features • Word Length Difference • Definition Overlap, etc

  25. Experimental Results • Task: Reconstruct taxonomies from WordNet and ODP • Not the entire WordNet or ODP, but fragments of WordNet or ODP • Ground Truth: 50 hypernym taxonomies from WordNet; 50 hypernym taxonomies from ODP; 50 meronym taxonomies from WordNet. • Auxiliary Datasets: 1000 Google documents per term or per term pair; 100 Wikipedia documents per term. • Evaluation Metrics: F1-measure (averaged by Leave-One-Out Cross Validation).

  26. Datasets

  27. Performance of taxonomy induction • Compare our system (ME) with other state-of-the-art systems • HE: 6 is-a patterns [Hearst 1992] • GI: 3 part-of patterns [Girju et al. 2003] • PR: a probabilistic framework [Snow et al. 2006] • ME: our metric-based framework

  28. Performance of taxonomy induction • Our system (ME) consistently gives the best F1 for all three tasks. • Systems using heterogeneous features (ME and PR) achieve a significant absolute F1 gain (>30%)

  29. Features vs. relations • This is the first study of the impact of using different features on taxonomy induction for different relations • Co-occurrence and lexico-syntactic patterns are good for is-a, part-of, and sibling relations • Contextual and syntactic dependency features are only good for sibling relation

  30. Features vs. abstractness • This is the first study of the impact of using different features on taxonomy induction for terms at different abstraction levels • Contextual, co-occurrence, lexical-syntactic patterns, and syntactic dependency features work well for concrete terms; • Only co-occurrence works well for abstract terms

  31. Conclusions • This paper presents a novel metric-based taxonomy induction framework, which • Combines strengths of pattern-based and clustering-based approaches • Achieves better F1 than 3 state-of-the-art systems • The first study on the impact of using different features on taxonomy induction for different types of relations and for terms at different abstraction levels

  32. Conclusions • This work is a general framework, which • Allows a wider range of features • Allows different metric functions at different abstraction levels • This work has a potential to learn more complex taxonomies than previous approaches

  33. THANK YOU AND QUESTIONShuiyang@cs.cmu.educallan@cs.cmu.edu

  34. Extra Slides

  35. FORMAL FORMULATION OF TAXONOMY INDUCTION • The Task of Taxonomy Induction: • The construction of a full ontology T given a set of concepts C and an initial partial ontology T0 • Keeping adding concepts in C into T0 • Note T0 could be empty • Until a full ontology is formed

  36. GOAL OF TAXONOMY INDUCTION • Find the optimal full ontology s.t. the information changes since T0 are least , i.e., • Note that this is by the Minimum Evolution Assumption

  37. Get to the Goal • Goal: Since the optimal set of concepts is always C Concepts are added incrementally

  38. Get to the Goal Plug in definition of information change Transform into a minimization problem Minimum Evolution objective function

  39. Explicitly Model Abstractness • Model Abstractness for each Level by Least Square Fit Plug in definition of amount of information for an abstraction level Abstractness objective function

  40. The Optimization Algorithm

  41. More Definitions Information in an Taxonomy T ∑ d( , ) = 2 distance = 1.5 distance = 2 ball distance =1 d( , ) = 1 distance =1 d( , ) = 4.5 table

  42. More Definitions Information in a Level L ∑ d( , ) = 2 ball d( , ) = 1 ball d( , ) = 1

  43. EXAMPLES OF FEATURES • Contextual Features • Global Context KL-Divergence = KL-Divergence(1000 Google Documents for Cx, 1000 Google Documents for Cy); • Local Context KL-Divergence = KL-Divergence(Left two and Right two words for Cx, Left two and Right two words for Cy). • Term Co-occurrence • Point-wise Mutual Information (PMI) • = # of sentences containing the term(s); or # of documents containing the term(s); or n as in “Results 1-10 of about n for …” in Google.

  44. EXAMPLES OF FEATURES • Syntactic Dependency Features • Minipar Syntactic Distance = Average length of syntactic paths in syntactic parse trees for sentences containing the terms; • Modifier Overlap = # of overlaps between modifiers of the terms; e.g., red apple, red pear; • Object Overlap = # of overlaps between objects of the terms when the terms are subjects; e.g., A dog eats apple; A cat eats apple; • Subject Overlap = # of overlaps between subjects of the terms when the terms are objects; e.g., A dog eats apple; A dog eats pear; • Verb Overlap = # of overlaps between verbs of the terms when the terms are subjects/objects; e.g., A dog eats apple; A cat eats pear.

  45. EXAMPLES OF FEATURES • Lexical-Syntactic Patterns

  46. EXAMPLES OF FEATURES • Miscellaneous Features • Definition Overlap = # of non-stopword overlaps between definitions of two terms. • Word Length Difference