1 / 45

Supervised Relation Extraction as Classification Task

Learn about using kernel-based approaches for relation extraction, evaluating different datasets, and the impact of NER on RE. Explore various evaluation methodologies and the role of syntactic and local context in improving RE performance.

ina
Télécharger la présentation

Supervised Relation Extraction as Classification Task

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INTRODUCTION TO ARTIFICIAL INTELLIGENCE Massimo PoesioSupervised Relation Extraction

  2. RE AS A CLASSIFICATION TASK • Binary relations • Entities already manually/automatically recognized • Examples are generated for all sentences with at least 2 entities • Number of examples generated per sentence isNC2 – Combination of N distinct entities selected 2 at a time

  3. GENERATING CANDIDATES TO CLASSIFY

  4. RE AS A BINARY CLASSIFICATION TASK

  5. NUMBER OF CANDIDATES TO CLASSIFY – SIMPLE MINDED VERSION

  6. THE SUPERVISED APPROACH TO RE • Most current approaches to RE are kernel-based • Different information is used • Sequences of words, e.g., through the GLOBAL CONTEXT / LOCAL CONTEXT kernels of Bunescu and Mooney / GiulianoLavelli & Romano • Syntactic information through the TREE KERNELS of Zelenko et al / Moschitti et al • Semantic information in recent work

  7. KERNEL METHODS: A REMINDER • Embedding the input data in a feature space • Using a linear algorithm for discovering non-linear patterns • Coordinates of images are not needed, only pairwise inner products • Pairwiseinner products can be efficiently computed directly from X using a kernel function K:X×X→R

  8. MODULARITY OF KERNEL METHODS

  9. THE WORD-SEQUENCE APPROACH • Shallow linguistic Information: • tokenization • Lemmatization • sentence splitting • PoStagging Claudio Giuliano, Alberto Lavelli, and Lorenza Romano (2007), FBK-IRST: Kernel methods for relation extraction, Proc. Of SEMEVAL-2007

  10. LINGUISTIC REALIZATION OF RELATIONS Bunescu & Mooney, NIPS 2005

  11. WORD-SEQUENCE KERNELS • Two families of “basic” kernels • Global Context • Local Context • Linear combination of kernels • Explicit computation • Extremely sparse input representation

  12. THE GLOBAL CONTEXT KERNEL

  13. THE GLOBAL CONTEXT KERNEL

  14. THE LOCAL CONTEXT KERNEL

  15. LOCAL CONTEXT KERNEL (2)

  16. KERNEL COMBINATION

  17. EXPERIMENTAL RESULTS • Biomedical data sets • AIMed • LLL • Newspaper articles • Roth and Yih • SEMEVAL 2007

  18. EVALUATION METHODOLOGIES

  19. EVALUATION (2)

  20. EVALUATION (3)

  21. EVALUATION (4)

  22. RESULTS ON AIMED

  23. OTHER APPROACHES TO RE • Using syntactic information • Using lexical features

  24. Syntactic information for RE • Pros: • more structured information useful when dealing with long-distance relations • Cons: • not always robust • (and not available for all languages)

  25. Zelenko et al JMLR 2003 • TREE KERNEL defined over a shallow parse tree representation of the sentences • approach vulnerable to unrecoverable parsing errors • data set: 200 news articles (not publicly available) • two types of relations : person-affiliation and organization-location

  26. ZELENKO ET AL

  27. CULOTTA & SORENSEN 2004 • generalized version of Zelenko’s kernel based on dependency trees (smallest dependency tree containing the two entities of the relation) • a bag-of-words kernel is used to compensate syntactic errors • data set: ACE 2002 & 2003 • results: syntactic information improves performance w.r.t. bag-of-words (good precision but low recall)

  28. CULOTTA AND SORENSEN (2)

  29. EVALUATION CAMPAIGNS FOR RE • Much of modern evaluation of methods is done by competing with other teams on evaluation campaigns like MUC and ACE • Modern evaluation campaigns for RE: SEMEVAL (now *SEM) • Interesting to look also at the problems of • DATA CREATION • EVALUATION METRICS

  30. SEMEVAL 2007 • 4th International Workshop on Semantic Evaluations • Task 04: Classification of Semantic Relations between Nominals • organizers: Roxana Girju, Marti Hearst, PreslavNakov, ViviNastase, Stan Szpakowicz, Peter Turney, DenizYuret • 14 participating teams

  31. SEMEVAL 2007: THE RELATIONS

  32. SEMEVAL 2007: DATASET CREATION

  33. SEMEVAL 2007: DATASET CREATION (2)

  34. SEMEVAL 2007 – DATASET CREATION (3)

  35. SEMEVAL 2007 – DATASET CREATION (4)

  36. SEMEVAL 2007: DATASET

  37. SEMEVAL 2007: COMPETITION

  38. SEMEVAL 2007: COMPETITION (2)

  39. SEMEVAL 2007: BEST RESULTS

  40. INFLUENCE OF NER ON RE

  41. INFLUENCE OF NER ON RE (2)

  42. GENERATING CANDIDATES

  43. GENERATING CANDIDATES

  44. GENERATING CANDIDATES

  45. ACKNOWLEDGMENTS • Many slides borrowed from • Roxana Girju • Alberto Lavelli

More Related