1 / 17

Jan 4 th 2013

Jan 4 th 2013. Event Extraction Using Distant Supervision Kevin Reschke. Event Extraction. … <Plane Crash> <Flight Number = Flight 14 > <Operator = Delta > <Fatalities = 40 > <Crash Site = Mississippi > …. “ … Delta Flight 14

thuraya-chaka
Télécharger la présentation

Jan 4 th 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jan 4th 2013 Event Extraction Using Distant Supervision Kevin Reschke

  2. Event Extraction … <Plane Crash> <Flight Number = Flight 14> <Operator = Delta> <Fatalities = 40> <Crash Site = Mississippi> … “… Delta Flight 14 crashed in Mississippi killing 40 …” Knowledge Base News Corpus

  3. Event Extraction Generate Candidates Flight 14 crashed in Mississippi. Run Named Entity Recognition on relevant docs 2) Classify Mentions 3) Aggregate Labels Features: (Unigram:Mississippi) (NEType:Location) (PrevWord:in) (ObjectOf:crashed) Label: CrashSite <Mississippi=CrashSite> <Mississippi=CrashSite> <Mississippi=CrashSite> <Mississippi=Operator> Final Label: CrashSite

  4. Training a Mention Classifier Need Labeled Training Data One year after [USAir]Operator [Flight 11]FlightNumber crashed in [Toronto]CrashSite, families of the [200]Fatalities victims attended a memorial service in [Vancouver]NIL. Problems: - Expensive - Does not scale

  5. Distant Supervision Solution: Use known events to automatically label training data. <Plane crash> <Flight Number = Flight 11> <Operator = USAir> <Fatalities = 200> <Crash Site = Toronto> Training Knowledge-Base One year after [USAir]Operator[Flight 11]FlightNumbercrashed in [Toronto]CrashSite, families of the [200]Fatalitiesvictims attended a memorial service in [Vancouver]NIL.

  6. Distant Supervision (High Level) • Begin with set of known facts. • Use this set to automatically label training instances from corpus. • Train and classify (handle noise)

  7. Distant Supervision for Relation Extraction • Slot filling for named entity relations. • Minz et al. 2009 (ACL); Surdeanu et al. 2011 (TAC-KBP). • Example: • Company: <ceo of>, <founder of>, <founding date>, <city of headquarters>, etc. • Known relations: founder_of(Steve Jobs, Apple) • Noisy Labeling Rule: Slot value and entity name must be in same sentence. • (+) Apple co-founder Steve Jobs passed away yesterday. • (-) Steve Jobs delivered the Stanford commencement address. • (+) Steve Jobs was fired from Apple in 1985.

  8. Distant Supervision for Event Extraction • Sentence level labeling rule doesn’t work. • Events lack proper names. • “The crash of USAir Flight 11” • Slots values occur separate from names. • The plane went down in central Texas. • 10 died and 30 were injured in yesterday’s tragic incident.

  9. Automatic Labeling: Event Extraction • Solution: Document Level Noisy Labeling Rule. • Heuristic: Use Flight Number as proxy for event name. • Labeling Rule: Slot value and Flight Number must appear in same document. …Flight 11 crash Sunday… …The plane went down in [Toronto]CrashSite… Training Fact: {<Flight Number = Flight 11>, <CrashSite= Toronto>}

  10. Evaluation: 80 plane crashes from Wikipedia infoboxes. Training set: 32; Dev set: 8; Test set: 40 Corpus: Newswire data from 1989 – present.

  11. Automatic Labeling • 38,000 Training Instances. • 39% Noise: • Examples: Good: At least 52 people survived the crash of the Boeing 737. Bad: First envisioned in 1964, the Boeing 737 entered service in 1968.

  12. Extraction Models • Local Model • Train and classify each mention independently. • Pipeline Model • Classify sequentially; use previous label as feature. • Captures dependencies between labels. • E.g., Passengers and Crew go together: “4 crew and 200 passengers were on board.” • Joint Model • Searn Algorithm (Daumé III et al., 2009). • Jointly models all mentions in a sentence.

  13. Results

  14. Label Aggregation Exhaustive Aggregation Four <NIL> Four <Crew>Four <Crew> Four <Fatalities> Four <NIL> Four <NIL> <Crew> <Fatalities>

  15. Label Aggregation: Noisy-OR • Key idea: Classifier gives us distribution over labels: • Stockholm <NIL:0.8; Site: 0.1, Crew:0.01, etc.> • Stockholm <Site: 0.5; NIL: 0.3, Crew:0.1, etc.> • Compute Noisy-OR for each label. • If Noisy-OR > threshold, use label.

  16. Results: Noisy-OR

  17. Next Step • Compare Distant Supervision with state of the art supervised approach (Huang & Rilloff, ACL-2011). • MUC-4 Shared Task: Terrorist Attacks. • Slot Template: <Location>, <Perpetrator>, <Instrument>, <Target>, <Victim> • Distant Supervision Source: • http://en.wikipedia.org/wiki/List_of_terrorist_incidents • Short summaries of several hundred terrorist attacks.

More Related