1 / 79

TERQAS: Time and Event Recognition for Question Answering Systems

TERQAS: Time and Event Recognition for Question Answering Systems. TERQAS Group Final Review ARDA Workshop NRRC/MITRE July 22, 2002. TERQAS: 2002 Workshop Schedule. January 30-31: Kick-off Meeting; Setting Agenda March 11-15: Corpus Selection, Query Studies, TimeML

effie
Télécharger la présentation

TERQAS: Time and Event Recognition for Question Answering Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TERQAS:Time and Event Recognition for Question Answering Systems TERQAS Group Final Review ARDA Workshop NRRC/MITRE July 22, 2002

  2. TERQAS:2002 Workshop Schedule January 30-31: Kick-off Meeting; Setting Agenda March 11-15: Corpus Selection, Query Studies, TimeML April 22-26: TimeML Specification, Corpus Work May 8-15: Annotation Fest June 10-20: Algorithm Specification, Annotation July 15-22: Wrap-up and Evaluation Aug-Sept: Prepare Final Report time2002.org

  3. Relevance to Question Answering Systems • Is Gates currently CEO of Microsoft? • Were there any meetings between the terrorist hijackers and Iraq before the WTC event? • Did the Enron merger with Dynegy take place? • How long did the hostage situation in Beirut last? • When did the war between Iran and Iraq end? • When did John Sununu travel to a fundraiser for John Ashcroft? • How many Tutsis were killed by Hutus in Rwanda in 1994? • Who was Secretary of Defense during the Gulf War? • What was the largest U.S. military operation since Vietnam? • When did the astronauts return from the space station?

  4. Workshop Goals • TimeML: Define and Design a Metadata Standard for Markup of events, their temporal anchoring, and how they are related to each other in News articles. • TIMEBANK: Given the specification of TimeML, create a gold standard corpus of 300 articles marked up for temporal expressions, events, and basic temporal relations.

  5. Working Groups • TimeML Definition and Specification • Algorithm Review and Development • Article Corpus Collection Development • Query Corpus Development and Classification • TIMEBANK Annotation • TimeML and Algorithm Evaluation

  6. James Pustejovsky, PI Rob Gaizauskas Graham Katz Bob Ingria José Castaño Inderjeet Mani Antonio Sanfilippo Dragomir Radev Patrick Hanks Marc Verhagen Beth Sundheim Andrea Setzer Jerry Hobbs Bran Boguraev Andy Latto John Frank Lisa Ferro Marcia Lazo Roser Saurí Anna Rumshisky David Day Luc Belanger Harry Wu Andrew See TERQAS Participants Supported by

  7. Presentation Outline • TimeML 1.0 Specification • T3PO Algorithm Development • Tool Development Effort • TIMEBANK Annotation Status • Query Corpus Development and Classification • TIMEBANK Annotation • Future Projects

  8. TimeML 1.0 • Adopts the core of Setzer’s annotation framework (Sheffield Temporal Annotation Guidelines, STAG) • Remains compliant (as much as possible) with TIDES TIMEX2 annotation. • Introduces a TLINK tag: an object that links events/times to events/times. • Introduces an ALINK tag: an object that associates aspectual phases to events. • Introduces an SLINK tag: an object that subordinates events within modality, negation, or another event. • Enrich temporal relations: adds i-after, i-before, and aspectual relations. • Introduces eventidentity. • Introduces Temporal functions for doing temporal math without evaluation. • Introduces STATE as a possible event class.

  9. How TimeML Differs from Previous Markups • Extends TIMEX2 annotation; • Temporal Functions: three years ago • Anchors to events and other temporal expressions: • Identifies signals determining interpretation of temporal expressions; • Temporal Prepositions:for, during, on, at; • Temporal Connectives: before, after, while. • Identifies event expressions; • tensed verbs; has left, was captured, will resign; • stative adjectives; sunken, stalled, on board; • event nominals; merger, Military Operation, Gulf War; • Creates dependencies between events and times: • Anchoring; John left on Monday. • Orderings; The party happened after midnight. • Embedding; John said Mary left.

  10. Annotation in an Extension of STAG FAMILIES SUE OVER AREOFLOT CRASH DEATHS The Russian airline Aeroflot has been <EVENT eid=1 relatedToTime=1 timeRelType=BEFORE tense=PRESENT aspect=PERFECTIVE class=OCCURRENCE> hit </EVENT> with a writ for loss and damages, <EVENT eid=2 tense=NONE aspect=PERFECTIVE relatedToEvent=1 eventRelType=BEFORE class=OCCURRENCE> filed </EVENT> in Hong Kong by the families of seven passengers <EVENT eid=3 tense=NONE aspect=PERFECTIVE relatedToEvent=2 eventRelType=BEFORE class=OCCURRENCE relatedToEvent2=4 eventRel2Type=IS_INCLUDED signal2=1> killed </EVENT> <SIGNAL sid=1> In </SIGNAL> an air <EVENT eid=4 class=OCCURRENCE> crash </EVENT>.

  11. STAG Annotation, cont. All 75 people on board the Aeroflot Airbus <EVENT eid=5 tense=PAST aspect=PERFECTIVE relatedToEvent=6 eventRelType=IAFTER signal=2> died </EVENT> <SIGNAL sid=2> when </SIGNAL> it <EVENT eid=6 tense=PAST aspect=PERFECTIVE relatedToTime=2 timeRelType=IS_INCLUDED relatedToEvent=4 eventRelType=ID> ploughed </EVENT> into a Siberian mountain <SIGNAL sid=3> in </SIGNAL> <TIMEX tid=2 type=DATE calDate=031994> March 1994 </TIMEX>.

  12. Drawbacks of Event-Internal Relations in STAG • Triple attribute structure in EVENT: [([signalID] relatedToEvent eventRelType) | ([signalID] relatedToTime timeRelType)] • Same attribute structure appears in TIMEX: [(eid signalID relType)] • These three attributes are logically linked, allowingeventRelType, eventRelType,andeventRelTypeto be collapsed into singleattribute.

  13. EVENT attributes ::= eid class tense aspect eid ::= ID {eid ::= EventID EventID ::= e<integer>} class ::= 'OCCURRENCE' | 'PERCEPTION' | 'REPORTING' | 'ASPECTUAL' | 'STATE' | 'I_STATE' | 'I_ACTION' | 'MODAL' tense ::= 'PAST' | 'PRESENT' | 'FUTURE' | 'NONE' aspect ::= 'PROGRESSIVE' | 'PERFECTIVE' | 'PERFECTIVE_PROGRESSIVE' | 'NONE'

  14. TimeML Event Classes • Occurrence: • die, crash, build, merge, sell, take advantage of, .. • State: • Be on board, kidnapped, recovering, love, .. • Reporting: • Say, report, announce, • I-Action: • Attempt, try,promise, offer • I-State: • Believe, intend, want, … • Aspectual: • begin, start, finish, stop, continue. • Perception: • See, hear, watch, feel.

  15. The young industry's rapid growth also is attracting regulators eager to police its many facets. The young industry's rapid <EVENT eid="e1" class="OCCURRENCE"> growth </EVENT> also is <EVENT eid="e2" class="OCCURRENCE"> attracting </EVENT> regulators <EVENT eid="e4" class="I_STATE"> eager </EVENT> to <EVENT eid="e5" class="OCCURRENCE"> police </EVENT> its many facets.

  16. Israel will ask the United States to delay a military strike against Iraq until the Jewish state is fully prepared for a possible Iraqi attack. Israel will <EVENT eid="e1" class="I_ACTION"> ask </EVENT> the United States to <EVENT eid="e2" class="I_ACTION"> delay </EVENT> a military <EVENT eid="e3" class="OCCURRENCE"> strike </EVENT> against Iraq until the Jewish state is fully <EVENT eid="e4" class="I_STATE"> prepared </EVENT> for a possible Iraqi <EVENT eid="e5" class="OCCURRENCE"> attack </EVENT>

  17. TIMEX2 Tag Attributes

  18. Temporal Functions Temporal expressions where the calendar date is not referred to directly, but via an expression that acts as a temporal function over a TIMEX3 expression. Examples: • last week • last Thursday • the week before last • next week

  19. Pre-theoretic Treatment:DCT=DocCreationTime • last week = (predecessor (week DCT)) That is, we start with a temporal anchor, in this case, the DCT, coerce it to a week, than find the week preceding it. • last Thursday = (thursday (predecessor (week DCT)) Similar to the preceding expression, except that we pick out the day named 'thursday' in the predecessor week. • the week before last = (predecessor (predecessor (week DCT))) Also similar to the first expression, except that we go back two weeks. • next week = (successor (week DCT)) The dual of the first expression: we start with the same coercion, but go forward instead of back.

  20. TIMEX2 Annotation Sen. Alton Waldon, who served briefly in Congress <TIMEX2 VAL="199” MOD="BEFORE"> more than a decade ago</TIMEX2>, is <TIMEX2 VAL="PRESENT_REF"> Now </TIMEX2> retired.

  21. TimeML Treatment of Temporal Functions Sen. Alton Waldon, who served briefly in Congress more than a decade ago, is now retired. Sen. Alton Waldon, who <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> served </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> briefly in Congress <TIMEX3 tid="t1" type=”DATE" value=”199" mod=“BEFORE” temporalfunction=“TRUE”> more than a decade ago </TIMEX3> is <TIMEX3 tid="t2" type="DATE" value="PRESENT_REF"> now </TIMEX3> <EVENT eid="e2" class="STATE" tense="NONE" aspect="NONE"> retired </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/>. <TLINK eventInstanceID="ei1" relatedToTime="t1" relType=”IS_INCLUDED" /> <TLINK eventInstanceID="ei2" relatedToTime="t2" relType="HOLDS"/> <TLINK eventInstanceID="ei1" relatedToEvent=”ei2" relType=”BEFORE"/>

  22. Temporal Functions: Alternative Analysis Sen. Alton Waldon, who <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> served </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> briefly in Congress <TIMEX3 tid="t1" type="DURATION" value="P1E" mod="MORE_THAN"> more than a decade </TIMEX3> <SIGNAL sid="s1"> ago </SIGNAL>, is <TIMEX3 tid="t2" type="DATE" value="PRESENT_REF"> now </TIMEX3> <EVENT eid="e2" class="STATE" tense="NONE" aspect="NONE"> retired </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/>. <TLINK eventInstanceID="ei1" relatedToTime="t2" signalID="s1" relType="BEFORE" magnitude="t1"/> <TLINK eventInstanceID="ei2" relatedToTime="t2" relType="HOLDS"/>

  23. TLINK TLINK or Temporal Link represents the temporal relationship holding between events or between an event and a time, and establishes a link between the involved entities, making explicit if they are: Simultaneous (happening at the same time) Identical: (referring to the same event) John drove to Boston. During his drive he ate a donut. 3. One before the other: The police looked into the slayings of 14 women. In six of the cases suspects have already been arrested. 4. One after the other: 5. One immediately before the other: All passengers died when the plane crashed into the mountain 6. One immediately after than the other: 7. One including the other: John arrived in Boston last Thursday. 8. One being included in the other: 9. One holding during the duration of the other: 10. One being the beginning of the other: John was in the gym between 6:00 p.m. and 7:00 p.m. 11. One being begun by the other: 12. One being the ending of the other: John was in the gym between 6:00 p.m. and 7:00 p.m.. 13. One being ended by the other:

  24. SLINK SLINK or Subordination Link is used for contexts introducing relations between two events, or an event and a signal, of the following sort: 1. Modal: Relation introduced mostly by modal verbs (should, could, would, etc.) and events that introduce a reference to a possible world --mainly I_STATEs: John should have bought some wine. Mary wanted John to buy some wine. 2. Factive: Certain verbs introduce an entailment (or presupposition) of the argument's veracity. They include forget in the tensed complement, regret, manage: John forgot that he was in Boston last year. Mary regrets that she didn't marry John. John managed to leave the party 3. Counterfactive: The event introduces a presupposition about the non-veracity of its argument: forget (to), unable to (in past tense), prevent, cancel, avoid, decline, etc. John forgot to buy some wine. Mary was unable to marry John. John prevented the divorce. 4. Evidential: Evidential relations are introduced by REPORTING or PERCEPTION: John said he bought some wine. Mary saw John carrying only beer. 5. Negative evidential: Introduced by REPORTING (and PERCEPTION?) events conveying negative polarity: John denied he bought only beer. 6. Negative: Introduced only by negative particles (not, nor, neither, etc.), which will be marked as SIGNALs, with respect to the events they are modifying: John didn't forgot to buy some wine. John did not wanted to marry Mary.

  25. ALINK ALINK or Aspectual Link represent the relationship between an aspectual event, which will be annotated as a SIGNAL (section 2.3), and its argument event. Examples of the possible aspectual relations we will encode are: 1. Initiation: John started to read 2. Culmination: John finished assembling the table. 3. Termination: John stopped talking. 4. Continuation: John kept talking.

  26. Causation: 1 (1) The rains caused the flooding. The <EVENT eid="e1" class="OCCURRENCE" tense="NONE" aspect="NONE"> rains </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <EVENT eid="e2" class="OCCURRENCE" tense="PAST" aspect="NONE"> caused </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> the <EVENT eid="e3" class="OCCURRENCE" tense="NONE" aspect="NONE"> flooding </EVENT> <MAKEINSTANCE eiid="ei3" eventID="e3"/> <TLINK eventInstanceID="ei1" relatedToEvent="ei2" relType="IDENTITY"/> <TLINK eventInstanceID="ei2" relatedToEvent="ei3" relType="BEFORE"/>

  27. Causation: 2 (2') Kissinger secured the peace at great cost. Kissinger <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> secured </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> the <EVENT eid="e2" class="OCCURRENCE" tense="NONE" aspect="NONE"> peace </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> at great cost. <TLINK eventInstanceID="ei1" relatedToEvent="ei2" relType="BEFORE"/>

  28. Causation: 3 (3) He kicked the ball, and it rose into the air. He <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> kicked </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> the ball, and it <EVENT eid="e2" class="OCCURRENCE" tense="NONE" aspect="NONE"> rose </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> into the air. <TLINK eventInstanceID="ei1" relatedToEvent="ei2" relType="BEFORE"/>

  29. TLINK: 1 (4) John taught 20 minutes every Monday. John <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1" signalID="s1" cardinality="EVERY"/> <TIMEX3 tid="t1" type="DURATION" value="PT20M"> 20 minutes </TIMEX3> <SIGNAL sid="s1"> every </SIGNAL> <TIMEX3 tid="t2" type="DATE" value="XXXX-WXX-1"> Monday </TIMEX3> <TLINK eventInstanceID="ei1" relatedToTime="t1" relType="HOLDS"/> <TLINK eventInstanceID="ei1" relatedToTime="t2" relType="IS_INCLUDED"/>

  30. TLINK: 2 (6) John taught twice on Monday but only once on Tuesday John <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1" signalID="s1"/> <MAKEINSTANCE eiid="ei2" eventID="e1" signalID="s1"/> <MAKEINSTANCE eiid="ei3" eventID="e1" signalID="s2"/> <SIGNAL sid="s1"> twice </SIGNAL> <SIGNAL sid="s3"> on </SIGNAL> <TIMEX3 tid="t1" type="DATE" temporalFunction="true" value="XXXX-WXX-1"> Monday </TIMEX3> But only <SIGNAL sid="s2"> once </SIGNAL> <SIGNAL sid="s4"> on </SIGNAL> <TIMEX3 tid="t2" type="DATE" temporalFunction="true" value="XXXX-WXX-2"> Tuesday </TIMEX3> <TLINK eventInstanceID="ei1" signalID="s3" relatedToTime="t1" relType="IS_INCLUDED"/> <TLINK eventInstanceID="ei2" signalID="s3" relatedToTime="t1" relType="IS_INCLUDED"/> <TLINK eventInstanceID="ei3" signalID="s4" relatedToTime="t2" relType="IS_INCLUDED"/>

  31. TLINK: 3 (7) John taught 5 minutes after the explosion. <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <TIMEX3 tid="t1" type="DURATION" value="PT5M"> 5 minutes </TIMEX3> <SIGNAL sid="s1"> after </SIGNAL> the <EVENT eid="e2" class="OCCURRENCE" tense="NONE" aspect="NONE"> explosion </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> <TLINK eventInstanceID="ei1" signalID="s1" relatedToEvent="ei2" relType="AFTER" magnitude="t1"/>

  32. TLINK: 4 (8) John taught from 1992 through 1995. John <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <SIGNAL sid="s1"> from </SIGNAL> <TIMEX3 tid="t1" type="DATE" value="1992"> 1992 </TIMEX3> <SIGNAL sid="s2"> through </SIGNAL> <TIMEX3 tid="t2" type="DATE" value="1995"> 1995 </TIMEX3> <TLINK eventInstanceID="ei1" signalID="s1" relatedToTime="t1" relType="BEGUN_BY"/> <TLINK eventInstanceID="ei1" signalID="s2" relatedToTime="t2" relType="ENDED_BY"/>

  33. TLINK: 5 (9) John taught from September to December last year. John <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <SIGNAL sid="s1"> from </SIGNAL> <TIMEX3 tid="t1" type="DATE" temporalFunction="true" value="XXXX-09"> September </TIMEX3> <SIGNAL sid="s2"> To </SIGNAL> <TIMEX3 tid="t2" type="DATE" temporalFunction="true" value="XXXX-12"> December </TIMEX3> <TIMEX3 tid="t3" type="DATE" temporalFunction="true" value="XXXX" anchorTimeID="t4"> last year </TIMEX3> <TIMEX3 tid="t4" type="DATE" functionInDocument="CREATION_TIME" value="1996-03-27"> 03-27-96 </TIMEX3> <TLINK eventInstanceID="ei1" signalID="s1" relatedToTime="t1" relType="BEGUN_BY"/> <TLINK eventInstanceID="ei1" signalID="s2" relatedToTime="t2" relType="ENDED_BY"/>

  34. SLINK: 1 (12) John taught on Monday but not on Tuesday John <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1" signalID="s3"/> <MAKEINSTANCE eiid="ei2" eventID="e1" signalID="s4"/> <SIGNAL sid="s3"> on </SIGNAL> <TIMEX3 tid="t1" type="DATE" temporalFunction="true" value="XXXX-WXX-1"> Monday </TIMEX3> but <SIGNAL sid="s1"> not </SIGNAL> <SIGNAL sid="s4"> on </SIGNAL> <TIMEX3 tid="t2" type="DATE" temporalFunction="true" value="XXXX-WXX-2"> Tuesday </TIMEX3> <TLINK eventInstanceID="ei1" relatedToTime="t1" signalID="s3" relType="IS_INCLUDED"/> <TLINK eventInstanceID="ei2" relatedToTime="t2" signalID="s4" relType="IS_INCLUDED"/> <SLINK subordinatedEventInstance="ei2" signalID="s1" relType="NEGATIVE"/>

  35. (13) If Graham leaves today, he will not hear Sabine. <SIGNAL sid="s1"> if </SIGNAL> Graham <EVENT eid="e1" class="OCCURRENCE" tense="PRESENT" aspect="NONE"> leaves </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <SLINK subordinatedEvent="e1" signalID="s1" relType="MODAL"/> <TIMEX3 tid="t1" type="DATE" temporalFunction="true" value="XXXX-XX-XX"> today </TIMEX3> he <EVENT eid="e3" class="MODAL" tense="NONE" aspect="NONE"> will </EVENT> <MAKEINSTANCE eiid="ei3" eventID="e3"/> <SIGNAL sid="s2"> not </SIGNAL> <EVENT eid="e2" class="OCCURRENCE" tense="FUTURE" aspect="NONE"> hear </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> Sabine. <SLINK eventInstanceID="ei3" subordinatedEvent="e2" relType="MODAL"/> <TLINK eventInstanceID="ei1" relatedToEvent="ei2" relType="BEFORE"/> <SLINK subordinatedEvent="e2" signalID="s1" relType="NEGATIVE"/> SLINK: 2

  36. SLINK: 3 (14) Bill denied that John taught on Monday. Bill <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> denied </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> that <SLINK eventInstanceID="ei1" subordinatedEvent="e2" relType="NEG_EVIDENTIAL"/> John <EVENT eid="e2" class="OCCURRENCE" tense="PAST" aspect="NONE"> taught </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> <SIGNAL sid="s1"> on </SIGNAL> <TIMEX3 tid="t1" type="DATE" temporalFunction="true" value="XXXX-WXX-1"> Monday </TIMEX3> <TLINK eventInstanceID="ei2" relatedToTime="t1" relType="IS_INCLUDED"/>

  37. SLINK: 4 (15) Bill wants to teach on Monday. Bill <EVENT eid="e1" class="I_STATE" tense="PRESENT" aspect="NONE"> wants </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <SLINK eventInstanceID="ei1" signalID="s1" subordinatedEvent="e2" relType="MODAL"/> <SIGNAL sid="s1"> to </SIGNAL> <EVENT eid="e2" class="OCCURRENCE" tense="NONE" aspect="NONE"> teach </EVENT> <MAKEINSTANCE eiid="ei2" eventID="e2"/> <SIGNAL sid="s2"> on </SIGNAL> <TIMEX3 tid="t1" type="DATE" temporalFunction="true" value="XXXX-WXX-1"> Monday </TIMEX3> <TLINK eventInstanceID="ei2" relatedToTime="t1" relType="IS_INCLUDED"/>

  38. ALINK: 1 (17) The boat began to sink. The boat <EVENT eid="e1" class="ASPECTUAL" tense="PAST" aspect="NONE"> began </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <SIGNAL sid="s1"> to </SIGNAL> <EVENT eid="e2" class="OCCURRENCE" tense="NONE" aspect= "NONE"> sink </EVENT> <ALINK eventInstanceID="ei1" signalID="s1" relatedToEvent="e2" relType="INITIATES"/>

  39. ALINK: 2 (18) The search party stopped looking for the survivors. The search party <EVENT eid="e1" class="ASPECTUAL" tense="PAST" aspect="NONE"> stopped </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <EVENT eid="e2" class="OCCURRENCE" tense="NONE" aspect="PROGRESSIVE"> looking </EVENT> <ALINK eventInstanceID="ei1" relatedToEvent="e2" relType="TERMINATES"/> for the survivors

  40. time2002.org • DTD created • TimeML.dtd • Schema created • TimeML.xsd

  41. Confidence Measures attributes ::= tagType tagID [attributeName confidenceValue tagType ::= CDATA tagID ::= IDREF attributeName ::= CDATA confidenceValue ::= CDATA {confidenceValue ::= 0 < x < 1}

  42. Use of Confidence Measure The TWA flight <EVENT eid="e1" class="OCCURRENCE" tense="PAST" aspect="NONE"> crashlanded </EVENT> <MAKEINSTANCE eiid="ei1" eventID="e1"/> <TLINK eventInstanceID="ei1" signalID="s1" relatedToTime="t2" relType="BEFORE" magnitude="t1"/> on Easter Island <TIMEX3 tid="t1" type="DURATION" value="P2W"> two weeks </TIMEX3> <SIGNAL sid="s1"> ago </SIGNAL>. ... <TIMEX3 tid="t2" type="DATE" functionInDocument="CREATION_TIME" value="1999-12-20"> 12-20-1999 </TIMEX3>

  43. Domains and Data Sets • Document Collection (300): • ACE • DUC • PropBank (WSJ) • Query Corpus Collection: • Excite query logs • MITRE Corpus • TREC8/9/10 • Queries from TIMEBANK

  44. Corpus Analytics • Concordanced and indexed all training data • DUC subset • ACE subset • WSJ subset • Concordancing and indexing reference data • BNC • Brown Corpus • WSJ Corpus

  45. Graphical Annotation Tools • TimeML-Alembic: • Extensions to MITRE’s Alembic Workbench • Semi-Graphical Annotation Tool • Create links by ordering events and TIMEX3s

  46. Text Segmented Closure • System-prompted queries (a la Setzer): • Completes temporal ordering markup in a text • Performed on document segments: • Decreases the number of queries required to provide closure • Enrichments to Closure: • Persistence of states • Negative events

  47. Goals of Text Segmented Closure • Too many temporal relations in a large document. • The number of temporal relations is quadratic to the number of objects that are being linked temporally. An annotator may be prompted hundreds of times, especially for large documents where a lot of the relations are "unknown". • Some temporal relations are not interesting. • There does not seem to be a need to relate all time expressions to each other.

  48. Architecture for TSC 1. Perform initial closure on all links added by the annotator. 2. Alert the user to potential identity chains. This is the only occasion where a user may be asked to specify a non-local relation. 3. Create a sliding window of three sentences. Initially, the window will consist of sentences one through three. The sliding window implements the local context. The size of the sliding window can be parameterize. 4. Prompt the user to specify a relation type for two time objects that are not yet linked within the local context. If no temporal relation exists, the annotator may specify "unknown". 5. After each added relation, recompute the closure using the new fact. Do this till all time objects within the local context are related. 6. If all objects in the local context are related, move the window up one sentence. For example, if the previous local context was made up of sentences 3-5, then the next local context for the closure algorithm is sentences 4-6. Start prompting the user for the new context.

  49. Temporal Axioms • The axioms work with a normalized set of temporal relations (no axiom is needed for the relType "unknown"): PRE before, after, ibefore, iafter INC includes, is_included SIM simultaneous IDT identity • For PRE, normalization works as follows: Link<x,y,before> => Link<x,y,PRE> Link<x,y,ibefore> => Link<x,y,PRE> Link<x,y,after> => Link<y,x,PRE> Link<x,y,iafter> => Link<y,x,PRE>

  50. Precedence PRE1: [ x PRE y & y PRE z => x PRE z ] ----x---- ----y---- ----z---- PRE2: [ x PRE y & y SIM z => x PRE z ] PRE3: [ x PRE y & y IDT z => x PRE z ] ----x---- ----y---- ----z---- PRE4: [ x PRE y & x SIM z => z PRE y ] PRE5: [ x PRE y & x IDT z => z PRE y ] ----x---- ----y---- ----z---- PRE6: [ x PRE y & x INC z => z PRE y ] ----x---- ----y---- --z--

More Related