Kira Radinsky, Sagie Davidovich , Shaul Markovitch Technion - Israel Institute of Technology

1. Learning Causalityfor News Events Prediction • Kira Radinsky, SagieDavidovich, ShaulMarkovitch • Technion - Israel Institute of Technology

2. What is Prediction? “A description of what one thinks will take place in the future, based on previous knowledge.” [Online Dictionary] “…a rigorous, often quantitative, statement, forecasting what will happen under specific conditions.“ [Wikipedia]

3. Why is News Event Prediction Important? Strategic Intelligence Strategic planning Strategic planning Financial investments

4. Outline • Motivation • Problem definition • Solution • Representation • Algorithm • Evaluation

5. Problem Definition: Events Prediction • Ev is a set of events • T is discrete representation of time

6. Outline • Motivation • Problem definition • Solution • Representation • Algorithm • Evaluation

7. Causality Mining Process: Overview

8. Outline • Motivation • Problem definition • Solution • Representation • Algorithm • Evaluation

9. Modeling an Event • Comparison between events (Canonical) • (Lexicon & Syntax) Language & wording independent • (Semantic) Non ambiguous • Generalization / abstraction • Reasoning • Many philosophies • Property Exemplification of Events theory (Kim 1993) • Conceptual Dependency theory (Schank 1972)

10. Event & Causality Representation Time • Event Representation 5 • Causality Representation kill Quantifier Troops Action Theme Attribute Event2 Afghan 1/2/1987 11:15AM +(3h) Time-frame “5 Afghan troops were killed” Army bombs Caused Weapon warehouse US US Army Action Theme Actor Event1 1/2/1987 11:00AM +(2h) Location Time-frame Kabul “US Army bombs a weapon warehouse in Kabul with missiles” Instrument Missiles

11. Outline • Motivation • Problem definition • Solution • Representation • Algorithm • Causality Mining Process • Evaluation

12. Machine Learning Problem Definition Goal function: Learning algorithm receives a set of examples and produces a hypothesis which is goodapproximation of

13. Algorithm Outline Learning Phase • Generalize events • Causality prediction rule generation Prediction Phase • Finding similar generalized event • Application of causality prediction rule

14. Algorithm Outline Learning Phase • Generalize events • How do we generalize objects? • How do we generalize actions? • How do we generalize an event? • Causality prediction rule generation

15. Generalizing Objects the Russian Federation Eastern Europe Name official English Is in group RUS Russian Federation ISO3 Code China Land border FAOSTAT code 185 DBPedia ID Is predecessor of Russia Is successor of Currency Name UN Code USSR Rouble (Rub) 643

16. Ontology – Linked data

17. Generalizing Actions Levin classes (Levin 1993)– 270 classes

18. Generalizing Events:Putting it all together “NATO strikes an army base in Baghdad” US NATO Actor Present Event 1/2/1987 11:00AM +(2h) Location Time-frame Baghdad Generalization rule Theme Army base Action Actor: [state of Nato] Property: [Hit1.1] Theme: [Military facility] Location: [Arab City] Country Army strikes Military facility City Similar verb bombs US Army Weapon warehouse Action Theme Actor Past Event Location 1/2/1987 11:00AM +(2h) Time-frame Kabul “US Army bombs a weapon warehouse in Kabul with missiles” Instrument rdf:type Missiles

19. Generalizing Events: HAC algorithm

20. Generalizing Events:Event distance metric “NATO strikes an army base in Baghdad” US NATO Actor Present Event 1/2/1987 11:00AM +(2h) Location Time-frame Baghdad Theme Army base Action Country Army strikes Military facility City Similar verb bombs US Army Weapon warehouse Theme Action Actor Past Event Location 1/2/1987 11:00AM +(2h) Time-frame Kabul “US Army bombs a weapon warehouse in Kabul with missiles” Instrument rdf:type Missiles

21. Algorithm Outline • Learning Phase • Generalize events • Causality prediction rule generation

22. Prediction Rule Generation Time 5 kill Quantifier Troops Action Theme Attribute “5 Afghan troops were killed” Effect Event Afghan 1/2/1987 11:15AM +(3h) Time-frame Nationality Afghanistan Army Country bombs Caused Type Type Weapon warehouse Country US US Army Action Theme Actor Cause Event 1/2/1987 11:00AM +(2h) Time-frame Kabul Location EffectThemeAttribute=CauseLocationCountryNationality EffectAction=kill EffectTheme=Troops “US Army bombs a weapon warehouse in Kabul with missiles” Instrument Missiles

23. Algorithm Outline Prediction Phase • Finding similar generalized event • Application of causality prediction rule

24. Finding Similar Generalized Event 0.2 “Baghdad bombing” 0.7 0.3 0.8 0.65 0.2 0.1 0.75

25. Prediction Rule Application Time kill Troops Action Theme Attribute Predicted Effect Event T1 + ∆ Time-frame Nationality bomb Caused Theme1 Country Actor1 Action Theme Actor Input Event T1 Time-frame Location1 Location EffectThemeAttribute=CauseLocationCountryNationality EffectAction=kill EffectTheme=Troops Instrument Instrument1

26. Outline • Motivation • Problem definition • Solution • Representation • Algorithm • Evaluation

27. Prediction Evaluation Human Group 1: • Mark events E that can cause other events. Human Group 2: • Given: Random sample of events from E , predictions and time of events • Search the web and give estimation on the prediction accuracy

28. Prediction Accuracy Results

29. Causality Evaluation Human Group 1: • Mark events E for test for the second two control groups and the algorithm. Human Group 2: • Given: Random sample of events from E. • State what you think would happen following this event. Human Group 3: • Given: algorithm predictions + human (2nd group) predictions • Evaluate the quality of the predictions

30. Causality Results • The results are statistically significant

32. Accuracy of Extraction Extraction Evaluation Entity Ontology Matching

33. Related work Causality Information Extraction Goal: Extract causality relations from a text Techniques: Usage of handcrafted domain-specific patterns [Kaplan and Berry-Rogghe, 1991] Usage of handcrafted linguistic patterns[Garcia 1997],[Khoo, Chan, &Niu 2000], [Girju &Moldovan 2002] Semi-Supervised pattern learning approaches, based on text features [Blanco, Castell, &Moldovan 2008], [Sil & Huang & Yates 2010] Supervised pattern learning approaches based on text features [Riloff 1996],[Riloff & Jones 1999], [Agichtein & Gravano, 2000; Lin & Pantel, 2001]

34. Related work Temporal Information Extraction Goal: Predicting the temporal order of events or time expressions described in text Technique:learn classifiers that predict a temporal order of a pair of events based on a predefined features of the pair. [Ling & Weld, 2010; Mani, Schiffman, & Zhang, 2003; Lapata & Lascarides,2006; Chambers, Wang, & Jurafsky, 2007; Tatu & Srikanth, 2008; Yoshikawa, Riedel, Asahara, & Matsumoto, 2009]

35. Future work • Going beyond human tagged examples • Incorporating time into the equation • When will correlation mean causality? • Using other sources than news • Incorporating real time data (Twitter, Facebook) • Incorporating numerical data (Stocks, Weather, Forex) • Can we predict general facts? • Can a machine predict better than an expert?

36. Summary • Canonical event representation • Machine learning algorithm for events prediction • Leveraging world knowledge for generalization • Using text as human tagged examples • Causality mining from text • Contribution to machine common-sense understanding “The best way to predict the future is to invent it” [Alan Kay]