FUNNEL: Automatic Mining of Spatially Coevolving Epidemics

1. FUNNEL: Automatic Mining of Spatially Coevolving Epidemics Yasuko Matsubara, Yasushi Sakurai (Kumamoto University) Willem G. van Panhuis(University of Pittsburgh) Christos Faloutsos (CMU) Y. Matsubara et al.

2. Motivation Given: Large set of epidemiological data e.g., Measles cases in the U.S. Linear (Weekly) Y. Matsubara et al.

3. Motivation Given: Large set of epidemiological data e.g., Measles cases in the U.S. Linear Yearly periodicity (Weekly) Y. Matsubara et al.

4. Motivation Given: Large set of epidemiological data e.g., Measles cases in the U.S. Linear Yearly periodicity (Weekly) Vaccine effect Y. Matsubara et al.

5. Motivation Given: Large set of epidemiological data e.g., Measles cases in the U.S. Shocks, e.g., 1941 Linear Yearly periodicity (Weekly) Vaccine effect Y. Matsubara et al.

6. Motivation Given: Large set of epidemiological data e.g., Measles cases in the U.S. Goal: summarizeall the epidemic time-series, “fully-automatically” Linear Y. Matsubara et al.

7. Data description Project Tycho: infectious diseases in the U.S. 50states X 56diseases 1888 (> 125 years) Time (weekly) Y. Matsubara et al.

8. Data description Project Tycho: infectious diseases in the U.S. 50states X x 56diseases 1888 (> 125 years) Time (weekly) Element x : # of cases e.g., ‘measles’, ‘NY’, ‘April 1-7, 1931’, ‘4000’ Y. Matsubara et al.

9. Problem definition Given: Tensor X(disease x state x time) X Y. Matsubara et al.

10. Problem definition Given: Tensor X(disease x state x time) Find: Compact description of X, “automatically” X X E M FUNNEL Y. Matsubara et al. P1 P2 P3 P4 P5

11. Problem definition Seasonality Given: Tensor X(disease x state x time) Find: Compact description of X, “automatically” X X E M Discontinuities FUNNEL Y. Matsubara et al. P1 P2 P3 P4 P5

12. Problem definition Given: Tensor X(disease x state x time) Find: Compact description of X, “automatically” NO magic numbers ! Parameter-free! X X E M FUNNEL Y. Matsubara et al. P1 P2 P3 P4 P5

13. Roadmap • Motivation • Modeling power of FUNNEL • Overview – main ideas • Proposed model – idea #1 • Algorithm – idea #2 • Experiments • Discussion • Conclusions ✔ Y. Matsubara et al.

14. Modeling power of FUNNEL Questions about epidemics X Q1 Q3 Q4 Q2 Q5 Y. Matsubara et al.

15. Questions Q2 Q1 Q3 Q4 Q5 Q1 Are there any periodicities? If yes, when is the peak season? X Y. Matsubara et al.

16. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Radius: seasonality strength FUNNEL: Polar plot Angle: peak season SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 16

17. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 17

18. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? Q: Does Influenza have seasonality? If yes, when? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 18

19. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 19

20. Answers Q2 Q1 Q3 Q4 Q5 Seasonality P1 Influenza in Feb. Detected by FUNNEL (strong seasonality) Detected! SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 20

21. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 21

22. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? Q: How about measles ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 22

23. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 23

24. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Measles (children’s) in spring Detected! SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 24

25. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 25

26. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? Q: Which disease peaks in summer? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 26

27. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 27

28. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Detected! Lyme-disease (tick-borne) in summer SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 28

29. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 29

30. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? Q: Which disease has no periodicity? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 30

31. Answers Q2 Q1 Q3 Q4 Q5 P1 Seasonality Questions ? SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 31

32. Answers Q2 Q1 Q3 Q4 Q5 Seasonality P1 Detected! Gonorrhea (STD) no periodicity SIGKDD 2014 Y. Matsubara et al. Y. Matsubara et al. 32

33. Questions Q2 Q1 Q3 Q4 Q5 Q2 X Can we see any discontinuities? Y. Matsubara et al.

34. Answers Q2 Q1 Q3 Q4 Q5 P2 Disease reduction effect 1965: Detected by FUNNEL Measles Detected! 1963: Vaccine licensure Y. Matsubara et al.

35. Questions Q2 Q1 Q3 Q4 Q5 Q3 What’s the difference between measles in NY and in FL? Y. Matsubara et al.

36. Answers Q2 Q1 Q3 Q4 Q5 P3 area sensitivity FUNNEL’s guess of susceptibles (measles) Detected! CA NY, PA (more children) TX FL (fewer children) Y. Matsubara et al.

37. Questions Q2 Q1 Q3 Q4 Q5 Q4 Are there any external shock events, like wars? X Y. Matsubara et al.

38. Answers Q2 Q1 Q3 Q4 Q5 P4 external shock events Funnel can detect external shocks “fully-automatically” ! Detected! Detected by FUNNEL Scarlet fever World war II Y. Matsubara et al.

39. Questions Q2 Q1 Q3 Q4 Q5 Q5 How can we remove mistakes and incorrect values? X TYPO! Y. Matsubara et al.

40. Answers Q2 Q1 Q3 Q4 Q5 P5 mistakes It can also detect typos, “automatically” !! Detected! Mistake Typhoid fever cases Missing values Y. Matsubara et al.

41. Modeling power of FUNNEL Our model can capture 5 properties Seasonality Disease reductions Area sensitivity External events Mistakes P1 P2 P3 P4 P5 Y. Matsubara et al.

42. Roadmap • Motivation • Modeling power of FUNNEL • Overview – main ideas • Proposed model – idea #1 • Algorithm – idea #2 • Experiments • Discussion • Conclusions ✔ ✔ Y. Matsubara et al.

43. Problem definition Given: Tensor X(disease x state x time) Find: Compact description of X, “automatically” X X E M FUNNEL Y. Matsubara et al. P1 P2 P3 P4 P5

44. Two main ideas Idea #1: Grey-box model Idea #2: MDL for fitting X E M FUNNEL NO magic numbers ! (parameter-free) Y. Matsubara et al. P1 P2 P3 P4 P5

45. Two main ideas Idea #1: Grey-box model - domain knowledge X E M FUNNEL (SIRS+) : 6 parameters Shocks Vaccine Y. Matsubara et al. P1 P2 P3 P4 P5

46. Two main ideas Idea #2: Fitting with MDL -> parameter free! NO magic numbers Parameter-free! X E M FUNNEL Y. Matsubara et al. P1 P2 P3 P4 P5

47. Roadmap • Motivation • Modeling power of FUNNEL • Overview – main ideas • Proposed model – idea #1 • Algorithm – idea #2 • Experiments • Discussion • Conclusions ✔ ✔ ✔ Y. Matsubara et al.

48. Proposed model: FUNNEL X single epidemic states (a) FUNNEL-single (b) FUNNEL-full diseases Multi-evolving epidemics Time Y. Matsubara et al.

49. Proposed model: FUNNEL X single epidemic states (a) FUNNEL-single (b) FUNNEL-full diseases Multi-evolving epidemics Time Y. Matsubara et al.

50. FUNNEL – with a single epidemic Given: “single” epidemic sequence Find: nonlinear equation, model parameters e.g., measles in NY FUNNEL Y. Matsubara et al.