1 / 129

Finding patterns in large, real networks

Finding patterns in large, real networks. Christos Faloutsos CMU. Thanks to. Deepayan Chakrabarti (CMU/Yahoo) Michalis Faloutsos (UCR) Spiros Papadimitriou (CMU/IBM) George Siganos (UCR). Introduction. Protein Interactions [genomebiology.com]. Internet Map [lumeta.com].

travis
Télécharger la présentation

Finding patterns in large, real networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Finding patterns in large, real networks Christos Faloutsos CMU

  2. Thanks to • Deepayan Chakrabarti (CMU/Yahoo) • Michalis Faloutsos (UCR) • Spiros Papadimitriou (CMU/IBM) • George Siganos (UCR) C. Faloutsos

  3. Introduction Protein Interactions [genomebiology.com] Internet Map [lumeta.com] Food Web [Martinez ’91] Graphs are everywhere! Friendship Network [Moody ’01] C. Faloutsos

  4. Outline • Laws • static • time-evolution laws • why so many power laws? • tools/results • future steps C. Faloutsos

  5. Motivating questions • Q1: What do real graphs look like? • Q2: How to generate realistic graphs? C. Faloutsos

  6. Virus propagation • Q3: How do viruses/rumors propagate? • Q4: Who is the best person/computer to immunize against a virus? C. Faloutsos

  7. Graph clustering & mining • Q5: which edges/nodes are ‘abnormal’? • Q6: split a graph in k ‘natural’ communities - but how to determine k? C. Faloutsos

  8. Outline • Laws • static • time-evolution laws • why so many power laws? • tools/results • future steps C. Faloutsos

  9. Topology Q1: How does the Internet look like? Any rules? (Looks random – right?) C. Faloutsos

  10. Laws – degree distributions • Q: avg degree is ~2 - what is the most probable degree? count ?? degree 2 C. Faloutsos

  11. WRONG ! count ?? count degree 2 2 Laws – degree distributions • Q: avg degree is ~2 - what is the most probable degree? degree C. Faloutsos

  12. I.Power-law: outdegree O The plot is linear in log-log scale [FFF’99] freq = degree (-2.15) Frequency Exponent = slope O = -2.15 -2.15 Nov’97 Outdegree C. Faloutsos

  13. -0.82 att.com log(degree) ibm.com log(rank) II.Power-law: rank R • The plot is a line in log-log scale April’98 R Rank: nodes in decreasing degree order C. Faloutsos

  14. III.Power-law: eigen E Eigenvalue • Eigenvalues in decreasing order (first 20) • [Mihail+, 02]: R = 2 * E Exponent = slope E = -0.48 Dec’98 Rank of decreasing eigenvalue C. Faloutsos

  15. IV. The Node Neighborhood • N(h) = # of pairs of nodes within h hops C. Faloutsos

  16. IV. The Node Neighborhood • Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops? • Potential answer: 1 hop -> 3 neighbors 2 hops -> 3 * 3 … h hops -> 3h C. Faloutsos

  17. IV. The Node Neighborhood • Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops? • Potential answer: 1 hop -> 3 neighbors 2 hops -> 3 * 3 … h hops -> 3h WRONG! WE HAVE DUPLICATES! C. Faloutsos

  18. IV. The Node Neighborhood • Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops? • Potential answer: 1 hop -> 3 neighbors 2 hops -> 3 * 3 … h hops -> 3h WRONG x 2! ‘avg’ degree: meaningless! C. Faloutsos

  19. IV. Power-law: hopplot H H = 2.83 # of Pairs H = 4.86 # of Pairs Pairs of nodes as a function of hops N(h)= hH Hops Router level ’95 Dec 98 Hops C. Faloutsos

  20. ... Observation • Q: Intuition behind ‘hop exponent’? • A: ‘intrinsic=fractal dimensionality’ of the network N(h) ~ h1 N(h) ~ h2 C. Faloutsos

  21. Hop plots • More on fractal/intrinsic dimensionalities: very soon C. Faloutsos

  22. Any other ‘laws’? Yes! C. Faloutsos

  23. Any other ‘laws’? Yes! • Small diameter • six degrees of separation / ‘Kevin Bacon’ • small worlds [Watts and Strogatz] C. Faloutsos

  24. Any other ‘laws’? • Bow-tie, for the web [Kumar+ ‘99] • IN, SCC, OUT, ‘tendrils’ • disconnected components C. Faloutsos

  25. Any other ‘laws’? • power-laws in communities (bi-partite cores) [Kumar+, ‘99] Log(count) n:1 n:3 n:2 2:3 core (m:n core) Log(m) C. Faloutsos

  26. More Power laws • Also hold for other web graphs [Barabasi+, ‘99], [Kumar+, ‘99] • citation graphs (see later) • and many more C. Faloutsos

  27. Outline • Laws • static • time-evolution laws • why so many power laws? • tools/results • future steps C. Faloutsos

  28. How do graphs evolve? C. Faloutsos

  29. Time Evolution: rank R Domain level #days since Nov. ‘97 The rank exponent has not changed! [Siganos+, ‘03] C. Faloutsos

  30. How do graphs evolve? • degree-exponent seems constant - anything else? C. Faloutsos

  31. Evolution of diameter? • Prior analysis, on power-law-like graphs, hints that diameter ~ O(log(N)) or diameter ~ O( log(log(N))) • i.e.., slowly increasing with network size • Q: What is happening, in reality? C. Faloutsos

  32. Evolution of diameter? • Prior analysis, on power-law-like graphs, hints that diameter ~ O(log(N)) or diameter ~ O( log(log(N))) • i.e.., slowly increasing with network size • Q: What is happening, in reality? • A: It shrinks(!!), towards a constant value C. Faloutsos

  33. Shrinking diameter [Leskovec+05a] • Citations among physics papers • 11yrs; @ 2003: • 29,555 papers • 352,807 citations • For each month M, create a graph of all citations up to month M C. Faloutsos

  34. Shrinking diameter • Authors & publications • 1992 • 318 nodes • 272 edges • 2002 • 60,000 nodes • 20,000 authors • 38,000 papers • 133,000 edges C. Faloutsos

  35. Shrinking diameter • Patents & citations • 1975 • 334,000 nodes • 676,000 edges • 1999 • 2.9 million nodes • 16.5 million edges • Each year is a datapoint C. Faloutsos

  36. Shrinking diameter • Autonomous systems • 1997 • 3,000 nodes • 10,000 edges • 2000 • 6,000 nodes • 26,000 edges • One graph per day C. Faloutsos

  37. Temporal evolution of graphs • N(t) nodes; E(t) edges at time t • suppose that N(t+1) = 2 * N(t) • Q: what is your guess for E(t+1) =? 2 * E(t) C. Faloutsos

  38. Temporal evolution of graphs • N(t) nodes; E(t) edges at time t • suppose that N(t+1) = 2 * N(t) • Q: what is your guess for E(t+1) =? 2 * E(t) • A: over-doubled! x C. Faloutsos

  39. Temporal evolution of graphs • A: over-doubled - but obeying: E(t) ~ N(t)a for all t where 1<a<2 C. Faloutsos

  40. Temporal evolution of graphs • A: over-doubled - but obeying: E(t) ~ N(t)a for all t • Identically: log(E(t)) / log(N(t)) = constant for all t C. Faloutsos

  41. E(t) N(t) Densification Power Law ArXiv: Physics papers and their citations 1.69 C. Faloutsos

  42. E(t) N(t) Densification Power Law ArXiv: Physics papers and their citations 1 1.69 ‘tree’ C. Faloutsos

  43. E(t) N(t) Densification Power Law ArXiv: Physics papers and their citations ‘clique’ 2 1.69 C. Faloutsos

  44. E(t) N(t) Densification Power Law U.S. Patents, citing each other 1.66 C. Faloutsos

  45. E(t) N(t) Densification Power Law Autonomous Systems 1.18 C. Faloutsos

  46. E(t) N(t) Densification Power Law ArXiv: authors & papers 1.15 C. Faloutsos

  47. Summary of ‘laws’ • Power laws for degree distributions • …………... for eigenvalues, bi-partite cores • Small & shrinking diameter (‘6 degrees’) • ‘Bow-tie’ for web; ‘jelly-fish’ for internet • ``Densification Power Law’’, over time C. Faloutsos

  48. Outline • Laws • static • time-evolution laws • why so many power laws? • other power laws • fractals & self-similarity • case study: Kronecker graphs • tools/results • future steps C. Faloutsos

  49. Power laws • Many more power laws, in diverse settings: C. Faloutsos

  50. A famous power law: Zipf’s law log(freq) “a” • Bible - rank vs frequency (log-log) “the” log(rank) C. Faloutsos

More Related