Lecture 7 Centrality ( cont )

1. Lecture 7 Centrality (cont) Slides modified from LadaAdamic and DragomirRadev

2. Eigenvalues and eigenvectors • Eigenvalues and eigenvectors have their origins in physics, in particular in problems where motion is involved, although their uses extend from solutions to stress and strain problems to differential equations and quantum mechanics. • Eigenvectors are vectors that point in directions where there is no rotation. Eigenvalues are the change in length of the eigenvector from the original length. • The basic equation in eigenvalue problems is: Slides from Fred K. Duennebier

3. Eigenvalues and eigenvectors • In words, this deceptively simple equation says that for the square matrix A, there is a vector x such that the product of Ax such that the result is a SCALAR, , that, when multiplied by x, results in the same product. The multiplication of vector x by a scalar constant is the same as stretching or shrinking the coordinates by a constant value. (E.01)

4. The vector x is called an eigenvectorand the scalar , is called an eigenvalue. • Do all matrices have real eigenvalues? • No, they must be square and the determinant of A- I must equal zero. This is easy to show: • This can only be true if det(A- I )=|A- I |=0 • Are eigenvectors unique? • No, if x is an eigenvector, then x is also an eigenvector and  is an eigenvalue. (E.02) (E.03) A(x)= Ax = lx = l (x) (E.04)

5. How do you calculate eigenvectors and eigenvalues? Expand equation (E.03): det(A- I )=|A- I |=0 for a 2x2 matrix: (E.05) For a 2-dimensional problem such as this, the equation above is a simple quadratic equation with two solutions for . In fact, there is generally one eigenvalue for each dimension, but some may be zero, and some complex.

6. The solution to E.05 is: (E.06) (E.07) This “characteristic equation” does not involve x, and the resulting values of  can be used to solve for x. Consider the following example: Eqn. E.07 doesn’t work here because a11a22-a12a12=0, so we use E.06:

7. We see that one solution to this equation is =0, and dividing both sides of the above equation by  yields =5. Thus we have our two eigenvalues, and the eigenvectors for the first eigenvalue, =0 are: These equations are multiples of x=-2y, so the smallest whole number values that fit are x=2, y=-1

8. For the other eigenvalue, =5: This example is rather special; A-1 does not exist, the two rows of A- I are dependent and thus one of the eigenvalues is zero. (Zero is a legitimate eigenvalue!) EXAMPLE: A more common case is A =[1.05 .05 ; .05 1] used in the strain exercise. Find the eigenvectors and eigenvalues for this A, and then calculate [V,D]=eig[A]. The procedure is: Compute the determinant of A- I Find the roots of the polynomial given by | A- I|=0 Solve the system of equations (A- I)x=0

9. What good are such things? Consider the matrix: What is A100 ? We can get A100 by multiplying matrices many many times: Or we could find the eigenvalues of A and obtain A100 very quickly using eigenvalues.

10. For now, I’ll just tell you that there are two eigenvectors for A: The eigenvectors are x1=[.6 ; .4] and x2=[1 ; -1], and the eigenvalues are 1=1 and 2=0.5. Note that if we multiply x1 by A, we get x1. If we multiply x1 by A again, we STILL get x1. Thus x1 doesn’t change as we mulitiply it by An.

11. What about x2? When we multiply A by x2, we get x2/2, and if we multiply x2 by A2, we get x2/4 . This number gets very small fast. Note that when A is squared the eigenvectors stay the same, but the eigenvalues are squared! Back to our original problem we note that for A100, the eigenvectors will be the same, the eigenvalues 1=1 and 2=(0.5)100, which is effectively zero. Each eigenvector is multiplied by its eigenvalue whenever A is applied,

12. Outline • Degree centrality • Centralization • Betweenness centrality • Closeness centrality • Eigenvector centrality • Bonacich power centrality • Katz centrality • PageRank • Hubs and Authorities • Applications to Information Retrieval • LexRank

13. Eigenvector Centrality • An extension of degree centrality • Centrality increases with number of neighbors • Not all neighbors are equal • Having connection to more central nodes increases importance • Eigenvector centrality gives each vertex a score proportional to the sum of scores of its neighbors where and vi are eigenvectors

14. Eigenvector Centrality • As , we get • Hence, A x = k1x • where • Eigenvector centrality of a vertex is large either • it has many neighbors and/or • it has important neighbors

15. Eigenvector Centrality • Can be calculated for directed graphs as well • We need to decide between incoming or outgoing edges • A has no incoming edges, hence a centrality of 0 • B has only an incoming edge from A • hence its centrality is also 0 • Only vertices that are in a strongly connected component of two or more vertices or the out-component of such a component have non-zero centrality A B E C D

16. Katz centrality • Give each vertex a small amount of centrality • regardless of its position in the network or the centrality of its neighbors • Hence, • where ais a scaling vector, which is set to normalize the score (for the expression to converge a ≤ 1/k1) breflects the extent to which you weight the centrality of people ego is tied to Iis the identity matrix (1s down the diagonal) 1is a matrix of all ones

17. Katz Centrality: b • The magnitude of b reflects the radius of power • Small values of b weight local structure • Larger values weight global structure • If b > 0, ego has higher centrality when tied to people who are central • If b < 0, then ego has higher centrality when tied to people who are not central • With b = 0, you get degree centrality

18. Katz Centrality: examples b=.25 b=-.25 Why does the middle node have lower centrality than its neighbors when b is negative?

19. PageRank: bringing order to the web • It’s in the links: • links to URLs can be interpreted as endorsements or recommendations • the more links a URL receives, the more likely it is to be a good/entertaining/provocative/authoritative/interesting information source • but not all link sources are created equal • a link from a respected information source • a link from a page created by a spammer an important page, e.g. slashdot Many webpages scattered across the web if a web page isslashdotted, it gains attention

20. PageRank • An issue in Katz centrality measure is that a high centrality vertex pointing to large number of vertices gives all high centrality • Yahoo directory • This can be fixed by dividing the centrality with the out-degree of a vertex where Dii=max(kiout, 1)

21. Ranking pages by tracking a drunk • A random walker following edges in a network for a very long time will spend a proportion of time at each nodewhich can be used as a measure ofimportance

22. Trapping a drunk • Problem with pure random walk metric: • Drunk can be “trapped” and end up going in circles

23. Ingenuity of the PageRank algorithm • Allow drunk to teleport with some probability • e.g. random websurfer follows links for a while, but with some probability teleports to a “random” page • bookmarked page or uses a search engine to start anew

24. PageRank algorithm where p1,p2,...,pN are the pages under consideration, M(pi) is the set of pages that link to pi, L(pj) is the number of outbound links on page pj, and N is the total number of pages. d is the random jumping probability (d = 0.85 for google)

25. Exercise: PageRank • What happens to the relative PageRank scores of the nodes as you increase the teleportation probability? • Can you construct a network such that a node with low indegree has the highest PageRank? GUESS PageRank demo http://projects.si.umich.edu/netlearn/GUESS/pagerank.html

26. t=0 1 6 8 2 7 t=1 5 3 4 example: probable location of random walker after 1 step 20% teleportation probability

27. t=0 1 6 8 2 7 t=1 5 3 4 t=10 example: location probability after 10 steps

28. Matrix-based Centrality measures without constant term with constant term PageRank Degree centrality Divide by out-degree No division Katz centrality Eigenvector centrality

29. Hubs and Authorities • In directed networks, vertices that point to important resources should also get a high centrality • e.g. review articles, web indexes • recursive definition: • authorities are nodes that are linked to by good hubs • hubs are nodes that links to good authorities

30. Hyperlink-Induced Topic Search • HITS algorithm • start with a set of pages matching a query • expand the set by following forward and back links • take transition matrix E, where the i,jthentry Eij =1/ni • where i links to j, andni is the number of links from i • then one can compute the authority scores a, and hub scores hthrough an iterative approach:

32. Outline • Degree centrality • Centralization • Betweenness centrality • Closeness centrality • Eigenvector centrality • Bonacich power centrality • Katz centrality • PageRank • Hubs and Authorities • Applications to Information Retrieval • LexRank

33. Applications to Information Retrieval • Can we use the notion of centrality to pick the best summary sentence? • Can we use the subgraph of query results to infer something about the query? • Can we use a graph of word translations to expand dictionaries? disambiguate word meanings? • How might one use the HITS algorithm for document summarization? • Consider a bipartite graph of sentences and words

34. Centrality in summarization • Extractive summarization • pick k sentences that are most representative of a collection of n sentences • Motivation: • capture the most central words in a document or cluster • Centroid score [Radev & al. 2000, 2004a] • Alternative methods for computing centrality?

35. Sample multidocument cluster (DUC cluster d1003t) 1 (d1s1) Iraqi Vice President Taha Yassin Ramadan announced today, Sunday, that Iraq refuses to back down from its decision to stop cooperating with disarmament inspectors before its demands are met. 2 (d2s1) Iraqi Vice president Taha Yassin Ramadan announced today, Thursday, that Iraq rejects cooperating with the United Nations except on the issue of lifting the blockade imposed upon it since the year 1990. 3 (d2s2) Ramadan told reporters in Baghdad that "Iraq cannot deal positively with whoever represents the Security Council unless there was a clear stance on the issue of lifting the blockade off of it. 4 (d2s3) Baghdad had decided late last October to completely cease cooperating with the inspectors of the United Nations Special Commission (UNSCOM), in charge of disarming Iraq's weapons, and whose work became very limited since the fifth of August, and announced it will not resume its cooperation with the Commission even if it were subjected to a military operation. 5 (d3s1) The Russian Foreign Minister, Igor Ivanov, warned today, Wednesday against using force against Iraq, which will destroy, according to him, seven years of difficult diplomatic work and will complicate the regional situation in the area. 6 (d3s2) Ivanov contended that carrying out air strikes against Iraq, who refuses to cooperate with the United Nations inspectors, ``will end the tremendous work achieved by the international group during the past seven years and will complicate the situation in the region.'' 7 (d3s3) Nevertheless, Ivanov stressed that Baghdad must resume working with the Special Commission in charge of disarming the Iraqi weapons of mass destruction (UNSCOM). 8 (d4s1) The Special Representative of the United Nations Secretary-General in Baghdad, Prakash Shah, announced today, Wednesday, after meeting with the Iraqi Deputy Prime Minister Tariq Aziz, that Iraq refuses to back down from its decision to cut off cooperation with the disarmament inspectors. 9 (d5s1) British Prime Minister Tony Blair said today, Sunday, that the crisis between the international community and Iraq ``did not end'' and that Britain is still ``ready, prepared, and able to strike Iraq.'' 10 (d5s2) In a gathering with the press held at the Prime Minister's office, Blair contended that the crisis with Iraq ``will not end until Iraq has absolutely and unconditionally respected its commitments'' towards the United Nations. 11 (d5s3) A spokesman for Tony Blair had indicated that the British Prime Minister gave permission to British Air Force Tornado planes stationed in Kuwait to join the aerial bombardment against Iraq.

36. Cosine between sentences • Let s1 and s2 be two sentences. • Let x and y be their representations in an n-dimensional vector space • The cosine between is then computed based on the inner product of the two. • The cosine ranges from 0 to 1.

37. LexRank (Cosine centrality)

38. d3s3 d2s3 d3s2 d3s1 d1s1 d4s1 d5s1 d2s1 d5s2 d5s3 d2s2 Lexical centrality (t=0.3)

39. d3s3 d2s3 d3s2 d3s1 d1s1 d4s1 d5s1 d2s1 d5s2 d5s3 d2s2 Lexical centrality (t=0.2)

40. d3s2 d3s3 d2s3 d3s2 d4s1 d3s1 d2s1 d1s1 d4s1 d5s1 d2s1 d5s2 d5s3 d2s2 Lexical centrality (t=0.1) Sentences vote for the most central sentence…

41. LexRank • T1…Tn are pages that link to A, • c(Ti) is the outdegree of pageTi, and • N is the total number of pages. • d is the “damping factor”, or the probability that we “jump” to a far-away node during the random walk. • It accounts for disconnected components or periodic graphs. • When d = 0, we have a strict uniform distribution.When d = 1, the method is not guaranteed to converge to a unique solution. • Typical value for d is between [0.1,0.2] (Brin and Page, 1998). Güneş Erkan and Dragomir R. Radev, LexRank: Graph-based Lexical Centrality as Salience in Text Summarization

42. lab: Lexrank demo • how does the summary change as you: • increase the cosine similarity threshold for an edge • how similar two sentences have to be? • increase the salience threshold (minimum degree of a node) http://tangra.si.umich.edu/clair/lexrank/

43. Content similarity distributions for web pages (DMOZ) and scientific articles (PNAS) Menczer, Filippo (2004) The evolution of document networks.

44. what is that good for? • How could you take advantage of the fact that pages that are similar in content tend to link to one another?

45. What can networks of query results tell us about the query? • If query results are highly interlinked, is this a narrow or broad query? • How could you use query connection graphs to predict whether a query will be reformulated? Jure Leskovec, Susan Dumais: Web Projections: Learning from Contextual Subgraphs of the Web

46. How can bipartite citation graphs be used to find related articles? • co-citation: both A and B are cited by many other papers (C, D, E …) B A C D E • bibliographic coupling: both A and B are cite many of the same articles (F,G,H …) F G H B A

47. which of these pairs is more proximate • according to cycle free effective conductance: • the probability that you reach the other node before cycling back on yourself, while doing a random walk….

48. Proximity as cycle free effective conductance • Measuring and Extracting Proximity in Networks by Yehuda Koren, Stephen C. North, Chris Volinsky, KDD 2006 • demo: http://public.research.att.com/~volinsky/cgi-bin/prox/prox.pl

49. Using network algorithms (specifically proximity) to improve movie recommendations can pay off Source: undetermined

50. final IR application: machine translation • not all pairwise translations are available • e.g. between rare languages • in some applications, e.g. image search, a word may have multiple meanings • “spring” is an example in english But in other languages, the word may be unambiguous. • automated translation could be the key or or or