1 / 39

Retrieval Models and Ranking Systems

Retrieval Models and Ranking Systems. CSC 575 Intelligent Information Retrieval. Retrieval Models. Model is an idealization or abstraction of an actual process in this case, process is matching of documents with queries, i.e., retrieval

keola
Télécharger la présentation

Retrieval Models and Ranking Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Retrieval Models and Ranking Systems CSC 575 Intelligent Information Retrieval

  2. Retrieval Models • Model is an idealization or abstraction of an actual process • in this case, process is matching of documents with queries, i.e., retrieval • Mathematical models are used to study the properties of the process, draw conclusions, make predictions • Conclusions derived from a model depend on whether the model is a good approximation to the actual situation • Retrieval models can describe the computational process • e.g. how documents are ranked • note that inverted file is an implementation not a model • Retrieval variables: queries, documents, terms, relevance judgements, users, information needs • Retrieval models have an explicit or implicit definition of relevance Intelligent Information Retrieval

  3. Lexical analysis and stop words Information need Collections Pre-process text input Parse Index Query How is the index constructed? Rank How is the matching and scoring done? Result Sets

  4. Retrieval Models • Customary to distinguish between exact-match and best-match retrieval • Exact-match • query specifies precise retrieval criteria every document either matches or fails to match query • result is a set of documents • Best-match • query describes good or “best” matching document • result is ranked list of documents • result may include estimate of quality • Best-match models: better retrieval effectiveness • good documents appear at top of ranking • but efficiency is better in exact match (e.g., Boolean) Intelligent Information Retrieval

  5. Ranking Algorithms • Assign weights to the terms in the query • Assign weights to the terms in the documents • Compare the weighted query terms to the weighted document terms • Boolean matching (exact match) • simple (coordinate level) matching • cosine similarity • other similarity measures (Dice, Jaccard, overlap, etc.) • extended Boolean models • probabilistic models • Rank order the results • pure Boolean has no ordering Intelligent Information Retrieval

  6. Boolean Retrieval • Boolean retrieval most common exact-match model • queries are logic expressions with document features as operands • retrieved documents are generally not ranked • query formulation difficult for novice users • “Pure” Boolean operators: AND, OR, NOT • Most systems have proximity operators • Most systems support simple regular expressions as search terms to match spelling variants Intelligent Information Retrieval

  7. B A Boolean Logic • AND and OR in a Boolean query represent intersection and union of the corresponding documents sets, respectively • NOT represents the complement of the corresponding set Intelligent Information Retrieval

  8. Boolean Queries • Boolean queries are Boolean combination of terms • Cat • Cat OR Dog • Cat AND Dog • (Cat AND Dog) OR Collar • (Cat AND Dog) OR (Collar AND Leash) • (Cat OR Dog) AND (Collar OR Leash) • (Cat OR Dog) AND (Collar OR Leash) • Each of the following combinations works: Intelligent Information Retrieval

  9. Boolean Matching t1 t2 m1= t1t2t3 D9 D2 D1 m2= t1 t2t3 m3 = t1 t2t3 m5 m3 m6 m4 = t1t2t3 D11 D4 D5 m5 = t1t2t3 D3 m1 m6 = t1t2t3 D6 m2 m4 m7 = t1t2t3 D10 m8= t1t2t3 m7 m8 D8 D7 t3 Hit list for the query t1 AND t2  {D1, D3, D5, D9, D10, D11} ∩{D1, D2, D4, D5, D6} = {D1, D5} Intelligent Information Retrieval

  10. Psuedo-Boolean Queries • A new notation, from web search • +cat dog +collar leash • Does not mean the same thing! • Need a way to group combinations • Phrases: • “stray cat” AND “frayed collar” • +“stray cat” + “frayed collar” Intelligent Information Retrieval

  11. Faceted Boolean Query • Strategy: break query into facets • conjunction of disjunctions (conjunctive normal form) a1 OR a2 OR a3 b1 OR b2 c1 OR c2 OR c3 OR c4 • each facet expresses a topic or concept “rain forest” OR jungle OR amazon medicine OR remedy OR cure research OR development AND AND Intelligent Information Retrieval

  12. Faceted Boolean Query • Query still fails if one facet missing • Alternative: a form of Coordination level ranking • Order results in terms of how many facets (disjuncts) are satisfied • Also called Quorum ranking • Problem: Facets still undifferentiated • Alternative: assign weights to facets Intelligent Information Retrieval

  13. Boolean Model • Advantages • simple queries are easy to understand • relatively easy to implement • structured queries • queries can be automatically translated into CNF or DNF • Disadvantages • difficult to specify what is wanted • too much returned, or too little (acceptable precision generally means unacceptable recall) • ordering not well determined • query formulation difficult for novice users • Dominant language in commercial systems until the WWW Intelligent Information Retrieval

  14. Vector Space Model(revisited) • Documents are represented as “bags of words” • Represented as vectors when used computationally • A vector is an array of floating point (or binary in case of bit maps) • Has direction and magnitude • Each vector has a place for every term in collection (most are sparse) Document Ids a document vector nova galaxy heat actor film role A 1.0 0.5 0.3 B 0.5 1.0 C 1.0 0.8 0.7 D 0.9 1.0 0.5 E 1.0 1.0 F 0.7 G 0.5 0.7 0.9 H 0.6 1.0 0.3 0.2 I 0.7 0.5 0.3 Intelligent Information Retrieval

  15. Documents & Query in n-dimensional Space • Documents are represented as vectors in term space • Terms are usually stems • Documents represented by binary vectors of terms • Queries represented the same as documents • Query and Document weights are based on length and direction of their vector • A vector distance measure between the query and documents is used to rank retrieved documents Intelligent Information Retrieval

  16. The Notion of “Similarity” in IR • The notion of similarity is central to many aspects of information retrieval and filtering: • measuring similarity of the query to documents is the primary factor in determining what is returned (and how they are ranked) • similarity measures can also be used in clustering documents (I.e., grouping together documents with similar content) • the same similarity measures can also be used to group together related terms (based on their occurrence patterns across documents in the collection) Intelligent Information Retrieval

  17. Vector-Based Similarity Measures • Simple Matching and Cosine Similarity • Simple matching = dot product of two vectors • Cosine Similarity = normalized dot product • the norm of a vector X is: • the cosine similarity of vectors X and Y is: In other words, divide the dot product by the norms of the two vectors Intelligent Information Retrieval

  18. Vector-Based Similarity Measures • Why divide by the norm? • Example: • X = <2, 0, 3, 2, 1, 4> • ||X|| = SQRT(4+0+9+4+1+16) = 5.83 • X* = X / ||X|| = <0.343, 0, 0.514, 0.343, 0.171, 0.686> • Now, note that ||X*|| = 1 • So, dividing a vector by its norm, turns it into a unit-length vector • Cosine similarity measures the angle between two unit length vectors (i.e., the magnitude of the vectors are ignored). Intelligent Information Retrieval

  19. Computing a similarity score2D Example Intelligent Information Retrieval

  20. Computing Similarity Scores 1.0 0.8 0.6 0.4 0.2 0.2 0.4 0.6 0.8 1.0 Intelligent Information Retrieval

  21. Other Vector Space Similarity Measures Simple Matching: Cosine Coefficient: Dice’s Coefficient: Jaccard’s Coefficient: Intelligent Information Retrieval

  22. Vector Space Similarity Measures • Again consider the following two document and the query vectors: D1 = (0.8, 0.3) D2 = (0.2, 0.7) Q = (0.4, 0.8) • Computing similarity using Jaccard’s Coefficient: • Computing similarity using Dice’s Coefficient: sim(Q, D2) = 0.96 sim(Q, D1) = 0.73 Intelligent Information Retrieval

  23. Vector Space Similarity MeasuresExample Intelligent Information Retrieval

  24. Vector Space Similarity MeasuresExample Intelligent Information Retrieval

  25. Probabilistic Models • Attempts to be more theoretically sound than the vector space model • try to predict the probability of a document’s being relevant, given the query • there are many variations • usually more complicated to compute than v.s. • usually many approximations are required • Relevance information is required from a random sample of documents and queries (training examples) • Works about the same (sometimes better) than vector space approaches Intelligent Information Retrieval

  26. Basic Probabilistic Retrieval • Retrieval is modeled as a classification process • Two classes for each query: the relevant and non-relevant documents (with respect to a given query) • could easily be extended to three classes (i.e. add a don’t care) • Given a particular document D, calculate the probability of belonging to the relevant class • retrieve if greater than probability of belonging to non-relevant class • i.e. retrieve if P(R|D) > P(NR|D) • Equivalently, rank by a discriminant value (also called likelihood ratio) P(R|D) / P(NR|D) • Different ways of estimating these probabilities lead to different models Intelligent Information Retrieval

  27. Basic Probabilistic Retrieval • A given query divides the document collection into two sets: relevant and non-relevant • If a document set D has been selected in response to a query, retrieve the document if dis(D) > 1 where dis(D) = P(R|D) / P(NR|D) • is the discriminant of D • This criteria can be modified by weighting the two probabilities Relevant Documents P(R|D) P(NR|D) Non-Relevant Documents Document Intelligent Information Retrieval

  28. Estimating Probabilities • Bayes’ Rule can be used to “invert” conditional probabilities: • Applying that to discriminant function: • Note that P(R) is the probability that a random document is relevant to the query, and P(NR) = 1 - P(R) P(R) = n / N and P(NR) = 1 - P(R) = (N - n) / N where n = number of relevant documents, and N = total number of documents in the collection Intelligent Information Retrieval

  29. Estimating Probabilities • Now we need to estimate P(D|R) and P(D|NR) • If we assume that a document is represented by terms t1, . . ., tn, and that these terms are statistically independent, then • and similarly we can compute P(D|NR) • Note that P(ti|R) is the probability that a term ti occurs in a relevant document, and it can be estimated based on previously available sample (e.g., through relevance feedback) • So, based on the probability of the distribution of terms in relevant and non-relevant documents we can estimate whether the document should be retrieved (i.e, if dis(D) > 1) • Note that documents that are retrieved can be ranked based on the value of the discriminant Intelligent Information Retrieval

  30. Probabilistic Retrieval - Example Since the discriminant is less than one, document D should not be retrieved Intelligent Information Retrieval

  31. Probabilistic Retrieval (cont.) • In practice, can’t build a model for each query • Instead a general model is built based on query-document pairs in the historical (training) data • Then for a given query Q, the discriminant is computed only based on the conditional probabilities of the query terms • If query term t occurs in D, take P(t|R) and P(t|NR) • If query term t does not appear in D, take 1-P(t|R) and 1- P(t|NR) Q = t1, t3, t4 D = t1, t4, t5 Intelligent Information Retrieval

  32. Strong theoretical basis In principle should supply the best predictions of relevance given available information Can be implemented similarly to Vector Relevance information is required -- or is “guestimated” Important indicators of relevance may not be term -- though terms only are usually used Optimally requires on-going collection of relevance information Probabilistic Models Advantages Disadvantages Intelligent Information Retrieval

  33. Vector and Probabilistic Models • Support “natural language” queries • Treat documents and queries the same • Support relevance feedback searching • Support ranked retrieval • Differ primarily in theoretical basis and in how the ranking is calculated • Vector assumes relevance • Probabilistic relies on relevance judgments or estimates Intelligent Information Retrieval

  34. Extended Boolean Models • Weighted Boolean Queries • Weights are assigned to the operands in Boolean query A0.6ANDB0.75A1.0ORB0.3 • The weighting operation depends on the distance between document sets for A and B • a weight of 1.0 says that all of the corresponding document set is considered in the operation • a weight of 0 < w < 1 says that only a portion of the document set is considered • the documents added or deleted are those that are “closest” to the current set of documents Intelligent Information Retrieval

  35. Weighted Boolean Queries A1.0 AND B1.0 = A Ç B A1.0 OR B1.0 = A È B A1.0 AND B0.0 = AA1.0 OR B0.0 = A A1.0 OR B.75 = A È (75% ofB - A) A1.0 AND B.75 = (A Ç B) È (25% of A - B) A B A B Intelligent Information Retrieval

  36. Weighted Boolean Queries • Matching Algorithm 1. Find initial matching set (non-weighted Boolean document set) 2. Find the invariant document set (set of documents that are present both when operand weight is 1.0 and when the weight is 0.0); the optional set is the remaining items 3. Compute the centroid of the invariant set 4. Find the number of documents, say k, from the optional set that will potentially be added to the invariant set (determined by the weight of the query term) 5. Compute similarity between documents in the optional set and the centroid (of the invariant set) 6. Items to be added or deleted are the top k documents in the optional set with the highest similarity scores Intelligent Information Retrieval

  37. Demo of Extended Boolean Query* http://ir.exp.sis.pitt.edu/res2/data/66/ *Thanks to Michael Bombyk for discovering this applet! Intelligent Information Retrieval

  38. Weighted Boolean Queries - Example Q1(initial) = (D1, D2, D3, D4, D5, D6, D8) Q1(invariant) = (D3, D6, D8) Q1(optional) = (D1, D2, D4, D5) => 4 items No. selected docs. = Centroid(Q1) = (1/3) = (4.7, 0.7, 2.0, 2.0) Computing Similarity (using simple matching): SIM(Centroid,D1) = (4.7,0.7,2.0,2.0).(0,4,0,8) = 18.8 SIM(Centroid,D2) = (4.7,0.7,2.0,2.0).(0,2,0,0) = 1.4 SIM(Centroid,D4) = (4.7,0.7,2.0,2.0).(0,6,4,6) = 24.2 SIM(Centroid,D5) = (4.7,0.7,2.0,2.0).(0,4,6,4) = 22.8 So the final Hit list is : (D3, D6, D8) È (D4, D5) Query Q1 = A1.0 OR B.333 Intelligent Information Retrieval

  39. Weighted Boolean Queries - Example Q2(initial) = (D3, D4, D5) Q2(invariant) = (D3, D4, D5) Q2(optional) = (D1, D8) => 2 items No. selected docs. = Centroid(Q2) = (1/3) = Computing Similarity (using simple matching): SIM(Centroid,D1) = SIM(Centroid,D8) = Final Hit list is: (D3, D4, D5) È (D1) Query Q2 = C.75 AND D1.0 Intelligent Information Retrieval

More Related