1 / 68

Hypersearching the Web

Soumen Chakarbarti, Byron Dom, S. Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan & Andrew Tomkins. Hypersearching the Web. Hira Bashir - June 22, 2010. …. …. …. …. …. …. CHAOS!. …. …. …. …. I’m a Search Engine!. Enter your text here…. Search. I’m a Search Engine!.

winfred
Télécharger la présentation

Hypersearching the Web

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Soumen Chakarbarti, Byron Dom, S. Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan & Andrew Tomkins Hypersearching the Web Hira Bashir - June 22, 2010

  2. … … … … …

  3. … CHAOS! … … … …

  4. I’m a Search Engine! Enter your text here… Search

  5. I’m a Search Engine! Learn Pashto Language| Search

  6. I’m a Search Engine! Learn Pashto Language Search

  7. Words WebPages containing the word A Abate Abash . . . Webpage 1, Webpage 2, … , Webpage L Webpage 1, Webpage 2, … , Webpage M Webpage 1, Webpage 2, … , Webpage N . . .

  8. Index Words WebPages containing the word A Abate Abash . . . Webpage 1, Webpage 2, … , Webpage L Webpage 1, Webpage 2, … , Webpage M Webpage 1, Webpage 2, … , Webpage N . . .

  9. Index Words WebPages containing the word A Abate Abash . . . Webpage 1, Webpage 2, … , Webpage L Webpage 1, Webpage 2, … , Webpage M Webpage 1, Webpage 2, … , Webpage N . . . Challenge: Creating, maintaining, using

  10. Heuristics Ranking Function

  11. Pashto is the National language of Afghanistan. Pashto is extensively spoken by Pashtuns tribes in Pakistan’s rural cities. Pashto has two major Dialects. The Pashto spoken in Peshawar differs from the Pashto spoken in Afghanistan or Baluchistan. This text would cover the basic vocabulary to familiarize you with this language. If you know Urdu, it would be easier you learn written…………. Pashto is the National language of Afghanistan. Pashto is extensively spoken by Pashtuns tribes in Pakistan’s rural cities. Pashto has two major Dialects. The Pashto spoken in Peshawar differs from the Pashto spoken in Afghanistan. Pashto poetry and pashto songs have been a highlight of pashto language. Pashto has a rich vocabulary. Pashto is much older then…………………………………. Heuristics Ranking Function

  12. Pashto is the National language of Afghanistan. Pashto is extensively spoken by Pashtuns tribes in Pakistan’s rural cities. Pashto has two major Dialects. The Pashto spoken in Peshawar differs from the Pashto spoken in Afghanistan or Baluchistan. This text would cover the basic vocabulary to familiarize you with this language. If you know Urdu, it would be easier you learn written…………. Pashto is the National language of Afghanistan. Pashto is extensively spoken by Pashtuns tribes in Pakistan’s rural cities. Pashto has two major Dialects. The Pashto spoken in Peshawar differs from the Pashto spoken in Afghanistan. Pashto poetry and pashto songs have been a highlight of pashto language. Pashto has a rich vocabulary. Pashto is much older then…………………………………. Heuristics Learn Pashto Language Welcome to the world’s best resource to learn…. -Pashto Poetry -Pashto Poets -Greetings in Pashto Welcome to the world’s best resource to learn…. Ranking Function

  13. Pashto is the National language of Afghanistan. Pashto is extensively spoken by Pashtuns tribes in Pakistan’s rural cities. Pashto has two major Dialects. The Pashto spoken in Peshawar differs from the Pashto spoken in Afghanistan or Baluchistan. This text would cover the basic vocabulary to familiarize you with this language. If you know Urdu, it would be easier you learn written…………. Pashto is the National language of Afghanistan. Pashto is extensively spoken by Pashtuns tribes in Pakistan’s rural cities. Pashto has two major Dialects. The Pashto spoken in Peshawar differs from the Pashto spoken in Afghanistan. Pashto poetry and pashto songs have been a highlight of pashto language. Pashto has a rich vocabulary. Pashto is much older then…………………………………. What’s the problem? Heuristics Learn Pashto Language Welcome to the world’s best resource to learn…. -Pashto Poetry -Pashto Poets -Greetings in Pashto Welcome to the world’s best resource to learn…. Ranking Function

  14. Cheap airfare cheap airfare cheap airfare cheap airfare…………….!!!

  15. Cheap airfare cheap airfare cheap airfare cheap airfare…………….!!!

  16. Cheap airfare cheap airfare cheap airfare cheap airfare…………….!!! Ok, I do spamming but it ain’t just me!

  17. Cheap airfare cheap airfare cheap airfare cheap airfare…………….!!! Ok, I do spamming but it ain’t just me! Polysemy Synonymy

  18. Any Solution?

  19. WordNet Project Indexed-based Search Engine By George A. Miller & Colleagues At Princeton University

  20. WordNet Project Indexed-based Search Engine By George A. Miller & Colleagues At Princeton University Semantic Network Group of human linguistics

  21. WordNet Project Indexed-based Search Engine By George A. Miller & Colleagues At Princeton University Car Automobile Semantic Network

  22. WordNet Project Indexed-based Search Engine By George A. Miller & Colleagues At Princeton University Car Automobile Semantic Network Now search

  23. Any Challenges?

  24. Polysemy would aggravate!

  25. Polysemy would aggravate! And even with Synonymy….

  26. Polysemy would aggravate! Browse FAQs BOTs

  27. Clever Project -at IBM Are we ignoring something useful…

  28. Clever Project -at IBM Yes…

  29. Clever Project -at IBM . . .More than a billion carefully placed hyperlinks!

  30. Some issues in search queries: • Harvard example • IBM example • Website Design

  31. Some issues in search queries: • Harvard example • IBM example • Website Design Query List --- --- ---- Let’s fix this! Override with Predetermined RIGHT answers

  32. Some issues in search queries: • Harvard example • IBM example • Website Design Query List --- --- ---- Let’s fix this! An Observation Override with Predetermined RIGHT answers

  33. Clever Project -at IBM Underlying Approach/Idea LocationA link to Location B Location B An implicit endorsement of Location B by Location A

  34. Clever Project -at IBM Recommendation Recommendation Recommendation Recommendation Hub Hub Authority -My Fav Links -Commercial Links -Personal Inventories Hub Finding authoritative sites on broad topics with the help of hyperlinks

  35. Clever Project -at IBM What computational method is used to identify hubs and authorities? Page 1 Page 2 Page 3 . . . Candidate Pages Good Hub Good Authority Initial estimates by guessing Yes No Yes . . . No No Yes . . . Used to Improve Guess about Authorities Estimate about Hubs

  36. Clever Project -at IBM What computational method is used to identify hubs and authorities? Hub Hub Hub Hub Used to Improve Guess about Authorities Estimate about Hubs Hub

  37. Clever Project -at IBM What computational method is used to identify hubs and authorities? Hub Hub Hub Hub Used to Improve Guess about Authorities Estimate about Hubs Hub

  38. Clever Project -at IBM What computational method is used to identify hubs and authorities? Hub Hub Hub Hub Used to Improve Guess about Authorities Estimate about Hubs Hub Where does the best hubs point most heavily at?

  39. Clever Project -at IBM More light on implementation Topic: Acupuncture Initial List (200 pages) By any standard text index, such as Alta Vista

  40. Clever Project -at IBM More light on implementation Topic: Acupuncture 1000 - 5000 0 Initial List (200 pages) Augmented List By any standard text index, such as Alta Vista Pages that link to and from the pages in the Initial list

  41. Clever Project -at IBM More light on implementation Topic: Acupuncture 1000 - 5000 0 Initial List (200 pages) Augmented List Initial Authority Score Initial Hub Score Sum of hub scores of other locations pointing to it Sum of authority scores of other locations pointing to it Root Set

  42. Clever Project -at IBM More light on implementation Topic: Acupuncture 1000 - 5000 0 Initial List (200 pages) Augmented List Initial Authority Score Initial Hub Score Sum of hub scores of other locations pointing to it Sum of authority scores of other locations pointing to it Root Set These scores will be re-adjusted iteratively, until results are fine-tuned & start settling down!

  43. Clever Project -at IBM In visual terms . . .

  44. Clever Project Looking at the mathematics behind . . . -at IBM Matrix Iteratively Vector (Hub score & Authority score) Numerical values defining the hyperlinking structure of the root set Result Hub & Authority Vector (Eigen Vector) Equilibrated to a certain number!

  45. Clever Project Looking at the mathematics behind . . . -at IBM Matrix Iteratively Observations -If root set’s size is 3000 pages, 5 rounds of calculations will be enough to steady the Hub and Authority scores -Algorithm is independent of initial scores Vector (Hub score & Authority score) Numerical values defining the hyperlinking structure of the root set Result Hub & Authority Vector (Eigen Vector) Equilibrated to a certain number!

  46. Clever Project A By-product of Clever! -at IBM Iterative Process Separation of Websites Cluster 3 Cluster 1 Cluster 3

  47. Clever Project A By-product of Clever! -at IBM Iterative Process Abortion Separation of Websites Pro-choice Pro-life Cluster 3 Cluster 1 Cluster 3

  48. Clever Project A Larger Perspective . . . -at IBM Chaotic cover Inherent albeit inchoate order Based on how pages are linked!

  49. Clever Project Garfield Measure -at IBM Paper 1 … Refrences … paper 4 … … Paper 2 … Refrences … paper 4 … … Paper 3 … Refrences … paper 2 … … Paper 4 … Refrences … paper 1 … … Paper 5 … Refrences … paper 4 … … Reference: Eugene Garfield, founder of Science Citation Index

More Related