1 / 21

Making Sense of Twitter Search: Evaluation in Interactive Retrieval

Learn about the importance of test collections in evaluating Twitter search systems. Explore how to define research questions, collect data, determine metrics, and study user behaviors on Twitter. Understand the first steps in conducting evaluation in information retrieval.

cedric
Télécharger la présentation

Making Sense of Twitter Search: Evaluation in Interactive Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making Sense of Twitter Search Gene Golovchinsky / FXPAL Miles Efron / UIUC

  2. 100,000,000 people can’t be wrong

  3. All the cool researchers are doing it

  4. There is Science in everything

  5. We can build something cool

  6. Maybe we can build something useful!

  7. How do we know we’ve succeeded?

  8. Test collections (data sets) are one way to evaluate systems

  9. The fire hose is not a test collection

  10. Test collection to-do list • Research questions • Data collection methodology • Metrics • Data!

  11. What are we trying to study?The network?

  12. Or the person?

  13. We’re interested in interactive search

  14. What do people look for on Twitter? • Trending topics • Events • Specific tweets • Catching up • Etc.

  15. First Steps

  16. First Steps

  17. First Steps

  18. Evaluation in Information Retrieval • Need a collection • Need tasks • Need metrics • Ground Truth helps

  19. How do we sample the data? • By time, by topic, randomly? …

  20. How do we measure?

  21. Credits Title slide http://www.flickr.com/photos/matthamm/3383916444/ Matt Hamm http://www.flickr.com/photos/nixart/370553161/ nixArt “10,000 people can’t be wrong” http://www.flickr.com/photos/luc/1824234195/ luc legay “All the cool researchers are doing it” Gene Golovchinsky “There is science in everything” Gene Golovchinsky “We can build something cool” http://www.flickr.com/photos/donpezzano/3230179951/ urban don “Maybe we can build something useful!” http://www.flickr.com/photos/herzogbr/2274372747/ herzogbr “How do we know we’ve succeeded?” http://www.flickr.com/photos/14696088@N04/1606128341/ rstrawser “Test collections (data sets) are one way to evaluate systems” http://www.flickr.com/photos/_after8_/4052028795/ Régis Gaidot “The fire hose is not a test collection” http://www.flickr.com/photos/magh/2565471904/ Magh “Test collection to-do list” http://www.flickr.com/photos/ario/3362388566/ ario_ “What are we trying to study? The network?” http://www.flickr.com/photos/meg/3533025291 Meg Pickard “Or the person?” Gene Golovchinsky We are interested in interactive search http://www.flickr.com/photos/dullhunk/3389581452/ dullhunk What do people look for on Twitter? http://www.flickr.com/photos/kevglobal/3170506186/ kevglobal First steps (3) Gene Golovchinsky Evaluation in Information Retrieval http://www.flickr.com/photos/ozyman/155517365/ Ozyman How do we sample the data? http://www.flickr.com/photos/red_devil/18540700/ SeenyaRita http://www.flickr.com/photos/texasrobo/4185851361/ texasrobo How do we measure? http://www.flickr.com/photos/jurvetson/17945646/ jurvetson

More Related