Helping People Find Information Better

1. Helping People Find Information Better Jaime Teevan, MIT with Christine Alvarado, Mark Ackerman and David Karger

2. Overview:Understanding Search Directed • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what?

3. Overview:Understanding Directed Search • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what?

4. Haystack Haystack:Personal Information Storage Web pages Email Files Calendar Contacts

5. Directed Search in Haystack What was that paper I read last week about Information Retrieval? Haystack

6. Directed Search in Haystack Ah yes! Thank you. Haystack

7. …Or Elsewhere Ah yes! Thank you.

8. Related Work • Directed search • Lab studies [Capra03, Maglio97] • Log analysis [Broder02, Spink01] • Observational studies [Malone83] • Information Seeking • Marchionini, O’Day and Jeffries, Bates, Belkin, … • Evolving information need

9. Naturalistic Study of Current Tools • Subjects: 15 CS graduate students • Modified Diary Study • Ten interviews each • Asked about what they had just done • 2 interviews a day • Collected over 5 days

10. Let Me Interview You! • Email: • What’s the last email you read? What did you do with it? • Have you gone back to an email you’ve read before? • Web: • What’s the last Web page you visited? How did you get there? • Have you looked for anything on the Web? • Files: • What’s the last file you looked at? How did you get to it? • Have you looked for a file?

11. Interview Questions • Two question types • Last email/file/Web page looked at • Last email/file/Web page looked for • Qualitative data • Advantages • Naturalistic, exploratory • Gives a rich understanding • Can be coded  quantitative • Drawbacks • Overwhelming!

12. Overview:Understanding Directed Search • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what? Prefer to search in steps Because it’s easier Step size varies by person

13. Directed Search Today • Target: Connie Monroe’s office number  Type into a search engine: “Connie Monroe, office number”

14. What We Observed Interviewer: Have you looked for anything on the Web today? Jim: I had to look for the office number of the Harvard professor. I: So how did you go about doing that? J: I went to the homepage of the Math department at Harvard

15. What We Observed I:So you went to the Math department, and then what did you do over there? J:It had a place where you can find people and I went to that page and they had a dropdown list of visiting faculty, and so I went to that link and I looked for her name and there it was.

16. What We Observed J:I knew that she had a very small Web page saying, “I’m here at Harvard. Here’s my contact information.”

17. Strategies Looking for Information Teleporting Orienteering

18. Why Do People Orienteer? • The tools don’t work • Easier than saying what you want • You know where you are • You know what you find

19. Easier Than Saying What You Want • Describing the target is hard • Can’t • Prefer not to • Habit • “Whichever way I remember first.” • Search for source • E.g., Your last email search

20. You Know Where You Are • Stay in known space • URL manipulation • Bookmarks • History • Backtracking • Following an information scent • Never end up at a dead end

21. You Know What You Find • Context gives understanding of answer “I was looking for a specific file. But even when I saw its name, I wouldn’t have known that that was the file I wanted until I saw all of the other names in the same directory…” • Understanding negative results “I basically clicked on every single button until I was convinced… I don’t think that it exists…”

22. Individual Strategies • Search strategies varied by individual • People who pile information take small steps • People who file information take big steps • Where was the last email you found? • Inbox? • Elsewhere?

23. File or Pile Email Filer Piler

24. How Individuals Search For Files Filers Big steps Pilers Small steps

25. Applying What We Learned  Support orienteering • Advantages to orienteering • Easier than saying what you want • You know where you are • You know what you find • Individual differences in step size • Meta-info, source, flag sources with info • URL manipulation, paths apparent, all steps • Answer context, trusted sources, exhaustive • Allow for different step sizes

26. Structural Consistency Important All must be the same to re-find the information! New name

27. Do we need magic?

28. “Pick a card, any card…”

29. Abracadabra! Case 1Case 2Case 3Case 4Case 5Case 6

30. Your Card is GONE!

31. People Forget a Lot

32. Absolute Consistency Unnecessary New name Focus on search result lists

33. Focus on Search Results • Search results change a lot… • Tracked 10 queries on Google • 3 of top 10 results gone in first couple of weeks • Rate of change will increase • …But people repeat queries! • Lots of re-visitation on the Web • Re-finding consistently reported as a problem

34. Re:Search Engine ?

35. Merge Old and New Results Old Merged New

36. Change Blindness http://www.usd.edu/psyc301/ChangeBlindness.htm

37. Change Blindness http://www.usd.edu/psyc301/ChangeBlindness.htm

38. We still need to be psychic!

39. Memory Study • Participants issued self-selected query • After an hour, asked to fill out a survey • 100+ participants

40. Query Changes • Most changes are simple • Capitalization • Phrasing • Word ordering • Word form • New queries shorter • What about longer time horizons? • Recognition v. recall

41. Memorability a Function of Rank

42. Remembered Results Ranked High

43. Thank you! Jaime Teevan teevan@mit.edu