The Perfect Search Engine Is Not Enough

1. The Perfect Search Engine Is Not Enough Jaime Teevan†, Christine Alvarado†, Mark S. Ackerman‡ and David R. Karger† † MIT, CSAIL ‡ University of Michigan

2. Let Me Interview You! • Email: • What’s the last email you read? What did you do with it? • Have you gone back to an email you’ve read before? • Web: • What’s the last Web page you visited? How did you get there? • Have you looked for anything on the Web? • Files: • What’s the last file you looked at? How did you get to it? • Have you looked for a file?

3. Overview:Understanding Search Directed • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what? • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what?

4. Haystack Haystack:Personal Information Storage Web pages Email Files Calendar Contacts

5. Directed Search in Haystack What was that paper I read last week about Information Retrieval? Haystack

6. Directed Search in Haystack Ah yes! Thank you. Haystack “Perfect Search Engine”

7. Related Work • Directed search • Lab studies [Capra03, Maglio97] • Log analysis [Broder02, Spink01] • Observational studies [Malone83] • Information Seeking • Marchionini, O’Day and Jeffries, Bates, Belkin, … • Evolving information need

8. Modified Diary Study • Subjects: 15 CS graduate students • Ten interviews each (2/day x 5 days) • Two question types • Last email/file/Web page looked at • Last email/file/Web page looked for • Supplemented with direct observation and an hour-long semi-structured interview

9. Overview:Understanding Directed Search • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what?

10. Directed Search Today • Target: Connie Monroe’s office number  Type into a search engine: “Connie Monroe, office number”

11. What We Observed Interviewer: Have you looked for anything on the Web today? Jim: I had to look for the office number of the Harvard professor. I: So how did you go about doing that? J: I went to the homepage of the Math department at Harvard

12. What We Observed I:So you went to the Math department, and then what did you do over there? J:It had a place where you can find people and I went to that page and they had a dropdown list of visiting faculty, and so I went to that link and I looked for her name and there it was.

13. What We Observed J:I knew that she had a very small Web page saying, “I’m here at Harvard. Here’s my contact information.”

14. Strategies Looking for Information Teleporting Orienteering

15. Why Do People Orienteer? • Easier than saying what you want • You know where you are • You know what you find • The tools don’t work

16. Easier Than Saying What You Want • Describing the target is hard • Can’t • Prefer not to • Habit • “Whichever way I remember first.” • Search for source • E.g., Your last email search

17. You Know Where You Are • Stay in known space • URL manipulation • Bookmarks • History • Backtracking • Following an information scent • Never end up at a dead end

18. You Know What You Find • Context gives understanding of answer “I was looking for a specific file. But even when I saw its name, I wouldn’t have known that that was the file I wanted until I saw all of the other names in the same directory…” • Understanding negative results “I basically clicked on every single button until I was convinced… I don’t think that it exists…”

19. Individual Search Behavior • Search behavior varied by individual • Categorize based on email usage • Filers • Pilers • People who pile information take small steps • People who file information take big steps

20. How Individuals Search For Files Filers Big steps Pilers Small steps

21. More to Learn from the Data • Differences in finding v. re-finding • How organization relates to search • Importance of type (email, files and Web) • Looked at v. looked for  Keep in mind population

22. Applying What We Learned  Support orienteering • Advantages to orienteering • Easier than saying what you want • You know where you are • You know what you find • Individual differences in step size • Highlight source (e.g., flag sources with info) • Integrate tools used for steps • Support exhaustive search • Allow for different step sizes

23. More to Learn from the Data • Differences in finding v. re-finding • How organization relates to search • Importance of type (email, files and Web) • Looked at v. looked for  Keep in mind population

24. Structural Consistency Important All must be the same to re-find the information!

25. Preserve What User Remembers • Supports orienteering for re-finding • Allows access to new information

26. File or Pile Email Filer Piler

27. Searching Other Collections Ah yes! Thank you.

28. Keep Population in Mind • CS grad students not representative • Very familiar with search tools  Would expect to see lots of tool use

29. Relating How and What • People only keyword search 39% of the time • What people look for related to how they look Orienteer to specific information • Surprise:

30. Relating How and Corpus • Email and files: Almost never keyword searched • Easy to associate information with document • Web: Used keyword search much more often

31. Relating What and Corpus • Email searches were primarily for specific information • File searches were primarily for documents • Web searches were more evenly distributed