The Perfect Search Engine Is Not Enough

The Perfect Search Engine Is Not Enough Jaime Teevan, MIT with Christine Alvarado, Mark Ackerman and David Karger

Let Me Interview You! • Web: • What’s the last Web page you visited? How did you get there? • Have you looked for anything on the Web? • Email: • What’s the last email you read? What did you do with it? • Have you gone back to an email you’ve read before? • Files: • What’s the last file you looked at? How did you get to it? • Have you looked for a file?

Overview:Understanding Search Directed • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what? Prefer to search in steps Because it’s easier Step size varies by person

Haystack Haystack:Personal Information Storage Web pages Email Files Calendar Contacts

Directed Search in Haystack What was that paper I read last week about Information Retrieval? Haystack

Directed Search in Haystack Ah yes! Thank you. Haystack

…Or Elsewhere Ah yes! Thank you. “Perfect Search Engine”

Related Work • Directed search • Lab studies [Capra03, Maglio97] • Log analysis [Broder02, Spink01] • Observational studies [Malone83] • Information Seeking • Marchionini, O’Day and Jeffries, Bates, Belkin, … • Evolving information need

Modified Diary Study • Subjects: 15 CS graduate students • Ten interviews each (2/day x 5 days) • Two question types • Last email/file/Web page looked at • Last email/file/Web page looked for • Supplemented with direct observation and an hour-long semi-structured interview

Overview:Understanding Directed Search • Introduction • Related work • Methodology • What we learned • How? • Why? • Who? • So what?

Directed Search Today • Target: Connie Monroe’s office number  Type into a search engine: “Connie Monroe, office number”

What We Observed Interviewer: Have you looked for anything on the Web today? Jim: I had to look for the office number of the Harvard professor. I: So how did you go about doing that? J: I went to the homepage of the Math department at Harvard

What We Observed I:So you went to the Math department, and then what did you do over there? J:It had a place where you can find people and I went to that page and they had a dropdown list of visiting faculty, and so I went to that link and I looked for her name and there it was.

What We Observed J:I knew that she had a very small Web page saying, “I’m here at Harvard. Here’s my contact information.”

Strategies Looking for Information Teleporting Orienteering

Why Do People Orienteer? • The tools don’t work • Easier than saying what you want • You know where you are • You know what you find

Easier Than Saying What You Want • Describing the target is hard • Can’t • Prefer not to • Habit • “Whichever way I remember first.” • Search for source • E.g., Your last email search

You Know Where You Are • Stay in known space • URL manipulation • Bookmarks • History • Backtracking • Following an information scent • Never end up at a dead end

You Know What You Find • Context gives understanding of answer “I was looking for a specific file. But even when I saw its name, I wouldn’t have known that that was the file I wanted until I saw all of the other names in the same directory…” • Understanding negative results “I basically clicked on every single button until I was convinced… I don’t think that it exists…”

Individual Strategies • Search strategies varied by individual • People who pile information take small steps • People who file information take big steps • Where was the last email you found? • Inbox? • Elsewhere?

File or Pile Email Filer Piler

How Individuals Search For Files Filers Big steps Pilers Small steps

Applying What We Learned  Support orienteering • Advantages to orienteering • Easier than saying what you want • You know where you are • You know what you find • Individual differences in step size • Meta-info, source, flag sources with info • URL manipulation, paths apparent, all steps • Answer context, trusted sources, exhaustive • Allow for different step sizes

Structural Consistency Important All must be the same to re-find the information!

Preserve What User Remembers • Supports orienteering for re-finding • Allows access to new information

More to Learn from the Data • Differences in finding v. re-finding • How organization relates to search • Importance of type (email, files and Web) • Looked at v. looked for  Keep in mind population

Questions? Teevan, J., Alvarado, C., Ackerman, M. S. and Karger, D. R. (2004). The Perfect Search Engine is Not Enough: A Study of Orienteering Behavior in Directed Search. To appear in Proceedings of CHI 2004. (Linked from http://www.teevan.org)

Relating How and What • People only keyword search 39% of the time • What people look for related to how they look Orienteer to specific information • Surprise:

Relating How and Corpus • Email and files: Almost never keyword searched • Easy to associate information with document • Web: Used keyword search much more often

Relating What and Corpus • Email searches were primarily for specific information • File searches were primarily for documents • Web searches were more evenly distributed

The Perfect Search Engine Is Not Enough