1. Taking the Library Back from Google Abe Lederman, President and CTO iGroup, October 18-20, 2007

2. Abe Lederman Background • Earned B.S. and M.S. Computer Science degrees, MIT • Began work in information retrieval in 1988 • Co-founded Verity • Developed one of the earliest web-based search & retrieval applications, 1994 • Pioneered “Deep Web” searching in 1999 • Founded Deep Web Technologies, 2002

3. The “Google Myth” If you can’t find it on Google, the information doesn’t exist.

4. Why Do Students Like Google and Google Scholar? • Quickly searches accessible information • Easy to search • Multi-Lingual support • FREE

7. What Students Don’t know • Links for many articles on Google Scholar require subscriber login or an access fee. • Many libraries have already paid for subscriptions. • Other databases may be more relevant to the topic. • Google Scholar advanced search is not very powerful.

8. So Why NOTGoogle Scholar? • “Scholarly content” intermixed with “non-scholarly” content • Spotty coverage, requires publisher cooperation • Poor relevance (citation counts don’t seem to help) • Unable to limit searching to specific sources • Difficult to find the needles in the haystack

9. Bringing Relevance Back to the Library • Checking the Library Website… • Reference Libraries • Deep Web Databases • Specialized Search Directories • Specialized Search Engines • Subscription Databases • Library Catalogs “Ask A Librarian!”

10. Federated Search… Allows you to search, retrieve, and merge de-duped and relevantly ranked results from multiple high-quality information sources.

11. In Other Words…One Search, Many Sources Subscription Sources Subscription Sources Wikis Blogs Library Catalog Public Web Sources

12. Federated search by any other name…. * Number of occurences on Google (Sept. 21, 2007)

13. Tennant’s Tenets • Only librarians like to search, everyone else likes to find. • All things being equal, one place to search is better than two or more. • Services should be placed as close to the user as possible. Roy Tennant: Internationally known speaker and writer on library and information technology issues.

14. Tennant’s Tenets (cont.) • “Good enough” is often just that. • Our ability to create effective one-stop searching is dependent on our ability to appropriately target user needs. • The size of the result set doesn’t matter as much as how the results are presented. (‘the Google Lesson’)

15. Increase efficiency of information search Make better decisions & fewer mistakes How can Federated Search help your Patrons? More Users, Find More Information Gather the most relevant, important information

16. Benefits of Federated Search • One-Stop Access • Support of More Efficient Research • Consolidated, Formatted Results • Authentication and Access Control

17. One-Stop Access • Single query search across multiple disparate sources in real-time: • Subscription services (journals, bibliographic databases, etc.) • In-house databases • Unstructured data (white papers, theses, reports, etc.) • Catalogs (web-based OPAC) • Publicly available websites Information Discovery

18. Support of More Efficient Research • Results across all selected sources are: • Merged • Presented in relevance-ranked order • Sorted • Clustering

19. Consolidated, Formatted Results • Filtering and removal of duplicates • Ranking by relevance • Displaying results by source • Permitting results to be • Downloaded • Printed • Automatically emailed using Alerts • Flagging of results from subscribed sources

20. Authentication and Access Control • Authorization based on: • IP address • User ID and password • Implementation using: • Proxy server • Management of session cookies • Different access levels for different user groups • Virtual server support

21. #1 Benefit to Libraries Improve your ROI on Licensed Content!

22. What does Federated Search look like? Integrating Trusted Science + Technology Research • 15 leaders in science and technology research • 3 million documents • 150 years of knowledge

29. Bringing Federated Search to Your Patrons

30. Clearly Define Requirements • Compile your list of sources to federate • Access to Internal Sources • Licensed Product vs. Managed Solution • Staff Resources • Timelines • Determine features important to users

31. Will the Real Federated Search Please Step Forward… • Easy-to-Use, Customizable Interface • Incremental display of real-time and indexed results • Clustering or Visualization • Relevance ranking of results on-the-fly • Support Boolean and fielded searching • Access sources via multiple protocols (XML gateway, HTTP, SR/U, SR/W, Z39.50)

32. Sample Features Checklist • Integration with URL Resolver • Integration with EndNote or other export capabilities • Spell Checker • Alerts Options • Export to an RSS Feed • Advanced Customizability

33. Vendor Evaluation • Demonstrations and Pilots • User reaction • Ease of search • Fulfillment of requirements and features • Relationship • Future Relationship • Flexibility

34. The Future of Federated Search • Multi-Lingual Searching • Personal Libraries • Automated Source Selection

35. Conclusion • Help Patrons to… • Improve their search • Find what they need • Access scholarly sources • Return to your library for information

36. Thank You! Abe Lederman abe@deepwebtech.com