Federated Search:The Technology For Making Global Discovery a Reality Abe Lederman, President and CTO Deep Web Technologies 2008 STIP Working Meeting, April 23, 2008
Who We Are… Founded in 2002 Headquartered in Santa Fe, New Mexico 20 Employees Over $2 million in R&D through DOE SBIR program DWT is focused on providing state-of-the-art federated search products and solutions which search, retrieve, aggregate and analyze content from web-based databases.
Abe Lederman-Background Earned B.S. and M.S. Computer Science degrees, MIT Began work in information retrieval in 1988, Verity Consulted to LANL, 1994-2000 Developed some of the first web-based applications that searched text-based content, 1994 Pioneered “Deep Web” searching in 1999, OSTI
Some of Deep Web’s Customers DOE Office of Scientific and Technical Information Department of Defense National Agricultural Library Scitopia.org The Intel Corporate Library Stanford University
What is Federated Search? Federated Search is an application or service that allows a user to submit a search in parallel to multiple, distributed information sources and retrieve aggregated, ranked and de-duped results.
In Other Words…One Search, Many Sources Subscription Sources Journal Articles Technical Reports Conference Proceedings Library Resources Public Web Sources
Benefits of Federated Search • One-stop access to multiple information sources • Users don’t need to know where/how to search • Saves researcher time and money • Improves utilization of information sources • Consolidated, ranked and de-duped results • Important results are not missed
Speeding up the Diffusion of New Scientific Knowledge Global Discovery 2
Math Databases: • Research Papers • Correspondence • Conferences Global Discovery Search Portal Knowledge Diffusion in Action Mathematician’s Scientific Discovery Math Community • Biology Databases: • Research Papers • Correspondence • Conferences Biology Researcher’s Scientific Discovery Biology Community • Physics Databases: • Research Papers • Correspondence • Conferences Physics Scientific Discovery Physics Community
Challenges of Global Discovery • Identify, configure and maintain access to reliable information sources • Automatically select sources to search • Perform hundreds or thousands of searches in parallel • Rank and organize results retrieved Goal: Search 1000 sources by February 2009
Science News Portal Global Discovery Search Portal Science Daily Science News Online Eureka Science News Eureka Alert! Pub Med ETOC Agricola DEFF USGS SciELO NASA TRS Vascoda IEEE APS SPIE IOP Nuclear fission
Biology & Medicine Math Engineering Computer Science Nuclear Physics Geosciences Chemistry Renewable Energy Using Federated Search at your DOE Lab Workstation Internal Sources Public sources including Science.gov worldwidescience.org Subscription Sources
Turning Vision into Reality Abe Lederman firstname.lastname@example.org www.deepwebtech.com Thank you! 11