Edward A. Fox fox@vt fox.cs.vt CS DLRL Internet TIC

1. Digital Libraries: From Theory to Applications in Education and BusinessICADL 2000 – Seoul, KoreaDecember 7, 2000 Edward A. Fox fox@vt.edu http://fox.cs.vt.edu CS DLRL Internet TIC Virginia Tech, Blacksburg, VA, USA

2. Outline • Introduction (5S) • Education (CSTC, NDLTD) • OAI • MARIAN • Conclusions

3. Acknowledgements (Selected) • Conference Organizers and Sponsors • Mentors: JCR Licklider, Michael Kessler, Gerard Salton • Sponsors: Advance Auto Parts, CNI, DLF, IBM, NLM, NSF, OCLC, UNESCO, US Dept. of Ed. (FIPSE), … • VT Faculty/Staff: Tony Atkins, Debra Dudley, John Eaton, Jim Hicks, Lance Matheson, Gail McMillan, James Powell, … • VT Students: Fernando Das Neves, Robert France, Marcos Goncalves, Neill Kipp, Paul Mather, Ryan Richardson, Ohm Sornil, Hussein Suleman, Omar Vasnaik, Marc Vass, … • Visitors: Mann-Ho Lee (Korea), Byongsun Kim (Korea), Shalini Urs (India), Akira Maeda (Japan)

4. Internet TechnologyInnovation Center Supported by Virginia’s Center for Innovative TechnologyStatewide University Partners - Governing Board: • Christopher Newport University • William Winter, William Muir, Virginia Electronic Commerce Technology Center / Southeastern Virginia Network (VECTEC/SEVAnet) • George Mason University • Scott Martin, Internet Multimedia Center (ICM) • Steven Ruth, International Center for Applied Studies in IT (ICASIT) • University of Virginia • Alf Weaver, Internet Commerce Group (InterCom) • Jim French, Internet Digital Library • Virginia Tech • Edward Fox, Digital Library Research Laboratory (DLRL), CC, CS • Scott Midkiff, Center for Wireless Telecomm. (CWT), VTISC, ECpE

5. JCDL 2001 • First Joint ACM/IEEE Conference on Digital Libraries (+ NSF DLI-2 PI mtg) • http://www.jcdl.org • June 24-28, 2001 in Roanoke, VA • Conference Committee: • General Chair: Edward A. Fox, Virginia Tech • Program Chair: Christine Borgman, UCLA • Treasurer: Neil Rowe, Naval Postgraduate School • Posters Chair: Craig Nevill-Manning, Rutgers U.

6. URLs • http://fox.cs.vt.edu • http://www.dlib.vt.edu (DLRL) • http://ei.cs.vt.edu/~dlib (Courseware) • www.ndltd.org & www.theses.org • www.cstc.org (CSTC and JERIC) • www.openarchives.org (OAI) • www.jcdl.org (JCDL’2001 – June 24-28)

7. Collaboration! U.S. – Korea Joint Workshop on Digital Libraries San Diego Supercomputer Center August 10 & 11, 2000 Sponsored by National Science Foundation, USA Ministry of Information & Communication, Korea Institute of Information Tech. Assessment, Korea San Diego Supercomputer Center University of Maryland Virginia Tech

8. Robert Allen University of Maryland rba@GLUE.UMD.EDU Dookwon Baik Korea University baik@SWSYS2.KOREA.AC.KR Ching-Chih Chen Simmons College, Boston chen@SIMMONS.EDU Su-Shing Chen University of Missouri - Columbia schen@ECN.MISSOURI.EDU Jonghoon Chun Myongji University jchun@WH.MYONGJI.AC.KR Gregory Crane Tufts University gcrane@PERSEUS.TUFTS.EDU Lois Delcambre Oregon Graduate Institute lmd@CSE.OGI.EDU Edward Fox Virginia Tech fox@VT.EDU Michael Gertz University of California, Davis gertz@CS.UCDAVIS.EDU Stephen Helmreich New Mexico State University shelmrei@CRL.NMSU.EDU Workshop Participants (1 of 3)

9. Ulf Hermjakob USC Information Sciences Institute ulf@ISI.EDU Soon Joo Hyun Information & Communications University (ICU) shyun@ICU.AC.KR Hyeon Kim Korea Research & Development Information Center hyeon@KORDIC.RE.KR Sung-Hyuk Kim Sookmyung Women’s University ksh@SOOKMYUNG.AC.KR Yongchae Kim Ministry of Information & Communication yongari@MIC.GO.KR Ron Larsen University of Maryland rlarsen@DEANS.UMD.EDU Sang-goo Lee Seoul National University sglee@MARS.SNU.AC.KR Sang Ho Lee Soongsil University shlee@COMPUTING.SOONGSIL.AC.KR Young-Suk Lee MIT, Lincoln Laboratory ysl@SST.LL.MIT.EDU Karl Lo University of California, San Diego klo@UCSD.EDU Workshop Participants (2 of 3)

10. Bruce Miller University of California, San Diego Rbmiller@UCSD.EDU Sung Been Moon Yonsei University sbmoon@YONSEI.AC.KR Reagan Moore San Diego Supercomputer Center moore@SDSC.EDU Sung Hyon Myaeng Chungnam National University shmyaeng@CS.CHUNGNAM.AC.KR Gang-Tak Oh National Computerization Agency, Seoul okt@NCA.OR.KR Sam-Gyun Oh SungKyunKwan University samgyun@YAHOO.COM samoh@YURIM.SKKU.AC.KR Hae-Chang Rim Korea University rim@NLP.KOREA.AC.KR Shalini Urs University of Mysore shaliniurs@HOTMAIL.COM Lee Zia National Science Foundation lzia@NSF.GOV Workshop Participants (3 of 3)

11. Some Observations • So many conferences! Lots of R&D! • Exhibits: a DL industry is emerging. • But: we don’t cite each other’s works; • nobody is asking “Why”; • we are not connecting theory + projects; • nobody is talking about OAI. • So, I’ve redone my talk, since you can see: • paper in proceedings • demo tomorrow (p. 327) and online • see tutorial notes (in book) and online

12. DL = Users Direct(Organized Artifact Mediated Communication) Roles Digital Library Author Teacher User Reader Sponsor Learner Reviewer Editor Publisher Librarian

13. DL = Users Direct(Organized Artifact Mediated Communication) Roles Digital Library Parts Supplier Inventory Sales Agent User Training Shopper Repair Manuals Garages Store Sales Partners B2B B2C Home Staff

14. CS 6604: Digital Libraries (Fall 2000) http://scholar.lib.vt.edu/imagebase/ DL of Images of Birds for Virginia Tech Museum of Natural History Student Team Ameya Datey Aniket Sule Supriya Angle Balaprasuna Chennupati and the Eagle Scouts Under the guidance of Dr. Edward Fox Ms. Llyn Sharp (VT Museum of Natural History) Mr. Anthony Atkins (Digital Library and Archives) Plus, 3-D VTMNH minerals in UH3004

15. Libraries of the FutureJCR Licklider, 1965, MIT Press:Unified Theory? • Not ready in 1960s • Analog – unified field theory in physics • “Mess” today – segmented field, specialities • Database <-> Knowledge <-> Content Mgmnt • Multimedia, Hypermedia, Hypertext • Logic, Algebra, Artificial Intelligence, … • Expensive, annoying for users • Don’t know where to look • Don’t know how to use services

16. 5S Layers Societies Scenarios Spaces Structures Streams

17. Definition: Digital Libraries are complex systems that • help satisfy info needs of users (societies) • provide info services (scenarios) • organize info in usable ways (structures) • present info in usable ways (spaces) • communicate info with users (streams)

18. Definition: 5S Framework • Societies: interacting people (, computers) • Scenarios: services, functions, operations, methods • Spaces: domains + constraints (e.g., distance, adjacency): 2D, vector, probability • Structures: relations, trees, nodes and arcs • Streams: sequences of items (text, audio, video, network traffic) • (5 Element System: Fire, Wood, Earth, Metal, Water)

19. 5S: Combinations • Societies + Scenarios = user model • Societies + Scenarios + Spaces = user interface • Streams + Structures = markup • Streams + Structures + Scenarios = object • Structures + Scenarios = DBMS

20. Outline • Introduction (5S) • Education (CSTC, NDLTD) • OAI • MARIAN • Conclusions

21. Portals & Clients Portals & Clients Portals & Clients referenced items & collections referenced items & collections Referenced Items & Collections Core Collection- Building Services harvesting Core Collection- Building Services persistence NSDL Services Core Collection- Building Services protocol mediation NSDL Services Other NSDL Services full-service collections full-service collections NSDL Collections Core Collection- Usage Services annotation CI Services query transform CI Services topic-map registry CI Services personalization CI Services discussion NSDL Spine (Slide from Dave Fulker, Bill Arms – 11/2/2000)

22. ARIADNE Screens (E. Duval)

23. CS Teaching Center (CSTC) • Instead of building large, expensive multimedia packages, that become obsolete and are difficult to re-use, concentrate on small knowledge units. • Learners benefit from having well-crafted modules that have been reviewed and tested. • Use digital libraries to build a powerful base of support for learners, upon which a variety of courses, self-study tutorials & reference resources can be built. • ACM Education Board and SIG support, new NSF grant with UNCW, Eduprise, TCNJ, … - iLumina Project • ACM J. of Educational Resources in Computing (JERIC)

25. Browsing (1)

26. Browsing (2)

30. Domain: graduate education, research Genre: ETDs = electronic theses & dissertations Submission: http://etd.vt.edu Collection: http://www.theses.org Project: Networked Digital Library of Theses & Dissertations http://www.ndltd.org (NDLTD– remember: ND LTD / NDL TD) (also, newer NUDL: Networked University Digital Library, with e-courseware, etc.) A Digital Library Case Study

31. ETD Initiative (and UMI) Education Access Students Learn about DL, EPub TDs become more expressive Global TDs become more accessible, archived Universities N. Amer. (T)Ds are accessible, archived UMI

32. Library Catalogs ETD, Access is Opened to the New Research WWW NDLTD Digital library access control

33. What are the long term goals? • Attract all TDs/yr: 50K D-US, 25K D-Germany, 10K TD-Canada, … • >200K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …) • Dramatic increase in knowledge sharing: literature reviews, bibliographies, … • Services providing lifelong access for students: browse, search, prior searches, citation links • Hundreds/thousands of downloads / year / work

34. The Networked Digital Library of Theses and Dissertations www.NDLTD.org Training Authors Expanding Access Preserving Knowledge Improving Graduate Education Enhancing Scholarly Communication Empowering Students & Universities Leader of the Worldwide ETD (Electronic Thesis and Dissertation) Initiative

35. Outline • Introduction (5S) • Education (CSTC, NDLTD) • OAI • MARIAN • Conclusions

36. Why do we need the Open Archives Initiative ? • Current standards are too complicated • Information wants to be free ! • We can decouple • Running an archive (DL content collection) • Running a service (DL system / operation) • So we can have more and better archives, that build on each other • So we can have better services, that work on multiple collections

37. OAI: Archives of Digital Objects Archive Access Protocol Handle (ID) Metadata terms and conditions Digital object

38. The Open Archives Initiativewww.openarchives.orga technical introduction Hussein Suleman (hussein@vt.edu) Virginia Tech DLRL December 2000

39. History • Santa Fe Convention (October 1999) • Electronic pre-print community • San Antonio (July 2000), Lisbon (Sept. 2000) • Broader interest from other parties • Ithaca Meeting (September 2000) • Formulation of general-purpose protocol • OAI Open Meetings (January –Feb. 2001) • Public release of specifications

40. Federation vs. OAI Harvesting • Federation • Sending out queries to remote sites and combining results • Harvesting • Gathering all metadata from remote sites into a central search system • Lightweight protocol • Robust • Less network traffic • Redundant servers

41. Dissert.Online (Germany) CIC SEALS (S.Africa) NDC (Greece) CBUC (Catalunya) OhioLINK MIT VT U. Bergen (Norway) www.theses.org BN.PT (Portugal) ADT (Australia) NSYSU (Taiwan) PhysDis ISTEC (Ibero America) CyberTheses (Francophone) Black Box OAI-ETD Perspective … …

42. Splitting Data & Services • Data Provider • Implements the OAI protocol on archive to allow external access to data • Service Provider • Uses the OAI protocol to access external archives and provide services (such as searching or linking) on their metadata

43. The Big Picture DL Repository 1 Repository 2 Repository 3 Repository 4

44. Requirements for OAI Protocol • Unique identifiers (URNs) for each record • Date-stamp for each record when last modified/created/deleted • HTTP server with scripting ability

45. OAI Harvesting Protocol v1 • Operates over HTTP • HTTP Requests and XML Responses • HTTP Error codes • 6 Service requests (verbs): • Identify, ListMetadataFormats, ListSets • ListIdentifiers, GetRecord, ListRecords

46. Identify - Response

47. ListMetadataFormats - Response

48. GetRecord - Response

49. Verb: ListRecords • Retrieves metadata for multiple records • Parameters • from – start date (O) • until – end date (O) • set – set to harvest from (O) • resumptionToken – flow control mechanism (X) • metadataPrefix – metadata format (R)

50. ListRecords - Response