The Mellon-Funded Fedora Project A Briefing for the Cornell University Library January 24, 2002

1. The Mellon-Funded Fedora ProjectA Briefing for the Cornell University LibraryJanuary 24, 2002 Sandy Payette Thorny Staples Ross Wayland

2. The Mellon Fedora Project History and Motivation

3. The FEDORA Open-source Development Project January 24, 2002

4. Digital Library Projects • Web sites with links to on-line resources • Specific, “boutique” collections • Large collections in one or two area • A broad research collection in all media types and content areas • Ideally, the digital library includes all information

5. Library Digital Centers

6. Library Digital Collections

7. Other Library Services • Electronic Cataloger in the Cataloging Department • Digital Library Research and Development Department • Digital Services Integration (DSI) Coordinator • Digital Library Production Services

8. Other Services Housed in the Library • The Institute for Advanced Technology in the Humanities • The Virginia Center for Digital History • The Teaching Technologies Initiative • The Media Studies Program Offices

9. Information Communities Community-oriented resources Richer collections Specialized access and delivery Discipline-specific services

10. Managing the Collection • Provide a way to universally name all resources without respect to machine address • Track all files for resources, metadata and computer programs consistently • Enforce appropriate policies for use of Library resources • Provide a high level of security • Support preservation activities appropriately

11. Delivering the Collection • Deliver tools with content • Allow every resource to be used in any number of contexts • Discovery searching across the full collection • Deep searching in particular collections • Move towards a library which aware user’s can configure for themselves

12. Supporting Digital Scholarship • Supporting the creation of digital scholarly projects • Collecting born-digital scholarly projects • For preservation • Taking over responsibility for primary delivery • Supporting information communities

13. Metadata • Descriptive – metadata that users use to find things, like traditional library catalog records • Administrative – metadata that the library uses to manage library resources • Structural – metadata about the relationships among resources • Behavioral – computer programs that deliver digital resources to users

14. Digital Library Management and Delivery System

15. The Flexible Extensible Digital Object Repository Architecture (FEDORA) • Developed as an NSF-funded research project at Cornell • Interpreted and re-implemented at UVA • Testbed of 10,000,000 digital objects with very good results • Mellon gave us $1,000,000 to develop a usable system around FEDORA

16. Repository DevelopmentProject Goals • An efficient, scalable, freely distributable FEDORA repository system ASAP • A complete basic management interface with the initial release • Add important digital library functionality in later releases • Create multiple testbed repositories to deploy and evaluate the software • Make all software open source

17. Deployment Group • The Digital Library group, Indiana U. • The Humanities Computing group, New York U. • The Digital Collections and Archives Department, Tufts U. • The Humanities Computing group, Kings College London • The Oxford Digital Library and The Refugee Studies Center, Oxford U. • Audio/Video Project, Library of Congress • A library/academic computing group, Northwestern University

18. Project Plan • Phase 1: Deliver the repository system and the full management interface • Phase 2: Add more production support • Security and policy enforcement • Collection objects • Disk management • Phase 3: Enhance end-user support • Versioning and Editions • Dynamic, Context Sensitive Behaviors • Efficiency and scale optimization

19. FEDORA Development Project Description: http://fedora.comm.nsdlib.org/

20. Fedora Architecture Research History and Overview

21. Management - of distributed digital content and services Access – via stable interfaces to digital objects Interoperability - for digital objects and repositories Extensibility – easy evolution of object behaviors Flexibility - community-defined content models Security - rights management and access control Preservation – of content and “look and feel” FEDORAOriginal Research Goals

22. Digital Object Containerfor aggregating any digital content Content disseminations based on behavior definitions Extensibility of behavior mechanisms Repository Service layer for “contained” Digital Objects Object lifecycle management Access management FEDORA Basic Architectural Abstractions

23. FEDORA Digital Object Globally unique persistent id Persistent ID ( PID ) Public view: access methods for obtaining “disseminations” of digital object content Disseminators Internal view: metadata necessary to manage the object System Metadata Protected view: content that makes up the “basis” of the object Datastreams

24. Persistent ID ( PID ) Persistent ID ( PID ) Image Disseminator Image Disseminator System Metadata System Metadata Datastream - mrsid Datastream – hres jpg Datastream – lres gif Datastream – thumb gif Datastream – tiff master Digital Object InteroperabilityCommon Behaviors for variable content Digital Object #2 Digital Object #1 Functional equivalency

25. Persistent ID ( PID ) Persistent ID ( PID ) Book Disseminator Book Disseminator Book Photo Disseminator System Metadata System Metadata Datastream Datastream Datastream Photo Collection Datastream Digital Object ExtensibilityAdding New Behaviors Digital Object #3 The same underlying content... to create new disseminations not originally conceived of can be operated on in novel ways…

26. FEDORA Digital Object Architecture Behavior Definition Object Data Object Persistent ID ( PID ) Method Definition Persistent ID ( PID ) Metadata System Metadata Disseminators Datastreams (specs) Behavior Mechanism Object System Metadata Persistent ID ( PID ) Method Implementation Metadata System Metadata Datastreams Datastreams (executables)

27. UVA Example: Shared Image Behavior Definitions

28. UVA Example: Default Behavior Definitions

29. PID PID Disseminators Disseminators System System Metadata Metadata Basis Basis (Datastreans) (Datastreans) FedoraRepository System Management Access Digital Objects with fine-grained access control Storage general-purpose access control

30. Access ManagementPolicy Enforcement • Semantics of policy language must parallel the behavioral semantics of digital objects • Fine-grained, context-sensitive policies • Extensibility for policies and enforcement mechanisms • Support for portability of digital objects • Decentralized policy management

31. Access Control Policies • General Purpose • “only repository managers can add new disseminators to digital objects in the repository.” • Object-Specific (“e.g., Lecture object”) • “guests may view course syllabus and slides 1-10 of Lecture 1, but may not view the lecture video or any other slides.” • “students may not view Lecture 2 video unless they submit assignment for Lecture 1.” See research at: http://www.cs.cornell.edu/payette/prism/security/policy.htm

32. UVA Prototypes UVA Content Models and Demos

33. Finding Aid Content Model (Finding Aid example)

34. TEI Letter Content Model (TEI letter example)

35. TEI Book Content Model (TEI book example)

36. General Image Content Model (Mycenae image example)

37. MrSID Image Content Model (Pavilion III image example)

38. 1-bit B/W TIFF Content Model (1-bit B/W TIFF example)

39. GDMS Content Model (Mycenae example) (lawn example)

40. Numerical Data Content Model (ICPSR survey example)

41. FEDORA Specifications – Part I Digital Object Storage

42. New Repository System

43. FEDORA XML using METS

44. Metadata Encoding and Transmission Standard (METS) • XML “standard” for encoding descriptive, administrative, and structural metadata of digital library objects • Developed under auspices of the Digital Library Federation • METS standard maintained by the Network Development and MARC Standards Office of the Library of Congress http://www.loc.gov/standards/mets/

45. METS Schema • METS is written in the XML Schema Language • METS defines four sections for an object • Descriptive metadata • Administrative metadata • File group • Structure map • METS goals include: • Facilitate management of objects within a repository • Provide a standard format for exchange of objects between repositories • Provide standard format for transmission of objects to users for rendering (via tools or applications)

46. Mapping Fedora to METS New in METS

47. METS : Sample Fedora Object Click here for image digital object

48. METS: Sample Fedora Behavior Definition Object Click here for Behavior Definition object for DC Click here for Behavior Definition object for UVA_Images

49. METS: Sample Fedora Behavior Mechanism Object Click here for Behavior Mechanism object for UVA_MARC_DC Click here for Behavior Mechanism object for UVA_Image_STD Click here for Behavior Mechanism object for UVA_Image_MRSID

50. Fedora Relational Database • Phase 1: Alternate form of object storage to support high-performance access (disseminations) • Repository system replicates from authoritative XML version of objects to relational database • Phase 2-3: Access sub-system works completely off the XML storage, as XML tools improve performance-wise.