1 / 12

Middleware development notes

This document outlines recent improvements in middleware development, focusing on result sets and accumulators. Key topics include enhancements to collection hierarchies and monitoring as well as services for ingest and management. We introduce vocabulary refinements that support per-collection customization, ensuring better aggregation and grouping. The implementation notes detail the structure of collections—both leaf and non-leaf—and the introduction of a new collection-level service for monitoring status. Insights into a thesaurus protocol for cross-references are also provided.

gwidon
Télécharger la présentation

Middleware development notes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Middleware development notes • Result set/accumulator improvements • Collection hierarchies • Collection monitoring • Ingest/management services • Vocabulary refinements

  2. Result sets and accumulators • Result accumulator • now: craps out on first error • new: runs until all threads finish or time out • Result set • tracks middleware and per-collection errors • maintains per-collection counts • still merges & ranks results • maintains mapping between results & collections? • interaction with collection hierarchies?

  3. Collection hierarchies • The problem: • flat namespace • people want hierarchies • aggregation • logical grouping • grouping & easy importing of remote collections

  4. Kinds of collections • From implementation perspective: two kinds of collections • leaf collections • “real,” have items • non-leaf collections • pure aggregations • may or may not expose sub-collections • collection-level metadata • static: custom or auto-generated • dynamic: auto-generated • independent of collection location • proxies act like symlinks or NFS mounts • forms a logical namespace of distributed content • From client perspective: all collections look alike*

  5. Exposing the hierarchy • Configuration service • reports hierarchy, collection statuses • Query results • Aggregates with hidden contents act like leaf collections • Result sets?

  6. Identifiers • Now: • collection:local-id • New: • collection/path/...:local-id • collection/path/...:hidden/path/...:local-id • Someday: GUID prefixes • Implementation note: identifier renamers must be uniformly employed

  7. Collection monitoring • New collection-level service: status • possible answers: up, down, partially up • aggregates return info on sub-collections • bucket99 driver: executes test query • Separate thread polls collections, maintains global map

  8. Ingest/management services • Add item • 3 ADL views plus other, arbitrary views • returns collection-local identifier • Replace view • Delete item • Create collection • name, static metadata • Replace collection static metadata • Delete collection • New: these are “lc” (local collection) specific

  9. generic webclient (JSP) ingest tools (JSP) middleware (Java/HTTP/RMI) middleware webapp directory (local file system) Ingest big picture

  10. Vocabulary refinements • Definition: a thesaurus C is a refinement of a thesaurus P if it has the following properties: • every preferred term in C is (transitively) narrower than one or more preferred terms in P • every non-preferred term in C has a use-instead relationship to one or more preferred terms in P and/or in C • Thesaurus protocol must support cross-thesaurus references • Hierarchical bucket is still associated with a root thesaurus • refinements support per-collection customization

  11. adl:types • Base: DCMI Type Vocabulary • http://dublincore.org/documents/dcmi-type-vocabulary/ • MIL extension: • DCMI:Dataset • digital elevation model • DCMI:Image • map • remote-sensing imagery • satellite image • aerial photograph

  12. adl:formats • Offline • digital • physical • Online • text • image • video • audio • application

More Related