Custom Parameters and Faceting in OpenSearch

1. Custom Parameters and Faceting in OpenSearch ESIP Discovery Cluster Planning Session Eric Rozell, Tetherless World Constellation ESIP Student Fellow – Discovery Cluster

2. Outline • Scope of Discovery • Use Cases and Prior Art • Valid Parameter Values • GSAC Custom Fields • Semantic Parameters • Apache Solr • OpenSearch SRU • Conclusion

3. Scope of Discovery • We usually want simple things from discovery… • Temporal constraints • Spatial constraints • Keyword constraints • Domain-specific discovery • (e.g., depth in physical oceanography) • Federation over custom fields is tricky • Even temporal constraints are tricky!

4. Scope of Discovery • ESIP Discovery covers… • Discovery mechanisms (Casting specs) • Federation capabilities (OpenSearch) • Integrated metadata for data & service descriptions (DCP-3) • What about faceting? • Faceting is important for fine-grained discovery • Need to think about specs for faceting for compatibility reasons

5. Courtesy of Peter Fox

6. Use Cases and Prior Art

7. Valid Parameter Values • Use Case from NSIDC (thanks to Ruth Duerrfor pointers to API) • Granule-level search has custom parameters (e.g., “Pass”, “Frequency”) • Each custom parameter has some expected values (e.g., “Frequency” => 37GHz) • How should these valid parameter values be encoded for clients?

8. Valid Parameter Values • NSIDC uses OpenSearch Query tag to encode valid parameters, e.g., <Query role=“nsidc:validparams”nsidc:Frequency=“36.5 GHz|23.8 GHz|6.9 GHz|89 GHz|18.7 GHz|10.7 GHz” /> • While this works well for the use case (valid parameter values only), it does not work for faceted search.

9. Summary NSIDC Valid Params. Valid Parameters

10. GSAC Custom Fields • Use case from UNAVCO (thanks to Jeff McWhirter for the pointers) • GSAC uses a type system for custom parameters to enable: • Valid parameter constraints • UI coupling to appropriate selector “widgets” • The types provided include: • string • enumeration • boolean • date_range • spatial_bounds • numeric_range

11. GSAC Custom Fields

12. Summary GSAC Custom Fields System NSIDC Valid Params. UI + Parameter Coupling Valid Parameters

13. Semantic Parameters • Define vocabulary for describing parameters including: • Labels • Descriptions • Valid parameter values • Fine-grained semantics • e.g., time collected vs. time processed • Linking parameters to relevant UIs • e.g., geo:box with OpenLayersbbox tool

14. Semantic Parameters in S2S Snapshot from VSTO 3.0 Demo, Part of Semantic eScience Framework Project

15. Summary Semantic Parameters Vocabulary GSAC Custom Fields System NSIDC Valid Params. UI + Parameter Coupling Valid Parameters

16. Semantic Parameters and Faceting • Valid parameter value lists are great for static interfaces • For faceted interfaces, dynamic lists are required • S2S links UIs to parameters and necessary queries • Enables custom, dynamic facets for search

17. Semantic Parameters and Faceting

18. Summary Semantic Parameters Vocabulary GSAC Custom Fields System NSIDC Valid Params. UI + Parameter Coupling Faceted Browsing Valid Parameters

19. Solr • I was going to talk about Solr, but Hook’s talk will follow. • Solr is great for faceting, but… • There should be concern for tying Discovery Cluster specifications to a particular technology (Solr, S2S, SRU) • Are there any broader specs to consider that cover each faceting service

20. Summary Semantics Parameters Vocabulary GSAC Custom Fields System NSIDC Valid Params. Apache Solr UI + Parameter Coupling Faceted Browsing Valid Parameters

21. OpenSearch SRU • Very closely tied to SRU* and CQL** • Allows granular constraints on facets including: • Paging (limit / offset) for facets • Sorting facets • Facets are not provided as HTTP URL parameters (queries encoded in CQL) • Facets are “learned” from query results * Search/Retrieval via URL ** Contextual Query Language

22. Final Notes • Discovery occurs at different levels • Custom parameters and faceting are important at the repository level • Why build two systems (one for federation and one for faceting)? • Not trying to “sell” S2S vocabularies per se, however I’m eager to participate in development of parameter vocabulary.

23. Remaining Questions • What technique for faceting is most used in geoscience community? (assuming Solr) • Should we tie Discovery Cluster specs for to Solr (or S2S or SRU or GSAC for that matter)? • Are there other use cases for “custom parameters”?

24. Questions? • Thanks for listening!

25. S2S Vocabulary