KNOWLEDGE BASES AND RELATED TOOLS: IMPROVING OPENURL EFFECTIVENESS

1. X X KBART ok ? K ? KNOWLEDGE BASES AND RELATED TOOLS: IMPROVING OPENURL EFFECTIVENESS Jason Price, PhD Claremont Colleges/SCELC KBART Working Group Member ER&L 2009 Conference UCLA

2. Today’s Outline • OpenURL Overview • Measure of success; Positives and negatives • KBART: Reviewing Problems & Seeking Solutions • KBART background, goals, membership • Main problem areas & Solutions • improve holdings data accuracy • Improve application of OpenURL syntax from “sources” • Improve knowledge of OpenURL & its importance & issues • KBART Deliverables

3. OpenURL Overview • The evolution of the OpenURL in reality: • If links fail, patrons will turn to the tool that always works • Three main problems with OpenURL today: • Bad data; Bad formatting; Lack of knowledge; ‘ ‘

4. The Measure of Success • Better access for patrons • Fewer false positives: saying it’s available when its not • Fewer false negatives: saying it’s not available when it is • Best-case scenario: • IF a patron is seeking an item, and her library offers access to it through exactly seven online resources, • THEN the OpenURL resolver returns exactly seven accurate links to the full text • AND the ‘best’ resources appear first

5. Why we do what we do… http://tinyurl.com/59txop

6. The OpenURL resolver window

7. Transport to the target database…

8. …containing the full text

9. OpenURL in Real Life… Dan in Real Life… • The Positives – it gets patrons to content they would not otherwise have found • It’s a great leap forward in library services • It’s fairly straightforward; it’s not incredibly complicated • The Negatives – it doesn’t get patrons to content as effectively as it should • Inaccurate data leads to bad and missing links • Incorrect implementation doesn’t transfer metadata properly • Lack of knowledge of its importance means: • some vendors aren’t using it • many of others aren’t investing in improved source implementation or more accurate & timely data transfer • But first, a bit of history

10. KBART: A History • UKSG 2007 research report by James Culling,“Link Resolvers and the Serials Supply Chain” (at http://www.uksg.org/projects/linkfinal) • Provided ideas on improving usage and accuracy • Recommended follow-up to address some specifics • NISO partnership to broaden reach and include US audience

11. KBART: An Introduction • Knowledge Bases And Related Tools • UKSG and NISO collaborative project • Get better data for everyone – • Those who provide data (publishers, aggregators) • Those who process data (link resolvers, ERMs, etc.) • Those who present data (libraries, consortia) • All for THOSE WHO USE DATA – library patrons • Ensuring timely transfer of accurate data to knowledgebases, ERMs, etc.

12. Who’s in KBART? • Core working group chaired by Peter McCracken (Serials Solutions) and Charlie Rapple (TBI Communications; formerly Ingenta) • Link resolver/ERM suppliers – Ex Libris, Serials Solutions • Publishers – British Medical Journal Group, Taylor & Francis • Subscription agents/aggregators – Credo, EBSCO, Swets • Consortia –California Digital Library, SCELC • Libraries – Claremont, Cornell, Edinburgh, Leicester, Princeton, Pacific Northwest Technical Lab • Monitoring group • More of these plus other related groups e.g. NASIG • Anyone can join monitoring group • sign up for updates: kbart_interest-subscribe@list.niso.org

13. age of data date granularity (day, month, season, year) file format format definitions;shoe-horning vol/issue vs date frequency informal structure accuracy format unclear responsibilities link syntax and granularity incentive accuracy Data & transfer Date coverage ownership duplication of effort title mapping contacts/feedback mechanisms Supply chain Title relations ISSN/ISBN variations abbreviations Licensing Compliance title changes accuracy re-use of ISSN format effect on licensing free content package variations genericism/granularity misrepresentation Problem Overview Knowledge bases

14. KBART: Examining the problems • “OpenURL’s Negatives” • Inaccurate holdings data leads to bad & missing links • Incorrect implementation doesn’t transfer metadata properly • Lack of knowledge means some vendors aren’t using it and the remainder aren’t improving it

15. Inaccurate Data – The problem

16. Inaccurate Data – Impact Listing of ≈ 120,000 articles needed correction (based on estimated ave. 6x/yr & 10 articles/issue)

17. Inaccurate Data – Current responses • REACTIVE – correcting data for individual articles that patrons report as inaccessible • But what about the (large) majority that go unreported • (esp. the false negatives that prove that “Google has lots of content ’not available through the library’”) • PROACTIVE – before we get (or don’t get) complaints • title by title or package by package • extremely labor intensive • An example

18. Correcting Inaccurate Data – the hard way Proactive reconciliation of an ejournal package list • General Process – library, consortium or KB vendor • (Re-)Request updated access list from publisher • Sample publisher list for accuracy • Translate publisher list to match KB list • Number of titles never matches • Perform ISSN match with MS Access • Watch for & integrate title changes, mergers, acquisitions and losses • Watch for publisher-reuse of ISSNs/title combinations • Identify date discrepancies manually (inconsistent formats) • Decide when its ‘good enough’ and go live/distribute new list • Lather, Rinse, Repeat

19. Correcting Inaccurate Data – the hard way 226 titles = 16%

20. Inaccurate Data – The KBART Solutions • Standardize transfer of data within and among supply chain participants • Phase I - Best practices recommendations specifying: • means of data transfer • frequency of updates • File structure • Data elements – Mandatory and Optional • e.g. Start and end date format & granularity

21. KBART: progressive data element recommendations Under consideration (Mandatory or optional?) • Title level information • Issue completeness (includes all articles?) • Article completeness (Includes tables & figs?) • Full text format (html vs pdf) • Embargo period (granular specification) • Moving wall (a la Nature/Palgrave) • Genre • Freely accessible content listed separately • Ebook fields

22. If we build it, & they don’t come … • How do we handle incorrect data? • Grading? Policing? Shaming? • Biggest and most difficult problem to solve • Highlight to content providers how important completely accurate data is to their end users • Consider the ‘false positive’: arrrgh, that’s frustrating… • Consider the ‘false negative’: much, much worse: how would you ever know?

23. Incorrect Implementation – the problem printcollections gateways database article citation (SOURCE) publisher/providerholdings data publisherwebsite query (base URL+ metadata string) repository link resolver/knowledge base target (cited)article

24. Incorrect Implementation – an example • A book chapter citation in a database: • Cognitive psychology, new test design, and new test theory: An introduction. Snow, Richard E.; Lohman, David F.; In: Test theory for a new generation of tests. Frederiksen, Norman; Mislevy, Robert J.; Bejar, Isaac I.; Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc, 1993. pp. 1-17. [Chapter]

25. Incorrect Implementation – an example • No self-respecting OpenURL talk… • http://ry6af4uu9w.search.serialssolutions.com/?genre=bookitem&isbn=0805805931&issn=&atitle=Test+theory+for+a+new+generation+of+tests.&volume=&issue=&date=19930101&title=Cognitive+psychology%2c+new+test+design%2c+and+new+test+theory%3a+An+introduction.&aulast=Snow%2c+Richar&spage=1&pages=1-17&sid=XXXX:PsycINFO&pid=%3Cui%3E1992-98936-001%3C/ui%3E&%3Cdate%3E19930101%3C/date%3E&%3Cdb%3EPsycINFO%3C/db%3E

26. Incorrect Implementations – the Example made worse • http://ry6af4uu9w.search.serialssolutions.com/?genre=article&isbn=0805805931&issn=&atitle=Test+theory+for+a+new+generation+of+tests.&volume=&issue=&date=19930101&title=Cognitive+psychology%2c+new+test+design%2c+and+new+test+theory%3a+An+introduction.&aulast=Snow%2c+Richar&spage=1&pages=1-17&sid=XXXX:PsycINFO&pid=%3Cui%3E1992-98936-001%3C/ui%3E&%3Cdate%3E19930101%3C/date%3E&%3Cdb%3EPsycINFO%3C/db%3E • Genre: Openurl 0.1 vs 1.0

27. Incorrect Implementations – the impact • Study at Claremont • 5 Results in each of 5 genres from each of 5 databases • Journal Articles, Books, Chapters, Newspaper arts, [Dissertations] • Measure success rate, cause of each failure • Preliminary analysis shows: • Journal articles have significantly lower failure rate • Source URL formation as major cause of failure • Relative consistency within a database/genre combination

28. Solving the Problems: Lack of Knowledge • Some content providers simply aren’t aware of what OpenURL does and why it benefits them • Education & advocacy • Follow recommendations of Culling/SIS report; provide useful information to those content providers • How to implement correctly • Offer contacts for those needing assistance • The remainder may not recognize the value of and their role in improving OpenURL effectiveness

29. Solving the Problem: Lack of knowledge • Help content providers determine what is working, and what isn’t • Cornell project to focus on source OpenURLs • Identify correct and incorrect implementations • Give opportunity for vendors to grade selves • Offer more & better examples of why open OpenURL matters • Quiet challenge (ok, at least out loud) to ER community: produce and distribute studies of the effect of OpenUrl (& poor implementations) on usage • There is one underway at Claremont, we need many more • So many possibilities, so little time (shortsighted? –we’re treating the symptoms, not curing the disease)

30. Summary: KBART Deliverables • Create a report that provides general guidance on problematic issues • Data problems • Incorrect implementation • Limited knowledge • Offer best practices guidelines for how to effectively transfer accurate data among parties • Provide better understanding of supply chain

31. Challenges • Figuring out how to deal with data accuracy questions • Ensuring uptake among smaller or less-committed content providers • Providing ongoing support for new participants

32. Thanks! • http://www.uksg.org/kbart • http://www.niso.org/workrooms/kbart • Peter McCracken (NISO co-chair) • peter@serialssolutions.com • Co-founder & Director for Research, Serials Solutions • Charlie Rapple (UKSG co-chair) • charlie.rapple@tbicommunications.com • Head of Marketing Development, TBI Communications • Jason Price (Working group member) • jason.price@libraries.claremont.edu • Head of Collections @ Claremont; SCELC ejournal package analyst