Crunching Numbers: OPAC Log Analysis of WebVoyage

1. EndUser 2007 Session 34 Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries

2. Overview • Why analyze your log files? • How to do it • What we found • The changes we made • What do the latest logs say? • Improvements needed

3. Why Analyze? • To see how your users search when you’re not watching • To resolve internal disagreements over default searches, limits, etc. • To see whether changes to WebVoyage really improved search results • As a counterpoint to task-based user testing

4. C.S. Lewis Lewis, C. S. (Clive Staples) LION WITCH? LION, WITCH? LION, WITCH, AND WARDROBE? Lewis, C. S. (Clive Staples)

6. Issues to Think About • Does Voyager capture the data you need? • Privacy concerns • Does your network organize data the way you need? • Staff vs. public IP addresses • Do you want all searches or a sample?

7. How To • Read the documentation • Technical Manual, Chapter 15, Popacjob • Begin logging your data • Extract data into Access database • Clean up data as needed • Run queries • Scratch head and contact Tech Support

9. 5-MAR-07 WebOpac 20061016102711 Title keyword (TKEY New) AND (TKEY York) AND (TKEY Times) Y West Campus Library K Y N 7 1 W 999.999.99.999 AMDB20020820112825 N

10. Data Fields

11. Data Fields (cont.)

12. Data Fields (cont.)

14. SQL for Count of Search Type SELECT [Spring 2007 OPAC log].Search_type, Count([Spring 2007 OPAC log].Search_type) AS CountOfSearch_type FROM [Spring 2007 OPAC log] WHERE ((([Spring 2007 OPAC log].Client_type)="W") AND (([Spring 2007 OPAC log].Search_tab)="1") AND (([Spring 2007 OPAC log].Hyperlink)=“N") AND (([Spring 2007 OPAC log].Client_ip) Not Like "128.194.8[4-7].*" And ([Spring 2007 OPAC log].Client_ip) Not Like "165.91.39.*")) GROUP BY [Spring 2007 OPAC log].Search_type ORDER BY [Spring 2007 OPAC log].Search_type;

15. Author Browse – 334 Author headings – 943 Author keyword – 1587 Builder – 6 Call Number Browse – 777 Command – 16 Documents Call Number – 15 Expert keyword (rel) – 5 Expert keyword (rev) – 2378 Journal title keyword – 793 Journal title – 1375 Keyword – 82 Keyword (Relevance) -- 6179 Keyword Search – 1574 LC Call Number browse -- 14 Locally Assigned Call#-- 2 Simple Search -- 58 Subject Browse -- 851 Subject headings -- 128 Subject Hds keyword -- 251 Title keyword -- 2045 Title Redirect -- 615 Title starts with – 1677 Results

16. June 2006 (Voyager 5)

17. September 2006 (Voyager 5) • Changed interface • Defaults • Kept Tab at Simple Search • Changed Search to Keyword (CMD* with javascript) • Changed result sort to by relevance

18. Fall 2006 • Preparing to upgrade to Voyager 6.1 • New keyword searches with ^ to automatically AND words together • Some people unhappy with recent changes • Default search • Search results sort order • Decided to look at the data

19. Decisions upgrading to V6 • Basic data • Where are our searchers • What search tab are they using • How are they searching • Default search • Order of title searches • Simple limits

20. Where Are Our Searchers?

21. What Search Tab Used?

22. Default Search: Discussion • Title search (TALL) • What we traditionally had used • Reference’s preference • General keyword search (new GKEY^*) • What users are used to in a Google world • More forgiving search

25. Default Search: Decision • General keyword search (new GKEY^*) • User preference • Fewer No Hit results

26. First Title Search: Discussion • Left anchored title (TALL) • Preferred by Reference • Title keyword (new TKEY^*) • More forgiving

27. Title Search (TALL): Problems

28. Title Search: Decision • Title keyword • Left-anchored title had too many problems

29. Simple Limits • Several additional location limits requested • Concern that too many would be confusing

30. Search Limits Used

31. Simple Limits: Decision • Added new limits and will evaluate with more data

33. Analysis of Voyager 6 Logs • Search frequencies • No hit frequencies • Title search problems • Journal title search hits • Search limits

34. Keyword and Subject Searches

35. Author Searches

36. Title Searches

37. No Hit Title Searches: Do We Own Them?

38. No Hit Title Searches: Problems

39. Location Limits Used

40. Comparison of Limits

41. Have Changes Helped? • Search frequency • No hits percentage

42. Search Frequency

43. No Hit Percentages • Some improvement but no major change

45. Improvements: Spelling • Spellchecking • Automatic searching of variant spellings • “&” or “and” • British vs. American spellings • Numbers • Abbreviations • Did you mean? Suggestions based on field

46. Improvements: Help • More granular no hits help • Specific search types • Any search with “conference” or “proceedings” in it • Journal title searches including “vol.”, “no.”, or a number • Searches with more than 4 or 5 words • More granular help for too many hits

47. Improvements: Specific Searches • Keyword searches • Automatic stemming • Ignore punctuation and spacing • Ignore stop words • Title searches • Ignore initial article

48. Journal search results layout • Whether to include the index field in the journal title search results • Same search results but the order of the results is change by the inclusion of the index field • Primarily a problem for single word titles that retrieve more than 1 screen of results – Science, Nature, etc.