Academic Libraries and Virtual Reference

1. Academic Libraries and Virtual Reference User Preferences & Usability of Reference Services Rebecca Croxton, MLIS Anthony S. Chow, Ph.D. July 12, 2011

2. Purpose of Study • Better understand information seeking preferences of library users in an academic setting • Develop an understanding of user perceptions of “usability” of different virtual reference mediums offered by an academic library

3. Research Questions • Which virtual reference mediums do library reference users prefer? (Email, Telephone, Online Chat, Video Conferencing, Text Messaging) • Are there differences in user preferences for virtual reference mediums based on type of question? • Are there differences in user preferences for virtual reference mediums based on type of user and type of question? • Which virtual reference mediums do library users find most usable?

4. Findings in the Literature Findings vary across studies

5. Study Methodology • Campus-wide Survey (936 participants) • Usability Study (31 university student participants) • Focus Group of Reference Library Staff (13 participants) • Interview with Head of Reference at Academic Library

6. Participants – Campus Wide Survey Campus-Wide Survey - 936 respondents • Sent to a randomly selected sample of university students (N=2,552 / 14.2% response rate) • All permanent faculty and staff (N=3,840 / 14.3% response rate)

7. Instrumentation – Campus Wide Survey • 11-item online survey • Questions: • Prior experience with and knowledge of reference services • Rank order virtual reference seeking preferences for answering a procedural based, research question and a quick, factual question • Choose one reference service for procedural, research question and quick, factual question – this time including face-to-face reference as option

8. Key Findings – Campus-Wide Survey Fifty-four percent of respondents had prior experience with at least one reference medium.

9. Key Findings - Campus-Wide SurveyResearch Questions Email is most popular choice for research questions.

10. Key Findings – Campus-Wide SurveyResearch Questions If face-to-face is an option, survey respondents prefer face-to-face for research questions. • Faculty prefer email over face-to-face reference. • After face-to-face, undergraduates nearly equally prefer email and online chat.

11. Key Findings – Campus-Wide SurveyQuick, Factual Questions Telephone is most popular for quick, factual questions.

12. Key Findings – Campus-Wide SurveyQuick, Factual Questions With face-to-face as an option, telephone is most preferred for factual questions. • Online chat and telephone were nearly equally preferred for undergraduates. • Faculty preferred email over all other choices.

13. Participants – Usability Study Usability Study - 31 participants • 12 Randomly Selected Undergraduate Students • 19 Randomly Selected Graduate Students Of these participants: • 15 Students Tested Reference Mediums at University A • 16 Students Tested Reference Mediums at University B Universities Utilized for Study: • Two mid-sized, doctoral granting state universities in the Southeast

14. Instrumentation – Usability Study • Participants given written instructions via email to contact a designated university library and ask a pre-assigned procedural, research based question and a quick factual question via: • Email • Telephone • Online Chat • Skype Video* • Text Messaging *University B did not offer Skype Video Reference

15. Instrumentation – Usability Study (Cont.) • Participants completed online surveys after testing each medium and at completion of study • Rank order virtual reference seeking preferences for answering a procedural based, research question and a quick, factual question • Choose one reference service for procedural, research question and quick, factual question – this time including face-to-face reference as option • Rate usability of virtual reference mediums using measures defined by ISO 9241-11:1998 of Effectiveness, Efficiency, and Satisfaction

16. Key Findings – Usability StudyResearch Questions Online chat is most preferred for research questions.

17. Key Findings – Usability StudyResearch Questions With face-to-face as an option, online chat is still most preferred for research questions

18. Key Findings – Usability StudyQuick, Factual Questions Online chat is most preferred for quick, factual questions.

19. Key Findings – Usability StudyQuick, Factual Questions With face-to-face as an option, online chat is still most preferred for quick, factual questions.

20. Key Findings – Research Questions • Campus-Wide Survey • Email is most preferred virtual reference medium • When face-to-face reference is an option, users prefer face-to-face over all other choices • Exception – Faculty preferred email over F2F • Usability Study (students only) • Online chat is preferred reference medium over all other choices (including face-to-face)

21. Key Findings – Quick, Factual Questions • Campus-Wide Survey • Telephone is most preferred virtual reference medium • When face-to-face reference is an option, users prefer telephone over all other choices • Exceptions: • Faculty prefer email over face-to-face reference • Students equally prefer telephone and online chat • Usability Study (students only) • Online chat is preferred reference medium over all other choices (including face-to-face)

22. Summary of Preferences

23. Usability - Defined “Usability is a quality attribute relating to how easy something is to use. … it refers to how quickly people can learn to use something, how efficient they are while using it, how memorable it is, how error-prone it is, and how much users like using it. If people can’t or won’t use a feature, it might as well not exist.” J. Nielsen & H. Loranger, Prioritizing Web Usability. (Berkeley, CA: New Riders, 2006).

24. Usability - Operationalized Per ISO 9241-11:1998, usability is measured using three factors: • Efficiency • Effectiveness • Satisfaction

25. Usability - Effectiveness • Effectiveness (Jordan, 2001) • Task Completion: “Were you able to successfully use [reference medium] to ask your questions and have them answered?” • Quality of Output: “How satisfied were you with the overall quality of the [reference medium] responses and the extent to which they answered your questions?” P.W. Jordan, An Introduction to Usability. (Philadelphia, PA: Taylor & Francis, 2001).

26. Usability - Efficiency • Efficiency (Jordan, 2001) • Deviations from Critical Path: “How satisfied are you that you were able to use [reference medium] without any problems?” • Error Rate: “How satisfied were you that you accomplished your goal with very few or no errors?” • Time-on-Task:“How satisfied are you that you accomplished your goal with minimal time wasted?” • Mental Effort: “How satisfied are you that you accomplished your goal with minimal mental effort?”

27. Usability - Satisfaction • Satisfaction (Jordan, 2001) • “Overall, how satisfied were you with the general usability of using [reference medium] to ask reference questions?”

28. Usability – Overall Results • Overall, research subjects found “online chat” to the be “most usable” virtual reference medium.

29. Instrumentation – Focus Group & Interview Focus Group • Reference Library Staff (n=13) at University A • 7 Questions re: thoughts, preferences, and future trends surrounding virtual reference • Sample Question: • “Based on your experience, which of the reference mediums (email, telephone, online chat, Skype video, or text messaging) is most popular with library patrons? Interview with Reference Dept Manager • 18 open-ended questions re: thoughts & opinions re: virtual reference services

30. Key Findings – Focus Group • Online chat is “without a doubt” the most popular reference medium choice for library patrons both for quick factual questions and research questions. • If individuals already have established relationships with particular librarians (such as a subject librarian or embedded in a course), email seems to be more popular. • Usage of online chat and email fluctuates depending on the time of day and time frame in which a library patron needs a response. • The needs of the users drive the transaction.

31. Key Findings – Interview with Reference Manager In the past 3 ½ years… • Telephone inquiries have gone down dramatically. • Online Chat inquiries have increased substantially (330% increase since 2005). • Email inquiries at reference desk have decreased, but emails with departmental liaisons have increased.

32. Answering the Research Questions • Which virtual reference mediums do library reference users prefer? • Results suggest library user preferences are dependent upon type of question and type of user. Overall for virtual reference, findings suggest users prefer Email for research questions and Telephone for quick, factual questions. • Are there differences in user preferences for virtual reference mediums based on type of question? • Yes! Preferences differ significantly depending upon type of question. • Are there differences in user preferences for virtual reference mediums based on type of user and type of question? • Yes! Preferences differ significantly depending upon type of question and type of user. • Which reference mediums do library users find most usable? • Study findings suggest users find “Online Chat” to be the most usable virtual reference medium.

33. Explaining the Results • Poole’s Principle of Least Effort. • Information seekers will attempt to minimize the overall work associated with seeking information and will seek the most convenient source to meet their information needs at any given time. • A person’s role may determine the depth of an information need, thereby influencing their choice of reference medium. • Familiarity, convenience, and ease-of-use appear to be primary factors behind information seeking preferences. Herbert Poole, Theories of the Middle Range (Norwood, NJ: Ablex, 1985).

34. Using Factor Analyses to Help Interpret the Results • Factor Analysis of Campus Wide Survey participants’ qualitative comments suggest: • Email – primary factors behind preferences focused on: • Convenience • Familiarity • Ease of Use • Written Record that was precise and in-depth • Online Chat, Telephone, and Text Messaging – primary factors behind preferences focused on • Speed and Quickness of Response • Convenience • Familiarity • Ease of Use

35. What Next? • Replicate study with other populations to see if findings remain consistent. • Watch the trends for video conferencing and text messaging reference, particularly among younger populations. • Millennial generation will start becoming faculty and staff at universities.

36. References Joel Cummings, Lara Cummings, and Linda Frederiksen, "User Preferences in Reference Services: Virtual Reference and Academic Libraries," Portal 7, no. 1 (January 2007). Corey M. Johnson, "Online Chat Reference," Reference & User Services Quarterly 43, no. 3 (Spring 2004): 241. J. Nielsen & H. Loranger, Prioritizing Web Usability. (Berkeley, CA: New Riders, 2006). P.W. Jordan, An Introduction to Usability. (Philadelphia, PA: Taylor & Francis, 2001). Herbert Poole, Theories of the Middle Range (Norwood, NJ: Ablex, 1985).