After Assessment: Using Results

1. After Assessment: Using Results Sue Crockford-Peters Head of Access Services Yale University Library March 11, 2005 Ivies+ Access Services Symposium Princeton University Pleased to be asked to speak about assessment at Yale… However, assessment isn’t what’s important, it’s what you do with the results So I’m going to talk about how we’ve used results to make improvements in access services at Yale. Pleased to be asked to speak about assessment at Yale… However, assessment isn’t what’s important, it’s what you do with the results So I’m going to talk about how we’ve used results to make improvements in access services at Yale.

2. Assessment Improvements Reader surveys point the way Tactics for leading improvement efforts Examples of improvement initiatives: Shelving time Shelving accuracy Delivery service Retrieval service Purchase requests In the next few minutes I’m going to coverIn the next few minutes I’m going to cover

3. Reader Surveys Point the Way Positive news Staff perceived to be friendly, helpful, knowledgeable No services considered sub-par Set priorities for improvements Focus on areas readers identified 1998, 2002 surveys identified gap between reader expectations for excellence and perceptions of the services we deliver Results published Focus efforts for improvement on services identified in the surveys as being high priority for readers 1998, 2002 surveys identified gap between reader expectations for excellence and perceptions of the services we deliver Results published Focus efforts for improvement on services identified in the surveys as being high priority for readers

4. Tactics for Leading Improvement Efforts Department level Organization level A little later I’ll be giving examples of both kinds of efforts we’ve done at YaleA little later I’ll be giving examples of both kinds of efforts we’ve done at Yale

5. Tactics for Leading Improvement Efforts At the organization level Service Quality Improvement Council (SQIC) Sponsors reader surveys + improvement efforts in response to results Sets priorities for improvements Assists with assessing the outcomes of improvement efforts Develops improvement process skills in staff Works to develop quality improvement process skills in staff throughout the library in order to promote department level actionWorks to develop quality improvement process skills in staff throughout the library in order to promote department level action

6. What do readers want? Prompt, accurate shelving Prompt, reliable delivery Accurate catalog entries Helpful, clear signs and websites Highlights of some points related to access servicesHighlights of some points related to access services

7. Prompt, Accurate Shelving At the department level Access Services Goals Every item shelved within 24 hours after discharge Every item shelved correctly Actions Increase productivity from discharge to shelf Confirm accuracy Sterling Memorial Library (research) Cross Campus Library (intensive use) Sterling Memorial Library (research) Cross Campus Library (intensive use)

8. From Discharge to Shelf Focus on -- pre-shelving work Reorganized the pre-shelving area 25% time savings for sorting books Started checking trucks 100% accurate pre-shelving Actions: 1st - focused on pre-shelving work Reorganized the pre-shelving area minimum 25% time saving achieved for sorting books after discharge Began checking trucks to confirm they are accurately set up by floor and call number sequence prior to being sent for shelvingActions: 1st - focused on pre-shelving work Reorganized the pre-shelving area minimum 25% time saving achieved for sorting books after discharge Began checking trucks to confirm they are accurately set up by floor and call number sequence prior to being sent for shelving

9. From Discharge to Shelf (con’t) Focus on – training new shelvers Achieve 100% accuracy setting up trucks Begin shelving in Cross Campus Monitor accuracy daily Daily follow up with actual errors Actions: 2nd - increased focus on new shelvers don’t shelve until they can set up trucks that are consistently accurate begin in CCL – a small collection 250000with one classification system shelving accuracy monitored daily errors used for training follow up Actions: 2nd - increased focus on new shelvers don’t shelve until they can set up trucks that are consistently accurate begin in CCL – a small collection 250000with one classification system shelving accuracy monitored daily errors used for training follow up

10. From Discharge to Shelf (con’t) Focus on – accuracy Not enough resources to shelf read Often searching for items recently used Began checking each item Give staff feedback about errors Record daily statistics Recently – last 3 years or less Will accuracy checking be cost effective? Maintain weekly statistics Recently – last 3 years or less Will accuracy checking be cost effective? Maintain weekly statistics

11. Accuracy check results 1/03 – 2/05 Volumes checked: 633,006 Number of errors: 13,536 Error rate: 2.1% Time to check: 5048 hours Average time/error: ~22.5 minutes Average time/search: ~30 minutes 1-31-03 to 2-21-05 (in Sterling) Total volumes checked: 633,006 Total errors: 13,536 Total hours: 5048 Average error rate: 2.1% Average hours/error: 0.37 hours (~22.5 minutes) Reminder – estimated time to search for a mis-shelved book – 30 minutes When we analyze the data by work done by students, we found a slightly higher error rate, but a lower checking time1-31-03 to 2-21-05 (in Sterling) Total volumes checked: 633,006 Total errors: 13,536 Total hours: 5048 Average error rate: 2.1% Average hours/error: 0.37 hours (~22.5 minutes) Reminder – estimated time to search for a mis-shelved book – 30 minutes When we analyze the data by work done by students, we found a slightly higher error rate, but a lower checking time

12. Prompt, Accurate Shelving (con’t) At the organization level YUL top strategic priority is collection space management: Puts focus on access Lower stack congestion Rational shelving arrangements Prompt retrieval & delivery Spoke about work being done in the Access Services department We are getting help system-wide with our efforts to improve the browsability and findability of items Strategic planning Main priority is improving access by creating an improved environment for browsing on campus collections CSM program Determine best practices Set priorities for LSF transfers Allocate resources for improved bibliographic records moving and processing volumes into the LSF shift collections to use freed space Improvement support both readers and staff locating materials for use or deliverySpoke about work being done in the Access Services department We are getting help system-wide with our efforts to improve the browsability and findability of items Strategic planning Main priority is improving access by creating an improved environment for browsing on campus collections CSM program Determine best practices Set priorities for LSF transfers Allocate resources for improved bibliographic records moving and processing volumes into the LSF shift collections to use freed space Improvement support both readers and staff locating materials for use or delivery

13. Prompt, Reliable Delivery At the department level Access Services Goals Retrieve “in process” items within 48 hours Notify requesters within 48 hours Actions Improve inter-departmental communication and cooperation Measure results

14. Retrieving “In Process” Items Communication and cooperation Monthly meeting of supervisors Access Services Acquisitions Shelf Preparations Cataloging Preservation Access Services - Frontlog, Backlog Acquisitions - Receiving, Selector Review Shelves, FastCat Shelf Preparations - plating + labeling, periodical binding Cataloging - copy, RUSH and priority items Preservation - monograph binding, collections care, conservation Access Services - Frontlog, Backlog Acquisitions - Receiving, Selector Review Shelves, FastCat Shelf Preparations - plating + labeling, periodical binding Cataloging - copy, RUSH and priority items Preservation - monograph binding, collections care, conservation

15. Retrieving “In Process” Items (con’t) Measure results <> Review statistics Identify problem areas Celebrate successes 94% overall fill rate 49% filled in 10 hours another 26% filled within 24 hours another 8% filled within 48 hours Another 11% filled after 48 hours 100% readers notified within 48 hours

16. Retrieving “In Process” Items (con’t) I know you can’t read this…I know you can’t read this…

17. Prompt, Reliable Delivery (con’t) Eli Express Service Offered at every circulating library Includes deliveries to + from LSF (off campus shelving) Special collections

18. Eli Express Deliveries SQIC sponsored improvement process Created a cross-department team Engaged consultant Examined service as it was then Proposed improvements Identified changes to achieve improvements

19. Eli Express Deliveries (con’t) Improvements included Revised delivery route Increased # of daily deliveries from 1 to 2 Increased # of hold available notices from 1 to 3 Decreased delivery time by one day (37%) Decreased cost/volume from $0.23 to $0.15 Added no additional resources Data collected on all aspects – used to analyze workflowData collected on all aspects – used to analyze workflow

20. Prompt, Reliable Delivery (con’t) Purchase request service (via OPAC webform) SQIC sponsored improvement process Goals Reduce number of RUSH orders (#1 priority) Add choices for pick up locations Add course reserve purchase option Improve routing for searches for missing books Subset of “Selection – Acquisitions” Created a cross-department team Engaged consultant Examined service as it was then Proposed improvements Identified changes to achieve improvements Subset of “Selection – Acquisitions” Created a cross-department team Engaged consultant Examined service as it was then Proposed improvements Identified changes to achieve improvements

21. Requests for Purchase & Delivery Webform improvements included Requester chooses ordering priority Requester chooses pickup location Link to redesigned course reserve purchase request form Each library receives searching messages prior to replacement ordering Just add to the collection. No need to notify me.Notify me when available (usually within 4-8 weeks).Acquire as soon as possible and notify me. Just add to the collection. No need to notify me.Notify me when available (usually within 4-8 weeks).Acquire as soon as possible and notify me.

22. Requests for Purchase & Delivery (con’t) #1 priority assessed 5/03 – 2/04: 1,030 rush orders 5/04 – 2/05: 683 rush orders Requester indicated ASAP 33% fewer Just add to the collection. No need to notify me.Notify me when available (usually within 4-8 weeks).Acquire as soon as possible and notify me. Just add to the collection. No need to notify me.Notify me when available (usually within 4-8 weeks).Acquire as soon as possible and notify me.

23. Closing Thoughts Assessment results help shape priorities for improvement Improvement efforts can be grassroots Not always intuitive which data will be needed Not all helpful data can be gathered Improvement efforts can be fun Results are satisfying for staff and readers

24. Thank you! Please send email to me if you have questions: susan.crockford-peters@yale.edu

25. URLs for More Information Collection Space Management http://www.library.yale.edu/csm Service Quality Improvement Council Initiatives: http://www.library.yale.edu/Administration/SQIC/SQICloc.html

26. URLs for More Information (con’t) Eli Express delivery service http://www.library.yale.edu/circ/eliexprs.html (YUL information webpage) http://www.library.yale.edu/Administration/SQIC/Eli%20Express%20Process%20Improvement%20QI%20Forum%20010604.ppt (Power Point)

27. URLs for More Information (con’t) Functional Redesign Operations Group (FROG) – “Selection to Acquisitions” http://www.library.yale.edu/~bturner/frog/index.html (website) Purchase request webform http://www.library.yale.edu/Internet/bkreq.html