Sarah Palacios-Wilhelm Reference & Instruction Librarian Eckles Library Elizabeth Edwards Reference/Technology Libra

1. Two Programs, One Service:Recursive Assessment in the Gelman Library System's IM Reference ServiceAmerican Library Association Annual Conference2008 MERS Virtual Poster Session Sarah Palacios-WilhelmReference & Instruction LibrarianEckles Library Elizabeth EdwardsReference/Technology LibrarianGelman Library E. Edwards & S. Palacios-Wilhelm

2. Two Programs, One Service: 2008 MERS Virtual Poster SessionAbstract This virtual poster session presents the recursive implementation process utilized by the George Washington University’s Gelman Library System in the context of its successful IM reference service. In this presentation, librarians from two campus libraries discuss the advantages of using pilot testing in a recursive cycle to inform testing, implementation, and evaluation of services, software, training, and assessment models. This process has resulted in a climate that encourages innovation and decreases anxiety amongst service providers. Specific examples of recursive implementation are explored, as well as challenges faced throughout the process. The presentation closes with a list of lessons learned that will inform institutions that wish to follow a similar model. E. Edwards & S. Palacios-Wilhelm

3. Two Programs, One Service: 2008 MERS Virtual Poster SessionBackground In 2005, the Gelman Library System pilot tested the delivery of reference services via IM.  This pilot test began at Eckles Library, which serves a primarily residential and freshman population.  Since its inception, this service has expanded to include Gelman Library, which serves the entire University population.During the 2006-2007 school year the IM reference service fielded 1,458 questions, and was nominated for a campus-wide Service Excellence Award.  The success of this service is in no small part a result of the innovative testing and evaluation model that integrates the two campuses in a recursive cycle. E. Edwards & S. Palacios-Wilhelm

4. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe George Washington University • Located in the heart of the Foggy Bottom neighborhood of Washington, D.C. • Comprised of nine schools: • Columbian College of Arts & Sciences • School of Medicine & Health Sciences • Law School • School of Engineering & Applied Science • Graduate School of Education & Human Development • School of Business • Elliott School of International Affairs • School of Public Health & Health Services • College of Professional Studies • High research activity according to Carnegie ratings. • 17,494 students enrolled full-time in Fall 2007. • 7,092 degrees conferred in 2007, including 264 Doctorates. E. Edwards & S. Palacios-Wilhelm

5. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Gelman Library System E. Edwards & S. Palacios-Wilhelm

6. Gelman Library Main library for urban campus 30,000 visitors per week Reference desk staffed by 17 librarians with assistance from student workers IM service established in Fall 2006 Eckles Library Small library serving residential campus 7,000 visitors per week Single service point staffed by 2 librarians, 3 staff, and 7 students IM service piloted in 2005, established in 2005 Two Programs, One Service: 2008 MERS Virtual Poster Session E. Edwards & S. Palacios-Wilhelm

7. Two Programs, One Service: 2008 MERS Virtual Poster SessionModels of Implementation The “Normal” Implementation Process The Recursive Implementation Process E. Edwards & S. Palacios-Wilhelm

8. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe "Normal" Implementation Process E. Edwards & S. Palacios-Wilhelm

9. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe "Normal" Implementation Process • Each phase has a defined beginning and end. • Each phase occurs in order. • Pilot programs lead directly to implementation. • Test groups are organized as necessary, but do not persist after the implementation process is complete. E. Edwards & S. Palacios-Wilhelm

10. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe "Normal" Implementation Process: An Example • A problem is identified: patrons are eating in the library, which is in violation of library policy. • A solution is proposed: create a space where patrons can eat without violating library policy. • Pilot testing: a student lounge is opened, and patrons attempting to bring food into the library are directed to this new space. • Implementation: the student lounge is used for a special event, drawing attention to the new space. • Evaluation: the problem of patrons eating in the library has not decreased. A new solution must be devised. E. Edwards & S. Palacios-Wilhelm

11. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Implementation Process  E. Edwards & S. Palacios-Wilhelm

12. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Implementation Process • In the recursive implementation process, testing, implementation, evaluation, and reflection are cyclical, with each step returning to the pilot setting. • Problems can be identified at any location, and solutions are proposed based on previous experience, understanding of patron needs, and possible resources. • Pilot testing occurs with a selected population, informed by feedback from ongoing brainstorming and evaluation from both locations. • Based on success in the pilot phase, implementation expands to include both locations, with the next phase of brainstorming or pilot testing beginning at the smaller location. • Evaluation is an ongoing process, incorporating both formal and informal methods of assessment. E. Edwards & S. Palacios-Wilhelm

13. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice Pilot Testing Service Hours Staffing Models Training Assessment Platform Testing E. Edwards & S. Palacios-Wilhelm

14. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice: Pilot Testing • Experience with virtual reference collaboratively staffed with other local academic libraries resulted in hesitation about new forms of electronic reference. • Librarians at Eckles, in an attempt to provide enhanced service for their small, tech-savvy population, recognized the potential of IM for service delivery and experimented with IM reference with limited hours. • Following the positive reception at Eckles, the IM service expanded to Gelman with limited hours, with ongoing evaluation occurring at both locations. E. Edwards & S. Palacios-Wilhelm

15. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice: Service Hours • After the initial pilot, Eckles expanded service hours. • Eckles' statistics revealed peak hours, resulting in an expansion of Gelman's service hours. • Demand for Eckles’ service decreased as a result of Gelman's expansion. • Statistics allow both campuses to anticipate service needs and inform discussions of staffing models. E. Edwards & S. Palacios-Wilhelm

16. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice: Staffing Models • Eckles' service was initially staffed by one librarian, with additional support from student workers. • Gelman's service was initially staffed by librarians from individual desks. • Gelman’s distributed model proved problematic, so service moved to librarian staffing at the reference desk, with assistance from student workers. • Eckles' service moved to triage staffing model using student workers, with librarians available for questions. • Gelman is exploring new staffing models in the context of moving to a single service point and incorporating new departments into the IM reference service. E. Edwards & S. Palacios-Wilhelm

17. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice: Training • Technology-adept student workers at Eckles complete reference service training modules developed for new Gelman librarians. • Gelman librarians receive training on new technologies first piloted at Eckles. • The exchange of training modules draws on the expertise and improves the skills of service providers at both campuses. E. Edwards & S. Palacios-Wilhelm

18. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice: Assessment • Usage statistics from both locations are used to inform how the services are tested, implemented, and expanded. • Question-type statistics from both locations are used to inform staffing models and training. • Ongoing qualitative transcript analysis is used to inform service definition, training, and staffing models. • "Secret Shopper" evaluation used in student worker training at Eckles identified larger training needs, which will inform future training strategies that will be used at Gelman. E. Edwards & S. Palacios-Wilhelm

19. Two Programs, One Service: 2008 MERS Virtual Poster SessionThe Recursive Model in Practice: Platform Testing • Eckles adopted Trillian and Pidgin/GAIM. • Gelman adopted Trillian. • Eckles abandoned Pidgin/GAIM for Trillian. • Eckles tested and adopted Trillian Pro. • Gelman adopted Trillian Pro. • Platform testing remains an ongoing project, with Meebo and other widgets under consideration for implementation. E. Edwards & S. Palacios-Wilhelm

20. Two Programs, One Service: 2008 MERS Virtual Poster SessionChallenges of the Recursive Model Scalability Service Providers Patrons Trying new things is always hard E. Edwards & S. Palacios-Wilhelm

21. Two Programs, One Service: 2008 MERS Virtual Poster SessionChallenges of the Recursive Model: Scalability • Organizational challenges differ between locations. • Recursive cycle does not always flow evenly. • The larger the organization or the greater the bureaucracy, the more staff time and energy will be required to encourage and implement innovation. E. Edwards & S. Palacios-Wilhelm

22. Two Programs, One Service: 2008 MERS Virtual Poster SessionChallenges of the Recursive Model: Service Providers • Service providers vary in technology skills, attitudes towards service, and interpretations of service requirements. • Training models vary based on staff needs, which will affect the service provided at each location. • Staffing models directly affect the quality of service provided. • Technology in different contexts may affect providers’ ease of use. • Familiarity with appropriate resources will affect the service provided. E. Edwards & S. Palacios-Wilhelm

23. Two Programs, One Service: 2008 MERS Virtual Poster SessionChallenges of the Recursive Model: Patrons • Even the most representative pilot group will be different than the full population of both sites. • Patrons may be less patient with the new service or changes in service. • The service may not be able to meet the expectations of patrons used to more personalized service in the pilot location. • Patron usage may shift away from the pilot location as services expand, which may result in a smaller population for future testing. E. Edwards & S. Palacios-Wilhelm

24. Two Programs, One Service: 2008 MERS Virtual Poster SessionChallenges of the Recursive Model Trying new things is always hard. E. Edwards & S. Palacios-Wilhelm

25. Two Programs, One Service: 2008 MERS Virtual Poster SessionLessons Learned from the Recursive Model Effective Pilot Groups Identifying Pilot Groups What Not to Do Reflections E. Edwards & S. Palacios-Wilhelm

26. Two Programs, One Service: 2008 MERS Virtual Poster SessionLessons Learned from the Recursive Model Effective Pilot Groups • Create a safer environment for testing, which inspires innovation. • Include both providers and patrons. • Facilitate a two-way conversation. • Provide both positive and negative feedback. • Allow your audience to engage in the testing process. • Work collaboratively: with you rather than for you. E. Edwards & S. Palacios-Wilhelm

27. Two Programs, One Service: 2008 MERS Virtual Poster SessionLessons Learned from the Recursive Model Identifying Pilot Groups • Look for groups with a population that changes regularly: • Freshmen or graduating seniors • Student workers • Groups with a small population: • Special or departmental libraries • Branch libraries • Cohort-based programs E. Edwards & S. Palacios-Wilhelm

28. Two Programs, One Service: 2008 MERS Virtual Poster SessionLessons Learned from the Recursive Model What Not to Do • Don’t use your staff as a pilot group. • Needs are not the same as those of patrons. • Expectations are not the same as those of patrons. • Feedback will reflect service provider concerns and biases rather than genuine patron experience. • Don’t forget about staff input. • While not representative of patron experience, the service provider’s perspective is critical to any new service. E. Edwards & S. Palacios-Wilhelm

29. Two Programs, One Service: 2008 MERS Virtual Poster SessionLessons Learned from the Recursive Model Reflections • Innovation encourages cycles of evaluation and improvement in other service areas. • Challenges or failures will not derail other library services when done in parallel. • Established framework for comparison and improvement builds an expectation of ongoing evaluation. • Stagnation is prevented because something interesting is always happening. E. Edwards & S. Palacios-Wilhelm

30. Two Programs, One Service: 2008 MERS Virtual Poster SessionLessons Learned from the Recursive Model You Can Do It! The process has been challenging, but has proved to be both successful and rewarding. Creating an environment that facilitates innovation also sparks excitement among service providers, ensuring that library service remains dynamic and relevant for the patron. E. Edwards & S. Palacios-Wilhelm

31. Two Programs, One Service: 2008 MERS Virtual Poster SessionContact Information Sarah Palacios-WilhelmReference & Instruction LibrarianEckles Librarypalacios@gwu.eduAIM/Google Talk/ Yahoo!: sarahpwilhelmElizabeth EdwardsReference/Technology LibrarianGelman Libraryeme@gwu.eduAIM: leepforwardAIM/Google Talk/Yahoo!: GelmanInfo and EcklesInfo E. Edwards & S. Palacios-Wilhelm

32. Two Programs, One Service: 2008 MERS Virtual Poster Session Presentation prepared for: 2008 American Library Association Annual Conference MARS Management of Electronic Resources and Services (MERS) Committee Virtual Poster Session: Evaluation Virtual Reference Services E. Edwards & S. Palacios-Wilhelm