Measuring Your Impact: Using Evaluation for Library Advocacy GAIN SPRING MEETING, MERCER UNIVERSITY, MACON, GA April 24, 2014
Instructor Andrew Youngkin NN/LM SEA Evaluation and Technology Coordinator firstname.lastname@example.org
Workshop objectives By the end of the workshop you will: • Understand your library’s value in terms of the mission of the larger organization • Describe some tools used to assess your library, its users and stakeholders • Identify the tools and methods used for data collection and analysis • Design an evaluation plan for a service in your library • Communicate evaluation results
What this workshop is not… • Custom-tailored • A quick fix • One size fits all
“What gets measured, gets done.“ Proving to your administration that money spent in your library is going to pay student achievement dividends is well worth the time and effort http://www.doug-johnson.com/dougwri/what-gets-measured-gets-done.html
Why demonstrate value? • To show the impact of your library on the larger organization’s mission and goals • To show accountability for your resources; your library’s contribution to the bottom line • To demonstrate that your library provides value-added services • To use as an advocacy and marketing tool
How & what to evaluate • What you decide to evaluate depends on: • What you need or want to know • What your users feel is important • What certain stakeholders want to have evaluated • Use “smallball” evaluation • Base hits or home runs?
Library service is value-added • Cost savings • Institutional & Personal • Time savings • Travel, discovery, skills acquisition, opportunity for other uses of time • Access • Opportunity to use/apply information • Impact on learning, research
The bottom line • You are either generating revenue, or supporting those who do OR … • You are helping to control operating expenses or supporting those who do OR … • You are creating expenses that add recognized value OR … • You are creating expenses that must be controlled or eliminated to reduce overhead Will Welton, PHD, Director, MHA Program, University of Washington
The big question! What is the bottom line and what is your library doing to support it?
What you need is a plan • Know the vision • Look at the environment • Create the plan • Collect and analyze the data • Communicate the value
Know what you want to do and why • Align your library’s mission & goals with the organization’s mission & goals • Your library’s value = library’s contribution to achieving organization’s mission & goals
Mission statements • dedicated to cultivating the inherent gifts, skills and talents of children in Grades 6 through 12 so that they are prepared for success in college and life. • to provide resources that support the curriculum, help our students to develop habits to become lifelong learners and self-actualized citizens, and promote understanding and appreciation of our Marianist heritage.
Mission statements • ...foster in our students academic, physical and creative fulfillment, together with strength of character, while helping them become productive members of our school community. • …support and … foster the active and creative pursuit of academic and artistic achievement through the effective use of information and the encouragement of curiosity and scholarly inquiry.
HAIKU creative fulfillment,academic and physical;productive community ...foster in our students academic, physical and creative fulfillment, together with strength of character, while helping them become productive members of our school community effective information useencourages curiosity andscholarly achievement …support and…foster the active and creative pursuit of academic and artistic achievement through the effective use of information and the encouragement of curiosity and scholarly inquiry
Write a haiku • Write your mission statement or another message • Turn that into a haiku • Standard haiku is 5/7/5 syllables, but English haikus are more flexible because of our syllable structure • Share your haiku
Made to stick • SUCCESs • Simple – core, compact • Unexpected – gets attention • Concrete – specific, not abstract • Credible – testable • Emotional – more than self interest • Stories – stimulate, inspire (From: Made to Stick. Heath, Chip. New York: Random House, 2007)
Why look at the environment? • Understand needs, desires and problems in context • Validate assumptions about your contributions and services • Provide a baseline for future evaluation • Help to develop the blueprint to plan and evaluate your contributions and services
Components of environment • The organization • Your library • Clients/users • Stakeholders • The community
Assessing the environment • User/stakeholder input • Surveys, focus groups, interviews • Unsolicited feedback • Conversations • Observation • Assessment by walking around • Know the language of your users • Information/literature • Journal/serial/magazine
Assessing the environment • Library statistics and records • Collection use • Librarians’ work including reference, teaching, etc. • Get out of your library and talk to people • Understand the role of information in their work • Learn how information is acquired, applied • Discover information problems they are trying to solve • SWOT analysis • Understanding strengths, weaknesses, opportunities, threats • Consider how these affect your work, your assessment
Librarians’ top 10 tasks • School Librarians say… • Help students select books • Collaborate with teachers • Read and book talk with classes • Teach research and use of technology to students and teachers • Keep library organized including cataloging, weeding, inventory, late notices, vacuuming, dusting • Study standards to prepare library lessons • Troubleshoot technology issues around the building • Help with extracurricular activities including laminating, school plays, lunch duty, etc • Prepare book, equipment and supply orders • Organize special library events like book fairs, author visits, contests
Librarians’ top 10 tasks • School Principals say… • Help students access information and books • Help faculty access information and books • Share technology expertise with students and teachers • Select “appropriate” materials • Model love for reading • Collaborate with teachers • Provide equipment and technology • Provide leadership with technology • Teach research skills, teach about books and databases • Provide an inviting environment How does your boss see you? Proof that principals value librarians. School Library Journal http://www.slj.com/2012/09/careers/how-does-your-boss-see-you-proof-that-principals-value-librarians/#_
Librarians say Principles say
SWOT analysis Strengths • Internal • Positive statements about your library Weaknesses • Internal • Statements about what is lacking in your library Opportunities • External • What do stakeholders want you to do that you are not doing? Threats • External • Factors that can adversely impact your library’s goals
The logic model - a tool to… • Help organize your thoughts – both before and during a project/program • Provide a framework for planning and evaluating programs • Clarify intended outcomes • Acts as a communication tool Good source for more information: W.K. Kellogg Foundation Logic Model Development Guide (pdf on your thumb drive; see class bibliography for link)
Pieces of the logic model • Outcomes • Activities • Resource • Outputs
Outcomes • Outcomes - changes in attitude, behavior, skills, knowledge or situation • Short, medium and/or long term • Intended or unintended • Positive as well as negative • S.M.A.R.T.
SMART Outcomes Outcome: Health resources Web pages are available on the library website by the end of the project • Specific: one or more web pages with links to EBM resources are available on the library website • Measurable:done or not; do nurses access resources • Action-oriented: created and posted; nurses access resources • Realistic: It is possible within the timeframe • Timed: “when” is articulated
OUTCOMES Outcome • Long Term: • Patients and their families will have improved health information literacy • Intermediate Term: • Staff will have information for timely clinical decision-making • Short Term: • Health resources Web pages are available on the library website by the end of the project
Activities and resources • Activities • What will you do? • Resources • What you have • Income • Equipment • Collection • What you need • Operating expenses (e.g., personnel, acquisitions, maintenance, etc.) • Funds for new initiatives or services
Outputs • Outputs • How many did you do? • How many attended? • How many were distributed? • How many times was it used?
Putting it all together Resources If we get these resources… Activities/Outputs Conduct these activities and deliver these products Outcomes We will see these outcomes
The model may change over time Resources Activities Outcomes Outputs
Logic model • Outcomes: ST: Health resources Web pages are available on the library website by the end of the project; IT: Staff will have information for timely clinical decision-making
Logic model group activity • Identify outcome(s) • Articulate short/intermediate/long term • List • activities • outputs • resources
The evaluation plan • Builds on the logic model • Evaluates success • Assesses value
Consider the purpose • Who is your audience? This may be different than the beneficiaries of your services • Your users • Administrators • Patients/families • How will the information be used? • Financial savings or justification • Intangible or non-monetary value of program benefits to community • Marketing and advocating for your library
What do you want to know? Have you achieved your stated outcome? • Short Term: • Health resources Web pages are available on the library website by the end of the project • Intermediate Term: • Staff will have information for timely clinical decision-making • Long Term: • Patients and their families will have improved health information literacy
When do you want to know? • Consider objective(s) • Articulate the time frame
Evaluation plan • Outcome (from logic model):Staff will have information for timely clinical decision-making
Example part 1 • Outcome (from logic model):Staff will have information for timely clinical decision-making