Showing it ’ s worth it: Measuring the impact of volunteering

1. Showing it’s worth it: Measuring the impact of volunteering • @IVRtweets • @VolunteeringEng • Nissa Ramsay • 11th February2012

2. Introduction How important is impact measurement to your volunteering? • Very Important • Not important • I’m going to sit on the fence

3. Introduction • Who here thinks they could measure the impact of their volunteering project? • I’d be ace at this • I could blag it • I don’t know what you’re talking about

4. Today… My vision: To enhance the capacity of the student volunteering sector Specifically, • To increase how you value impact measurement • To improve your understanding of impact measurement

5. Lesson 1… • Always consider who is asking the question, to who, what their relationship is and the context Lesson 2… • The best evaluation questions are part of the delivery process: It should not be hard work Lesson 3… • Measuring impact is not an exact science but a balancing act between the ideal world information you would want and the reality of working in your role

6. Why bother? You should do this if: • You have a genuine desire to improve the volunteer experience • You have to as a requirement of your funding • You want to apply for funding • You want to shout about how amazing your work is • Your volunteers are always deserting you • You can’t convince any one to volunteer for you You should think twice about doing this if: • You have no time/money/resources you’re willing to devote • You are only doing this as a requirement of your funding • You have no use/goal for the results

7. Lesson 4… • Measuring impact is also about improving the quality of your work and delivery (not just the end statistics) Lesson 5… • Measuring impact can help focus your project/ organisation on its core purpose (BUT can also take you off course) Lesson 6… • Measuring impact can work on a range of levels: • Project, Thematic area, organisation

8. Where to start 1. Theory of change Articulates how, when and why a change happens as a result of the project/intervention = measure your impact by testing the assumptions behind this = show if you create positive outcomes and why YOU have = Can be on a project/programme or organisation level

9. Case Study: Student Hubs… www.studenthubs.org/xwiki/bin/View/main http://www.youtube.com/watch?v=TLXTnc2YNuk

11. A mission statement/overall aim What? • A mission statement • Describes the overall impact of the programme • The bigger picture – you aren’t measuring this directly Why? • This is your impact: it sets the context as to why you are doing this Student Hubs empowers students to create positive social change, now and in the future

12. Specific aims What? • Precise statements about change • Break down mission in to components and target groups • And/or consider your activities – why are you doing all of them and what do they amount to Why? • This will help identify target groups and the core components of your work/ impact/beneficiaries Student Hubs will inspire students to take social action

13. Outcome indicators What? • Statements of change that we can measure, link to specific aim • Achievable and a change measure (perceptions/feelings) Students feel more motivated to be socially active Output indicators What? • Activities and numbers/targets involved Why? • This will help decide who to involve in the evaluation, why and what questions you will ask them 500 students involved have taken on new volunteer roles

14. The connections Vision Aim 2 Aim 1 Aim 3 Activities = Outputs Change = Outcomes

15. If your aim is: To provide high quality volunteering opportunities • Volunteers’ views on support, management and training • Their volunteer manager (approach / availability etc) • Perceptions of bureaucracy (e.g. CRB, interviews) • Importance of payment of out-of-pocket expenses • Recognition received • Workload and level of responsibility • Whether they see their involvement as meaningful • What they see as the impact of their volunteering • Whether they perceive their help as being needed • Whether they perceive any barriers limiting this

16. Your turn… Group discussion: Choose either • A project one of you are running • Student Hubs • Identify • Specific aims: To increase awareness/perception… • Outcome indicators: Volunteers have increased skills • Output indicators: 25 volunteers have been trained in…

17. Lesson 7… • Measuring impact has to start with understanding what you are trying to achieve and how

18. Methodologies and tools • Surveys • Online (e.g. Surveymonkey) • Access to a sample group (e.g. email addresses for online) • Response rate and representativeness (using incentives) • Focus groups • Do not have managers in the same room! • In-depth interviews • Informal approaches (e.g. walking interviews) • Observation • Volunteer testimonials and case studies • Life history interviewing • Mapping exercises

19. Involving volunteers in the process • Peer research and volunteer involvement in the process • Accessing new skills and experiences • Ability of volunteers to form different relationships with respondents, as peers • Quality of data vs. professional development of volunteers • Project guidance and steering groups • Volunteer / user perspective (e.g. the development of research tools)

20. Improving the quality of research • Gain organisational buy-in and support • Ask only what you need to know • Avoid leading questions • Always pilot surveys • Don’t cherry-pick respondents but pick randomly • Opportunities for tracking / longitudinal research

21. Using toolkits • The Volunteer Impact Assessment Toolkit • Hard copy and website • Assesses impact on volunteers, the organisation, service users and wider community • Guidance on tools and methods • £29.99 plus postage

22. Limitations, challenges and pitfalls • Self-reported nature of satisfaction • Unavoidable to a certain degree • Can attempt to triangulate the results (e.g. speak to staff, clients) • Ensuring a good response rate • Can the results speak on behalf of all volunteers if changes are to be made? • Working with volunteers with specific resource needs • e.g. disability, low confidence, illiteracy, English as a second language • Cost and time implications but everyone needs to have the opportunity to participate

23. Taking it forwards: options

24. Communication and dissemination • Accessibility of findings • Full report vs. summary • Plain English and awareness of audience • Warts and all (honesty and accuracy) • Will increase trust in findings • Feedback to the respondents and the volunteer body • Recommendations / changes need to be communicated as well as reasons for not changing something • Review of changes / developments

25. Conclusion • How important is impact measurement? • Very Important • Not important • I’m going to sit on the fence • Who here thinks they could measure the impact of volunteering within a project? • I’d be ace at this • I could blag it • I’m not really sure

26. The results • At the start of my session • 92% felt impact measurement was ‘very important’ • 0% felt it was ‘not important’ • 8% felt they were ‘going to sit on the fence’ • At the end of my session • 96% felt impact measurement was ‘very important’ • 0% felt it was ‘not important’ • 4% felt they were ‘going to sit on the fence’

27. Conclusion At the start of my session: • 8% of you thought you’d be ace at measuring impact • 45% of you thought you could blag measuring impact • 28% of you ‘don’t know what I’m talking about’ At the end of my session: • 10% of you thought you’d be ace at measuring impact • 73% of you thought you could blag measuring impact • 2% of you ‘don’t know what I’m talking about’

28. The results As a result of me running this session today: 28% more people can blag impact measurement

29. Thank you Nissa.ramsay@ivr.org.uk 0207 520 8900 www.ivr.org.uk