Paula White & Amohia Boulton Aotearoa New Zealand Evaluation Association (anzea) Conference August 2011

1. Towards a kiwi evaluation “No.8 wire” approach Our potential for innovative evaluation in Aotearoa through building relationships and rubrics Paula White & Amohia Boulton Aotearoa New Zealand Evaluation Association (anzea) Conference August 2011

2. Starting places • Our roots, our connections

3. Compass points • Responding to our unique context • On why navigating relationships is so important • Connection between relationships and values • International trends and messages • Participatory tradition in Aotearoa NZ • A No. 8 wire approach to rubric development? • Tensions and opportunities

4. Our unique context • Complex relationships space: • Political, cultural, social, environmental, economic: Treaty partnership and multiculturalism • Evaluation practitioners, commissioners and users: located in communities, services and in government • Evaluation stakeholder interface: between funders, government, NGO and communities • Global shifts: geopolitical power; demography, climate, resource pressures; rapid technological advancement; values

5. Navigating relationships “We have discovered that our own self knowledge and cultural capital informs our choice of evaluation approach and our processes of engagement with stakeholders. Our experiences testify to the idea that evaluation is as much about who we are as evaluators, and where we position ourselves in relation to others in our work, as it is about what we do and how we do it (Greene, 2005).” White P &Boulton A. (2011). Sailing through relationships? On discovering the compass for navigating 21st Century evaluation in the Pacific. New Directions for Evaluation, Forthcoming, Vol 131

6. Why relationships matter • Why are relationships so crucial to the conduct of high quality Māori evaluation? • evaluation relies on the goodwill of participants for access to the site, informants, data • for Māori evaluators whakawhanaungatanga, making connections, is a cultural imperative • the ongoing sustainability of evaluation activity at a community, iwi, hapu or even whānau level relies on our workforce Gifford H &Boulton A. (2007). Conducting Excellent Research with Indigenous Communities: Balancing Commitment to Community and Career. AlterNative, Volume 3, No. 2, Special Supplement 2007

7. Relationships and values • Building good relationships in evaluation is about negotiating our values as… • Commissioners • Evaluators • Participants • Communities • Providers • How do we do this? • Engage in a values ‘Tug o War’ or a process of negotiation?

8. Our evaluation strengths • Aotearoa is leading the discourse on evaluation practices that ensure meaningful participation by indigenous, and other vulnerable or marginalised communities • Underpinned by a strong, pragmatic, action research tradition (Williams, 2007) • Participatory approaches common in Māori and community settings (with philosophical and practical advantages)

9. At the international evaluation frontiers • Putting ‘value’ back in evaluation • Social navigation: organised action focused on directing the course of social change toward a set of valued (but negotiable) goals (Schwandt) • Transformative and culturally responsive evaluation (Mertens) • A challenge: Extreme Genuine Evaluation Makeovers (XGEMs) or a ‘Rubric Revolution’? (Davidson)

10. Starting the values conversation “Over the past few years its become very clear to me that explicitly evaluative approaches are one of the most important, systematic and meaningful ways in which cultural values and worldviews can be built not just into how evaluations are conducted, but right into the heart of evaluative criteria, the evaluative interpretation/sense-making, and the evaluative conclusions[...]” E. Jane Davidson, Genuine Evaluation Blog, April 2010

11. Unpacking rubrics • A useful tool in our “evaluation toolkit” • Being explicit about the judgements that we make in evaluation • They comprise (Judy Oakden, 2011): • evaluative criteria (the criteria we use to judge the quality, or the value or the importance of the service provided) • dimensions of merit (the types of performance we focus on), and • merit determination (the level of performance we observe)

12. Rubric development –a No. 8 wire approach to evaluation? • Complex • Challenges • Tensions

13. Opportunities • Philosophical/political • Social transformation • Negotiating bicultural and/or multicultural approaches • Unpacking power relationships (critical theory)

14. Opportunities • Practical • Stakeholder buy-in • Focuses the evaluation work • Assisting in the evaluation planning process (fieldwork, data collection, report writing) • Supports a “no surprises” approach • Opportunity to get innovative and creative

15. Some takeaways • Relationships are key • Navigating relationships well requires a negotiation of the values of all stakeholders • Rubrics are a tool to judge quality; and may be useful for negotiating values and navigating relationships • Many resources to assist us as evaluators including our own local experts!

16. Contact us Paula White Organisational Performance Team Te Puni Kōkiri whitp@tpk.govt.nz www.tpk.govt.nz Amohia Boulton Whakauae Research for Māori Health and Development amohia.whakauae@xtra.co.nz www.whakauae.co.nz