150 likes | 225 Vues
MISSION POSSIBLE Evaluating Stakeholder Consultation Mechanisms. Presentation by: David Taylor and Unnati Vasavada Program Evaluation Directorate Transport Canada Canadian Evaluation Society Conference 2003 Vancouver, B.C. June 2, 2003. Presentation Overview.
E N D
MISSION POSSIBLEEvaluating Stakeholder Consultation Mechanisms Presentation by: David Taylor and Unnati Vasavada Program Evaluation Directorate Transport Canada Canadian Evaluation Society Conference 2003 Vancouver, B.C. June 2, 2003
Presentation Overview 1. Evaluating Consultation: How? 2. Case Study: Safety and Security Group, Transport Canada • Evaluation Approach • Key Study Findings • What Went Well • Challenges • Lessons Learned
How to evaluate consultation?**Privy Council Office. Consulting and Engaging Canadians: Evaluation Guidelines
Case Study Evaluation of consultation mechanisms used by directorates in Transport Canada responsible for safety regulation in aviation, marine, rail and road modes of transportation.
Evaluation Approach • Consultation defined: “Consultation” involves two-way exchanges of ideas and information before a decision is made.
Evaluation Approach Scope of evaluation determined: • formal consultative committees • stakeholder meetings • direct mailings • Internet discussion sites
Evaluation Approach 11 assessment criteria established: • Clarity and Achievement of Purpose • Stakeholders • Selection and Structure of Mechanisms • Decision-Making Process • Roles, Responsibilities and Accountabilities of Decision-Makers and Stakeholders
Evaluation Approach Assessment criteria (cont’d) • Resources • Time Frame • Information • Compatible with Context • Open and Credible to Those Being Consulted • Ongoing Review
Evaluation Approach Stakeholders selected for interview • random sampling • targeted sampling • population Interviews (mainly by telephone) conducted
Key Study Findings Need to: • Improve direct representation of general public in consultations • Provide regular and timely feedback and decision rationale. • Improve length of decision-making process • Maintain face-to-face meetings
Key Study Findings (cont’d) Need to: • Articulate and communicate consultation objectives, participants’ roles and responsibilities • Conduct periodic assessments of consultation exercises • Explore benefits of direct citizen engagement techniques
What Went Well? • Clear definition of consultation • Assessment criteria • Stakeholders appreciated evaluation exercise
Challenges • Low or non response rates • Poorlymaintained stakeholder lists • Varying levels of consultation experience among respondents • Length of time lapsed since participating in consultation • Managing qualitative data
Lessons Learned • Consider carefully who should evaluate: consultants vs. in-house experts • Plan carefully study timing and timelines • Use focused and pragmatic evaluation approach • Use mix of qualitative and quantitative research methods. • Be thorough in attending to administrative details • Consider informal means of evaluating consultation
Discussion Questions? Comments?