Download
1 / 29
310 likes | 320 Vues
Selecting performance indicators and setting targets. Joseph Naimoli The World Bank October 2008 Presented by Kate Gilroy Institute for International Programs, Johns Hopkins University. Health Results-Based Innovation Trust Fund (HRBF). Acknowledgements.
E N D
Selecting performance indicators and setting targets Joseph Naimoli The World Bank October 2008 Presented by Kate Gilroy Institute for International Programs, Johns Hopkins University Health Results-Based Innovation Trust Fund (HRBF)
Acknowledgements • Jennifer Bryce, Institute for International Programs, JHSPH • Susan Stout, Independent Consultant • Rena Eichler, Broad Branch Associates • Target Setting Guide (January 2004), Audit Commission, U.K. Government
Section 1: Results Management
Results management The ‘art’ of results management is defining and achieving outcomes that are meaningful to both provider and client/consumer, are measurable in a credible way, and are used in decision making. --Susan Stout
Outcomes of interest for RBF? Common evaluation framework for the scale-up for the health MDGs Results Process Training & Capacity building Procurement and supply Guidelines IEC Community mobilization Inputs Funding Planning & policies Harmonization & efficiency Outputs Health services delivery Quality Behavioural Interventions & knowledge Outcomes Service utilization and intervention coverage Behavior change Impact Mortality Morbidity Nutrition Reduced inequity Those addressing maternal, newborn and child mortality and nutrition (MDGs 4, 5, and 1b) Source: A common evaluation framework for the scale up to the health MDGs. Institute for International Programs, The Johns Hopkins University.
A step-wise approach to achieving effective coverage Source: A common evaluation framework for the scale up to the health MDGs. Institute for International Programs, The Johns Hopkins University.
Performance Indicators Section 2:
Reflect project objectives and priorities Are “true” (valid) measures of progress toward achieving MDGs 4, 5 and 1b, i.e.: Accurately reflect changes in service quality, utilization, intervention coverage, or behavior change Are feasible for use in low-income settings Can be repeated at frequent intervals as triggers for releasing incentive payments Can be influenced by recipients’ actions (attributable) Can be replicated and confirmed by others (reliable) Performance indicators for RBF Measurements that:
Other considerations for RBF indicators • Feasibility of measurement • Country’s existing HMIS • Availability of data at different levels, i.e., region, district, facility, community • Straightforward • Can be easily understood by key actors and stakeholders • Stakeholder consensus on importance • Iterative: start with a few key indicators…..
Performance indicators for RBF • “Measure aspects of service delivery that are important, rather than seeking to make things that are easily measured important” • --Audit commission, U.K. Government
Most reflect years of research and “learning by doing” Standards for definition and measurement are available Allows comparisons across RBF projects and with national and international results Already being collected in DHS/MICS and other country-level surveys Considering the use of standard MNCH consensus indicators makes sense for RBF
What are the standard indicators for MNC outcomes? • 2008 Countdown indicators for proven high-impact interventions addressing maternal, newborn and child mortality and nutrition • Definitions available at www.countdown2015mnch.organd at the workshop An ongoing technical review and field testing process to improve the measurement and reporting of Countdown coverage indicators is being established with support from the Gates Foundation.
How many indicators? No perfect answer • Too many can be counter-productive: difficult to communicate and manage, and can induce “indicator overload and fatigue” • Too few may focus attention and new resources on selected indicators at the expense of other important areas Your task: To select a few (3-5) key indicators of utilization, coverage, or behavior change that are directly related to the aims of your RBF project.
TARGETS Section 3:
Targets: What are they? • Commitments, expressed in quantitative terms, to achieve a certain level and/or quality of service within a well-defined time period
Targets: Examples • Absolute • Increase the total number of children under 1 immunized from 800 (baseline) to 1000 • Proportional or percentage achievement • Option 1: Increase immunization coverage of children under 1 by 25 percentage points (from 50% to 75%) • Option 2: Increase immunization coverage of children under 1 by 50 percent (from 50% to 75%) • Additional service provided • $5 US for each additional child vaccinated
Targets: Absolute or relative? • Absolute: • Increase immunization coverage of children under 1 to 80% at all health facilities • Relative to providers’ existing baseline: • Increase immunization coverage of children under 1 by 30 percent in each facility
Targets: Absolute or relative? Absolute: Increase immunization coverage of children under 1 to 80% at all health facilities Relative to providers’ existing baseline: Increase immunization coverage of children under 1 by 15 percentage points in each facility
Targets: Guiding Principles • Use targets to encourage improved performance and motivate people when there is a probability of success • Targets should be realistic but challenging • Too easy complacency • Too difficult discouragement, demotivation
Tips for setting targets • Be “SMART”: specific, measurable, attributable, realistic, and time-scaled • Be honest and unambiguous • Review them periodically (not set in stone) • Get buy-in from all stakeholders • Ensure targets at different levels are linked and define who is responsible for achievements • RBF experience to date: multiple targets at multiple levels (central, provincial, district, facility, community) in accordance with cascading performance contracts
Are there any recent trends for this indicator? What has similar activity achieved elsewhere? Does the target represent value for money? (what will it cost to collect the data?) What issues are likely to affect performance towards the target? Are there any barriers to achieving the target? Tips for setting targets Consider the following questions for each target:
Tips for setting targets Remember: • Larger increases are possible when starting from a low baseline; higher baselines often provide little room for growth or change • Targets should be attainable within a contract period • Targets are tied to performance payments
Section 4: TASK FOR COUNTRY TEAMS
Select potential indicators • Identify indicators most relevant to your country’s national priorities and objectives and the aims of your project; review indicator definitions • Review available data for each indicator in your country, using Countdown and other data sources • Assess each indicator relative to criteria presented here • Define priority indicators for relevant levels, the source of data, and how often it will be reported
Define targets Define targets, taking into account the guiding principles and “tips” presented here For each indicator, at each level:
