Monitoring and Evaluation

1. Monitoring and Evaluation Jela Tvrdonova Cordoba, 2010

2. Effects of the strategy in the development trend context • No can solve all problems of particular area, country or region. • But targets in certain time only some areas of life • It functions in certain contextual environment, which „goes its own way“ as well.

3. Figure. 1 Context of the program/strategy in rural development Rural areas as defined by national definition/all stakeholders in rural areas The area of intervention rooted in the EU Strategic guidelines EC reg. 1698/2005, eligible beneficiaries, EU objectives The intervention area rooted in National strategic plan for RD ,selected eligible beneficiaries – national objectives Other interventions and objectives • Context of the local development strategy

4. Contextofthestrategy and itseffects • In such a relatively complex situation is sometimes complicated to evaluate effects of the intervention. • The overall situation is located in changing environment, which can influence the intervention of the program/strategy in many different ways.

5. Development program/strategy Other programs effects Other Non-program effects Direct effects on eligible beneficiaries In-direct effects on non-beneficiaries In-direct effects on eligible beneficiaries Program/strategy´s effects

6. Program/strategy logic • Program/strategy effects depend to the high extent on the quality of its intervention logic design. • The development of the area goes with the line of its goals and objectives • It proves the efficient use of public and private funds in favour of whished change • Brings sustainable solutions of territory´s needs and challenges.

7. Change Needs, visions Contexte of development– all factors influencing development Change observable in reality during and after intervention Whished change in priority areas before intervention Overall /strategic goal Impact Specific objectives within development priorities Result Output Operational objectives of measures Measures Inputs Overall contextual baseline socio-economic and environmental situation before intervention Logic of the program/strategy

8. Monitoring and evaluation • It is necessary to distinguish among various processes – audit, monitoring and evaluation

9. Audit and Monitoring • Audit , control is focused on the correct administrative and financial management of measures • Monitoring is continuous and systematic observation of financial and other inputs, implemented projects and activities and intermediate outputs within the implementation of particular measures of the local development strategy.

10. Evaluation • Evaluation is focused on the assessment of results and impacts of the program/strategy, the same time looking at the effectiveness, efficiency and the relevance of particular measures and offers useful reflexion for re-formulating or overall change of policies. • In this process the quality of the evaluation substantially depends on the quality of available data and information collected in the monitoring processes or also contextually, which means that both processes must be interlinked and designed in interaction straight from the beginning.

11. Monitoring and evaluation • To set up proper monitoring mechanism – indicators – input and output indicators (reflecting the activities and projects level) • To set up proper evaluation system – result (reflecting specific objectives) and impact indicators (reflecting strategic goals) • To design the system together with the setting up objectives, goals, priorities, measures and activities

12. Example of the monitoring framework

13. Evaluation Evaluation: • On going evaluation is helping us to collect data which help us to carry the evaluation exercise in time periods – ex-ante, mid term and ex post • Ex ante evaluation – independent study, which helps us to understand if the projected intervention is really going to enable to achieve envisioned change expressed in objectives

14. Evaluation • Mid term evaluation is the independent exercise which is carried out in the middle of the strategy implementation. The changes on the level of specific objectives/development priorities shall be observed and results shall be measured • Ex post evaluation as the last independent study is carried out at the end of the strategy implementation. The impact of the strategy implementation shall be visible and strategic objectives shall be achieved

15. Self-evaluation Self-evaluation • is the process which is carried out by local action group/partnership itself • it shall help to steer/coach the strategy in such a way that objectives are followed • for this purpose any findings of not well prepared interventions within the self-evaluation process shall be discussed and necessary changes of the strategy shall be implemented as the lessons learned

16. Contextual monitoring and evaluation • This M/E is based on the prediction that the intervention of the strategy exist in the environment which might itself to strengthen or weaken our intervention instruments • It helps to distinguish to which extent the change was caused by our intervention and to which extent the change was caused by external factors • Contextual indicators help to carry this process

17. Components of monitoring and evaluation framework Inputs Activities Outputs Results Impacts • Inmediate results on the priority level • New quality, changes • New knowledge • skills • attitude • nové správanie sa • What plan does • Services • Training • Processing • Resources • money • people • supply • equipment • facilities • Products orservices • Numbers of products or services • Long term changes • Achievement of the strategy objectives • Improved situation

18. Indicators Are instruments for the measurement of interventions Indicators are on the level of: Monitoring – input and output (physical financial) Evaluation – results (priority level) and impacts (the cross strategy level)

19. Evaluation questions • Common – cross strategy evaluation questions and specific (sector/priority) – are those which shall be answered in the evaluation process. • Shall be well formulated so we will receive the answer if objectives were reached

20. Baselines • Are base data we were able to collect in resource audit or in any other additional data collection and are available in existing data collection systems, which help us to understand the initial situation before the intervention starts • Accessible data shall be in balance with M/E indicators or vise versa – the preferably we shall select indicators which can be filled with data

21. Target levels Are wished situations in which the development priorities shall be after intervention is completed and express the quantitative level of the change E.g. unemployment decrease in 25% is the target level

22. Example of the evaluation framework

23. Monitoring and evaluation framework • shall be part of the strategy from the beginning • It shall be organicaly incorporated into the strategy and shall be designed based on the situation analysis, needs and objectives, measures and activities

24. Evaluation questions Indicators Baseline situation (situation analysis,SWOT) Target levels Objectives Strategic /overall objectives Cross strategy /horizontal evaluation questions Impact indicators Baseline values of impact indicators Target levels for impact indicators Specific objectives Horizontal evaluation questions relating tospecific objectives/priorities Result indicators Baseline values of result indicators Target levels of result indicators Operational objectives, measures activities Measure specific evaluation questions Output indicators Baseline values of output indicators Target levels of output indicators Monitoring and evaluation framework

25. Partnership Territory Strategic objecive Impact Impact Indicators E v a l u a t i o n Vision I n t e r v e n t i o n Resource audit Analysis Priorities for development Specific objectives Result Result indicators Input indicators Measures Output Output Indicators Monitoring The strategy framework - completed