Decade of Roma Inclusion Progress monitoring

1. Decade of Roma Inclusion Progress monitoring Challenges, problems and possible approaches Andrey Ivanov, UNDP

2. Why evidence is crucial? • The Decade – MDGs for one of the most vulnerable groups, the Roma • Clear commitment from governments with strict time-frame set (2005-2015) • Major priority areas - poverty, employment, education, health and housing • National action plans for Roma Decades developed and Decade secretariats established in 2003-2005

3. The Decade as data challenge • Intends to address the needs of fluid population. Unclear what to put in the denominator when computing various indicators • NAPs need reliable estimate to allocate resources. • Both sides – governments and Roma – often speak different languages. High level of mistrust (‘pre-divorce’ phenomenon) • Susceptible to manipulation by both sides • Government has incentives to ‘report success’ • Roma CSO have incentives to report failures

4. The approaches so far • NAPs usually lack clear indicators. Input, output, outcome and impact often confused (not clear deliberately or not) • The monitoring role – taken over by non-governmental actors • The Decade Watch – published in 2007, assessment of the inputs to the Decade implementation as seen by Roma CSOs • WB/OSI/UNDP ‘Decade indicators working group’ – working since 2007 on internationally comparable indicators • UNDP – two rounds of data collection (2002 and 2004) and increasing supporting national-level efforts for NAPs implementation • The current pattern may reinforce the mutual distrust of parties directly involved (governments and Roma civil society)

5. UNDP approach to Decade monitoring • It is neither possible nor reasonable to invent and implement “Roma indicators”. • For the monitoring of the Decade a standard sets of socio-economic and human development indicators should be applied • Standard indicators must be fed with ethnically disaggregated data and the challenge boils down to availability of such data • Indicators should follow the policy purpose • Anything too simple is inevitably misleading • Choosing between national adequacy and international comparability the former is a priority (following the MDGs pattern)

6. UNDP approach to data process • National actors should be supported and not substituted in data issues • International organizations can test the grounds but not take over • There is a lot of data available, the challenge is to use it in creative way. Some possible approaches include: • Disaggregating hard statistics using personal identification numbers as a common denominator (key link) for mutually complementing data sets • Disaggregating hard statistics using territorial tags as ethnic markers combining quantitative data with qualitative information. Gives estimate of the real needs (adequate for policy-making) • Extending the samples of regular sample based surveys with ethnic boosters • Custom “on the spot” surveys conducted among recipients of different social services • Collecting data at a community level by Community-based data collectors and monitors • Using measurable proxies that are strongly correlated with Roma identity to estimate progress in sectoral priorities of the Decade

7. UNDP approach to NAPs • Consistently distinguish input, output, outcome, and impact indicators • Put different focus on different levels of monitoring (central and local) • Integrate the monitoring functions into the whole NAP implementation strategy (so far it is detached) • Support local governments in data collection, data analysis and data application capacities (skills to understand and use data are even more important than the existence of data per se) • Regularly update of the NAPs with M&E components with necessary indicators • Include Roma in the whole cycle (collection-processing-analysis-policy modification)

8. Comparing different approaches do data disaggregation

9. Comparing different approaches do data disaggregation

10. Conclusions • Defendable indicators for progress monitoring are crucial both for policy design and policy impact assessment. Without such indicators the Decade may fail in its promises • Disaggregating statistical data by ethnicity is possible even when exact number of Roma population is unclear. Constructing ethnically sensitive indicators is possible – both national and internationally comparable • Problems exist, however they are less of methodological, technical or financial but rather of political nature • Given the concerns regarding individual data integrity, such disaggregation and construction of indicators should be done by specially appointed agency operating in line within clear legislation on the matter. National statistics are best suited for such a role.