Joint Action on Health Workforce Planning and Forecasting Work Package 3 Evaluation of the Joint Action Finland M.Vallimies-Patomaki, J.Lammintakanen Brussels – April 12, 2013
Agenda of the WP3 kick-off meeting • Introduction of WP3 partners • Scope and objectives of WP3 • Deliverables • Preparation of the evaluation strategy • Discussion topics
Core team 3 • Dr Marjukka Vallimies-Patomäki, WP leader, • Ministry of Social Affairs and Health, Finland • Mr Jesmond Sharples, WP Co-leader, • Ministry of Health, Malta • Prof. Juha Kinnunen, Senior Management Scientist, • University of Eastern Finland, Finland • Prof. Johanna Lammintakanen, Management Scientist, • University of Eastern Finland, Finland
Partners • DG SANCO D2/EU, EAHC/EU • Ministry of Social Affairs and Health/FI, Ministry of Health/MT, Ministry of Public Health/BE, Ministry of Health/PL, Ministry of Health/SK • University of Eastern Finland/FI, Medical University of Varna/BG, NCPHA/BG, University of Bremen/DE, Semmelweis University/HU • CfWI/UK • EFN, EHMA
Scope and objectives • WP3 provides a systematic approach • for the internal evaluation of the Joint Action • Aim is to verify if the JA is being implemented as • planned and achieving the objectives: • - output evaluation is to verify • compliance to achievement of the objectives • -outcome evaluation is to assess • usefulness of outcomes, recommendations and • sustainability beyond the Joint Action • - process evaluation is to ensure • compliance to the rules and implementation
Engagementof the member states and stakeholders • Mentoring group: • - WP leaders and representatives of stakeholders • - Discussing and giving feedback of the evaluation strategy • Expert reference groups: • - Representatives of member states and stakeholders • - Discussing and appraising the quality of the deliverables • Focus groups: • - Representatives of member states and stakeholders • - Appraising the impact and sustainability of the JA
Deliverables • Evaluation strategy and evaluation tools,August 2013 • Expert reference group report, November 2014 • Intermediate evaluation report, January 2015 • Expert reference group report, December 2015 • Final evaluation report,March 2016
Schedule for the preparation of the evaluation strategy • 1 st draft by April 30/Commission & WP1 • Basic contents and evaluation indicators • Comments by May 22 • Evaluation indicators and spesific objectives: consultation through teleconference/Commission, WP1 &WP leaders • 2 nd draft by June 7 /Commission, WP1 & WP leaders • Comments by June 19(Dates changed by WP3) • Stakeholder analysis, in collaboration with WP2 (> into an appendix) • 3 rd version by June 30/Commission, WP1, WP leaders, partners, strategic stakeholders • Comments by July 31 • Final version to WP1 by August to be submitted to EB
Review of spesific objectives • Spesific • Spesify the target group and the factors that need to be • changed • Measurable • Formulate objectives in a measurable format • Appropriate • Make sure objectives are acchievable, attainable, • acceptable for the target group • Realistic • Ensure that you can realisticly acchieve the objectives given the project resources • Timebound • Connect objectives to a timeline
Key performance indicators • Ensure that WP leaders are involved in identifying KPIs for all WPs of the JA • KPI help the JA define and measure progress toward desired outcomes • KPIs are quantifiable (measurable) measurements, agreed to beforehand, that reflect the critical success factors of the JA School: graduation rates of the studentsSocial service organization: the number of clients assisted during the year • Source: www.management.about.com.
Overview of the discussion topics • Evaluation tools: • Find examples and experiences • Spesific objectives and evaluation indicators : • Review in collaboration with Commision, WP1 and WP leaders • Stakeholder analysis: • In collaboration with WP2 • Preparation of the evaluation strategy: • In collaboration with Commission, WP1, WP leaders, University of Bremen etc.
Evaluation process: in co-operation with all WPs 14 Output evaluation (WP2,4,5,6,7) Compliance to the content ACHIEVEMENT OF THE JA OBJECTIVES, IMPLEMENTATION AND SUSTAINABILITY OF THE JA 2016 Each WP’s objectives Deliverable: Final report (> EB) 2013: SETTING OBJECTIVES FOR THE JA AND WPs ”…to create a platform for collaboration and exchange between MSs to better prepere for the future of the health workforce” Outcome evaluation (+WP2,6,7) Lessons learnt and recommendations Process evaluation (+ WP 1) Compliance to the rules Interim report > EB