Evidence based policy making: trends and challenges in SEE

1. Evidence based policy making: trends and challenges in SEE Vienna, November, 2009 Tünde Kovač-Cerović

2. The researcher: Are policies using research evidence? • The policy maker: Are research agendas addressing policy dilemmas?

3. Conclusion from Bucharest BHC: Which path? high quality Based on impressions Based on evidence low quality 3

4. 3 reasons why evidence is more important in education • Lay theories strong • Learning cannot be legislated • Even if it could be, changing atmoshere in education

5. General impression/SEE • Context: • Politics/ideology flooding policy • Shrunk research potential (brain drain, lack of data, fragmented research space) • Development agenda does not prioritize education • Education underfunded, unattractive • Expectation: • Evidence based policy making and using evidence is a far ideal

6. However: • Evidence is increasingly important for policy decisions in SEE • Consequence of PISA • Consequence of political instabilities • Consequence of reporting requirements • Consequence of access to information • Consequence of project requirements • Policy dilemmas are increasingly informing research agendas in SEE • Funds for policy related research • Researchers’ fascination with big data sets • Motivational aspect: “Research can be relevant!” • New niches of evidence-based policymaking appeared

7. New niches of policy making • Responding to the economic crisis (efficiency, rationalization, financing mechanisms, “more for less”) • Responding to global policy challenges: • Inclusion/equity (Roma, SEN) • Early education • Opening long-standing unresolved issues: • Teachers/teacher education • Introducing formative assessment • Opening new agendas: • What is going on in the school? In the classroom? • Student academic motivation • Parent/student participation in decision-making

8. Current niches of research in Serbia • Responding to previous policy questions: • Bologna &TE • Analyzing PISA & TIMSS • Assessment of policies under implementation (equity implications): • Preschool • Scholarships • Evaluation of pilots: • Roma programs • VET reform • TT programs

9. Are there mismatches? • “dense” policy agenda – “thin” research capacities • Further needs

10. Finding connections between input and process variables which maximize learning and social outcomes Economic research input process outcomes Evidence- Based Teaching Structures Financing Management Revisit all 3 Learning outcomes Social outcomes Education research

11. Need 1: descriptive analysis of input or process variables

12. Missing analysis/neglected fields of policy research • Financing mechanisms • Corruption • Excluded children • Teaching force • Teacher education and • Classroom practices • Hidden curriculum • Parent/student participation

13. NEED 2: Pulling together existing knowledge on process

14. Overview on “Pulling together existing knowledge” • 3examples: • Learning: • Teaching strategies of learning and motivation • Assessment and feedback • Teaching: Teaching methods • Social interaction in the teaching/learning environment • Reflected in education research 14

15. Example A/ Research on learning • “effectiveteaching”, • + 100 x 10 x 12 studies each year • Re-conceptualizing learning: Educational psychologist 2009 • + meta-meta-analyses (Hattie, 2007, EARLI, on750+ meta-analyses, 50,000 studies, and 200+ million students)

16. Hattie: Main factors of achievement Teachers Students Home Peers Schools Principal

17. Influences on Achievement .40 .30 .50 .60 .15 Typical Teacher Effects .70 .80 Developmental Effects ZONE OF DESIRED EFFECTS 0 .90 REVERSE 1.0

18. John Hattie

19. Learning strategies in SEE– flip side • Teachers know? • Parents request? • Students do? • Private tutors? Assessment – flip side • Objective? • Oral examination: not objective, not reliable, rare • Relevant? • Includes irrelevant variables: verbal fluency, sensitivity for non-verbal signalisation... • Informative? • For teachers? • For students? Loss of possibility to gain complex learning outcomes

20. Example B/Teaching methods • Johnson & Johnson, 1983; 2000 • 158 studies on the effects of cooperative learning

21. Effective teaching methods for different goals

22. Effective cooperative teaching methods (2000)

23. Cooperative learning – flip side ? • Wide offer • Slim practice • Emphasis on knowledge of specific information Loss of possibility to gain complex learning outcomes

24. Example C/ research on Teacher/student interactions Research shows that quality of interaction matters, school can create barriers or support: 1. Hierarchy of motives: Crucial: 4th and 5th level: Respect basic to knowledge and understanding Respect need not felt if met, felt only if not met: source of misunderstandings Students’ memories 24

25. Teacher/student interactions 2. Expectations : Expectations of teachers function as self-fulfilling prophecies: • Capacity development (Rosenthal & Jacobson) • Motivation (Pelletier & Vallerand) 3. Complex predictors: Academic self-expectation best predictor of school success (Wigfield) Self’efficacy and internal locus of control the strongest predictors of school success after abilituies (Pajaros i Miller; Zimmerman i Bandura; Bandura)

26. Teacher/student interactions - flip side • Lack of praise and awards • Disregard • Students struggle for getting motivated to accomplish non-challenging tasks • Teachers’ expectations uninformed Loss of possibility to gain complex learning outcomes

27. Conclusions

28. Conclusion 1.: Learning still a rare event Attended ... Listened ... Heard ... Understood ... Remembered ... Will apply ...

29. Conclusion 2: Need to strengthen policy-research connections Using existing research results International instruments fostering research Education Developmental priorities transparent Education research in other countries

30. Conclusion 3: What do we need to do to strengthen the path: On policy-side On research-side Education research community in SEE Networking of research agencies Joint projects SEE education policy journal • Making the policy agenda transparent for researchers • Policy debates – national/SEE • Best practice Clearinghouse Hope the Conference will elaborate further on this

31. 2025 SCHOOL Personal benefits Social benefits regulated teachers textbooks curriculum equitable financing management assessment evaluation Research participatory Development Policies efficient accountable

32. Thank you!