Research Policies and Mechanisms: Key Points from the National Mathematics Advisory Panel

1. Research Policies and Mechanisms: Key Points from the National Mathematics Advisory Panel Joan Ferrini-Mundy Director, Division of Research on Learning in Formal and Informal Settings, National Science Foundation National Math Panel Forum, October 7, 2008, Washington DC

2. What standards were used for including research? • What observations can be made about the state of educational research as it relates to mathematics education policy? • First things first and continuous improvement; what can we do?

3. research about mathematics teaching and learning

4. research about mathematics teaching and learning research about mathematics teaching and learning that could provide evidence for policy making

5. research about mathematics teaching and learning research about mathematics teaching and learning that could provide evidence for policy making research about mathematics teaching and learning that could provide evidence for policy making and meets standards of evidence

6. Policy questions: cause and effect, inferences from samples to populations “Causal knowledge is essential to produce and to evaluate scientific research in crucial areas of national need, including mathematics education.” (p. 63)

7. research about mathematics teaching and learning that could provide evidence for policy making and meets standards of evidence

8. “The Panel’s assertions and recommendations, therefore, are grounded in the highest quality evidence available from scientific studies.”

9. Place the strongest confidence in studies that: • Test hypotheses • Meet the highest methodological standards (internal validity) • Have been replicated with diverse samples of students under conditions that warrant generalization (external validity)

10. Standards for studies of the effects of interventions • High quality: random assignment to conditions, low attrition • Moderate quality: matching statistical controls, or demonstration of baseline equivalence; low attrition or evidence of small attrition effects; valid and reliable measures

11. Where to find commentary about research • Final Report: • Chapter 10, Research Policies and Mechanisms • Appendix C, Standards of Evidence • Report of the Task Groups and Subcommittees • Chapter 2: Report of the subcommittee on standards of evidence • Chapters 3-9, individual task group and subcommittee reports

12. The panel and contractors considered ~16,000 research publications and policy documents • An unprecedented position from which to make comments about research about mathematics teaching and learning that could provide evidence for policy making

13. We need: • “Sufficient supply of competent researchers dedicated to areas of critical national need” • “Sufficient supply of willing schools and practitioners who have the time, resources, and motivation to be partners in research and use the findings of research in decision making” • “A sufficient and stable source of funding for quality research and training with appropriate peer review”

14. Something for all: • Mathematics education research community • Researchers from a variety of fields • Leaders of graduate programs • State and district education policy makers • Federal policy makers and funding agencies

15. Mathematics education research communityand collaborators from other fields: More research on the key questions that could provide evidence for policy making: • “Effective instructional practices and materials • “Mechanisms of learning” • “Ways to enhance teachers’ effectiveness, including teacher education that focuses on learning processes and outcomes” • “Item and test features that improve the assessment of mathematical knowledge”

16. Leaders of graduate programs: Redesign and improve programs and courses: • “Attention to research design, analysis, and interpretation for teachers” • “Increase national capacity to conduct and utilize rigorous research”

17. Researchers from a variety of fields Pursue critical research issues in mathematics education: • “Create cross disciplinary research teams” • “Include expertise from educational psychology, sociology, economics, cognitive development, mathematics, and mathematics education”

18. State and district education policy makers Collaborate in enabling educational research: • “Provide preK-12 schools with incentives and resources to provide venues for, and encourage collaboration in, educational research” • Follow progress of“resolutions to make individual student data available to researchers with appropriate safeguards for confidentiality”

19. A finding of the Instructional Practices group “All encompassing recommendations that instruction should be entirely ‘student centered’ or ‘teacher directed’ are not supported by research.”

20. An R&D Cycle for Improving K-12 Education: evaluate and generalize synthesize and theorize implement, study, and improve design, develop, and test hypothesize and clarify Adapted from NSF/DRL solicitations, 2007-08

21. Large field trials that address problems of major national importance” (p. 63) evaluate and generalize synthesize and theorize implement, study, and improve “smaller scale (less costly but highly informative) experiments (p. 63) design, develop, and test hypothesize and clarify “to be ready for even small scale experiments, basic research and intervention development studies are needed” (p. 63)

22. First things first and continuous improvement: • Interpretation for policy makers and practitioners • Scholarly discussion and critique • Research agenda initiatives • Research studies • Capacity building • New interdisciplinary collaborations