Internal Evaluation of MMP

1. Internal Evaluation of MMP Cindy M. Walker Jacqueline Gosz Razia Azen University of Wisconsin Milwaukee

2. Main Goal To determine the effect of the Milwaukee Mathematics Partnership (MMP) on mathematics achievement in Milwaukee Public Schools (MPS)

4. Design Issues • Within a school, it is ultimately the mathematics teacher that has the greatest capacity to impact student achievement in mathematics.

5. Design Issues • How is the MMP influencing or changing what is going in MPS classrooms? • In Year 2 we intend to capture school level differences, in terms of outcomes that should be affected by the MMP, such as differences in MTLs and Learning Teams. • Use hierarchical linear modeling to model student growth and determine if variability in these school level variables help to explain variability in student growth. • If this is true, then it is likely that changes are occurring at the classroom level.

6. Math Teacher Leader • Variables that the MMP should be directly influencing include: • Pedagogical Content Knowledge • Distributed Leadership Abilities • Quality of support • Variables that the MMP has little or no control over • Participation level in the MMP and Learning Team Meetings • Relationship with other teachers

7. Learning Team • Variables that MMP has indirect influence on by interactions with MTLs • Time spent discussing mathematics targets • Coherency and consistency of a pedagogically sound math program used both within and across grades • Quality of team functioning • Variables that MMP has limited or no control over • Quality of team functioning • Frequency of meeting • Level of support provided to MTL and other math teachers

8. Methodological Issues • Misspecification Error – Are we excluding important predictor variables? Can we even measure all the important variables of interest? • Continue to work closely with PIs, Math Specialists, and External Evaluation Team at UIUC • Begin to measure variables of interest at the individual classroom teacher level

9. Classroom Mathematics Teachers • What is being taught and how? • Are mathematics teachers engaging students in communication, problem solving, reasoning? • Measurement Error • Conduct psychometric analyses of measures used • Survey classroom teacher regarding quality of support and access to support to validate data obtained from MTL and Learning Teams • Survey Learning Team members (i.e. principals, literacy coaches, and MTLs) to validate data obtained from classroom teachers regarding what is being taught in classrooms and how • Pilot these instruments in Year 2, use on large scale basis in Year 3

10. Methodological Issues • Confound of MTL being a school level variable AND a teacher level variable • Dummy code students as to whether or not taught by a MTL • Philosophical question of including SES, race and other various student level covariates in the model • Fit model in various ways and compare

11. Data Issues • Missing data – at all levels • Use as an indicator of level of participation? • Who is actually teaching math? • Data collected from MTL for elementary and middle schools, if missing assume homeroom teacher, acknowledge error • Mobility – at all levels • Weight schools and/or teachers, use as covariate at student level

12. Data Issues • No standardization across the district • Merging of many databases • Data entry errors • Continue to work closely with MPS, especially the Technology Department, to ensure data used is as accurate as possible.