1 / 18

Maximizing Evaluation Impact by Maximizing Methods:

Maximizing Evaluation Impact by Maximizing Methods:. Social Network Analysis Combined with Traditional Methods for Measuring Collaboration Carl Hanssen, PhD & MaryAnn Durland, PhD American Evaluation Association Baltimore, MD November 7, 2007. Agenda. Social Network Analysis: The Method

damara
Télécharger la présentation

Maximizing Evaluation Impact by Maximizing Methods:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Maximizing Evaluation Impact by Maximizing Methods: Social Network Analysis Combined with Traditional Methods for Measuring Collaboration Carl Hanssen, PhD & MaryAnn Durland, PhD American Evaluation Association Baltimore, MD November 7, 2007

  2. Agenda • Social Network Analysis: The Method • SNA Results and Interpretation • Next Steps

  3. SNA Methodology • Network Analysis is the study of the relationships formed by the interaction or links between components in a “set”. • MMP sets are schools • The components are individuals: • Faculty, both math and non math • MTL (School level Math Teacher Leaders) • MTS (District level Math Teacher Specialists) • Relationship is communication about math

  4. Measures • Indegree – popularity • Density – how “thick”, how much, out of potential

  5. Role in Evaluation • How much does the communication structure actually fit the theory and the design of the project • Can the structure be correlated with other measures of implementation and impact? • Activities • Proximal measures

  6. Student Achievement Distal Outcomes Classroom Practice Teacher Content & Pedagogical Knowledge Proximal Outcomes Teacher Involvement Learning Team Effort School Buy-in MMP Activities New Courses Math Faculty Involvement District Buy-in UWM Buy-In MATC Buy-In MPA Ownership MMP Evaluation Logic Model

  7. MMP Report Card Indicators • 19 indicators in 7 domains derived from in-school data collection, online surveys, and MPS data • MTS Assessment • Collaboration • Learning Teams • Classroom Practice • Professional Development • Teacher MKT • Student Achievement

  8. SNA In Context: Evaluation Results Overall rating = 3.5 Gap MTL v. other teacher = .2 Teacher Engagement = 3.2 WKCE Mean % Proficient = 44% Student Achievement PD Hrs. = 17.8 Facilitation Hrs. = 1.0 PD Quality = 3.1 Classroom Practice Teacher Content & Pedagogical Knowledge MTS Assessment = 38.3 of 55 Teacher Involvement Overall IRT = -0.34 Algebra IRT = -0.18 Learning Team Effort School Buy-in Team Functioning = 3.5 MMP Principles = 3.6 LT Quality = 3.1 Network density = 6.7% / School density = 17.6% MTL Role = 13.8 / MTS Role = 5.3 SR MTL Engagement = 4.4 / MTS Quality = 3.0

  9. Data Collection • Math stakeholders in each school were asked to name individuals with whom the communicated about mathematics • Statistical analysis focused on • Network and in-school density • Importance of MTL and MTS

  10. MMP Impact Continuum Low High Tight Web MTL Central Many Links to MTL MTS Inside Many Links to MTS Loose Web MTL Not Central Few Links to MTL MTS Outside Few Links to MTS

  11. Low

  12. Medium

  13. High

  14. Student Achievement & In-School Network Density

  15. Student Achievement & MTL In Degree

  16. Conclusions • Distributed leadership—a key program goal is manifested by a tightly webbed network • School-level adoption of program principals is manifested by positioning of key individuals within the network • There may be a natural evolution of school networks that is indicative of program impact in that school

  17. Next Steps • Continue school-level analysis to strengthen our hypothesis about the relationship between social networks and other proximal and distal outcomes • Develop cross-school (or district-wide) networks

  18. Contact Information • Carl Hanssen, PhDHanssen Consulting, LLCcarlh@hanssenconsulting.com616-808-2867 • MaryAnn Durland, PhDDurland Consultingmdurland@durlandconsulting.com630-650-9944

More Related