Multiple Purposes of Data Collection

1. Multiple Purposes of Data Collection William T Grant i3 Learning Community October 14-15, 2012 Elma Ruiz and Alison Boardman

2. Welcome & Introductions • Who is in the room? With a show of hands: • University (IHE) • School District (LEA) • Community Based Organization (CBO) • Additional participants? • What drew you to this session? What piqued your interest? What questions or issues are you facing with regards to this topic? • Session goals • Share efforts to reduce burden, maximize utility of and maintain research standards in data collection and sharing • Discuss opportunities and challenges related to data sharing

3. CSR Colorado i3 Validation Grant CSR Colorado received a validation grant to brings the unique approach of Collaborative Strategic Reading (CSR) to DPS middle schools. Through research, classroom implementation, school-wide approach and community outreach, CSR Colorado provides a holistic approach to improving the reading comprehension skills of middle school students. The initiative is facilitated by a unique partnership between Denver Public Schools, the University of Colorado Boulder and Padres & Jóvenes Unidos.

4. Build Capacity at District and School Level

5. Capacity Building with a Focus on Results

6. Collaborative Strategic Reading Collaborative Strategic Reading (CSR) combines reading comprehension strategy instruction (e.g., Palincsar & Brown, 1984) and cooperative learning (e.g., Johnson & Johnson, 1989). In CSR, students read and discuss text through a combination of teacher led discussion activities and student-led cooperative group work. CSR promotes content learning, language acquisition, and reading comprehension in diverse classrooms that include English language learners and students with learning disabilities (Klingner, Vaughn, Argüelles, Hughes, & Ahwee, 2004; Klingner, Vaughn, & Schumm, 1998; Vaughn et al., 2011).

7. Coaching Data • Coaches input data into an online data management system using ipads. This data is used to: • Track and monitor coaching activities (e.g., consistency, amount) • Track teacher change over time. • Provide feedback to teachers. • Triangulate implementation fidelity data. • Report trends to school administrators.

8. Teacher Reported Implementation Data • Teacher Implementation Data • Monitor implementation. • Track change over time. • Link implementation to student outcomes. • Report to administrators with purpose of working with school to support implementation.

9. Facilitating data collection and sharing • Features we have developed to coordinate data needs: • Mobile online data collection system. • Considerations during data base development that facilitate quantitative and qualitative data analysis. • Different levels of access depending on role. • Ex. University researchers have access to all coaching data while school district coaches have access to coaching data for teachers they work with and selected fields for other teachers. • Various forms to capture a variety of coaching activities. • Reports created to make data easier to use for coaches. Forms and Reports

13. Coaching Report

14. Sharing data with schools • School Level • Principal update meetings to share trends across teachers and develop a shared vision of CSR in action • Sharing classroom and whole school implementation data • Principal dashboard with drill down pages • Landing page is visual representation of school-wide implementation

15. Principal Dashboard

16. Sharing Data with the District • District Level: Ownership of the Reform • Shared understanding at senior leadership level • Implementation Dashboard, access to all schools implementing reform • Briefings on impact and implementation and articulated role of Instructional Superintendents (IS) • Reform is called out and school improvement conversation include data to guide school improvement efforts (UIP) • Fidelity PD, informed by data, for district level coaches and key support staff • Aligned and identified in teacher evaluation system

17. Our Data Conundrum... • How can we remain true to IRB requirements around data sharing while providing enough information to support school implementation? • How can we make use of the extensive data collection? • The district sees an opportunity to benefit from the intensive data collection and support provided. • Access to videos of strong teachers. • Asking researchers to observe “effective teaching indicators” not part of the CSR intervention. • How does our research really influence schools? • The reality is that we impact more in schools than the implementation of the intervention we are validating. • We continue to be vigilant about supporting schools and the district while maintaining research integrity.

18. Consultancy Protocol: • The presenter gives an overview of the challenge and frames a question for the group to consider. (1-2 minutes) • The group asks clarifying and probing questions of the presenter. The presenter may respond to the group’s questions, but there is no discussion by the group of the presenter’s responses. (5 minutes) • The group talks with each other about the dilemma presented. (8-10 minutes) • Possible questions to frame the discussion: What did we hear? What didn’t we hear that they think might be relevant? What assumptions seem to be operating? • What questions does the dilemma raise for us? What might we do or try if faced with a similar dilemma? What have we done in similar situations? • Members of the group may suggest actions the presenter might consider taking. Most often however, they work to define the issues more thoroughly and objectively. The presenter doesn’t speak during this discussion, but instead listens and takes notes. • The presenter reflects on what s/he heard and on what s/he is now thinking, sharing with the group anything that particularly resonated for him or her during any part of the Consultancy. (5 minutes)

19. Table Conversations • Small or Table Groups (20 Min) • What is your biggest data collection/data sharing challenge?  • What have you done to address that challenge to date? • Pick one problem as a group to discuss and workshop - help one member of the small group actually work through a challenge, consultancy protocol. • Large group discussion (20 min) • What kinds of challenges did you discuss?  • What solutions did you come up with?  • What are some of the common themes?  • What still needs to be figured out?

20. www.csrcolorado.org

21. Questions?