Hosted by: Colorado Department of Education

1. Unified Improvement Planning: Root Cause Analysis (School Level) Hosted by: Colorado Department of Education Provided by : Center for Transforming Learning and Teaching

2. Introductions

3. Session Purpose Ensure planning teams are prepared to identify root causes of priority performance challenges as part of unified improvement planning.

4. Introductions Share: • Name, Job Title, School/District • Your role in supporting unified improvement planning within the district • Your most burning question about root cause analysis Write your question on a sticky note.

5. Materials

6. The materials used during this session were developed in partnership with the Center for Transforming Learning and Teaching in the School of Education and Human Development at the University of Colorado Denver.

7. Norms The standards of behavior by which we agree to operate while we are engaged in learning together.

8. Session Outcomes Engage in hands-on learning activities and dialogue with colleagues.Access additional resources.Complete follow-up activities. • Evaluate notable trend statements and priority performance challenges using the UIP quality criteria. • Describe how data other than performance data is used in root cause analysis. • Inventory locally available data. • Identify root cause(s) of at least one priority performance challenge. • Plan for completing root cause analysis. • Plan for finalizing the data narrative.

9. Agenda Role of Root Cause Analysis Priority Performance Challenges More than Performance Data Identify Root Causes Finalize the Data Narrative

10. Unified Improvement Planning Processes today Gather and Organize Data Preparing to Plan Section IV:Target Setting Section IV:Action Planning Section III: Data Narrative Review Performance Summary Describe Notable Trends Prioritize Performance Challenges Identify Root Causes Identify Major Improvement Strategies Set Performance Targets Ongoing: Progress Monitoring Identify Interim Measures Identify Implementation Benchmarks UIP Handbook, p. 3

11. Colorado Unified Planning Template Major Sections: • Summary Information about the school or District • Improvement Plan Information • Narrative on Data Analysis and Root Cause Identification • Action Plan(s)

13. Capturing Notes Today • Capture notes for the UIP Data Narrative in the Data Narrative Outline and Data Analysis Worksheet • Use your partially completed version • Blank versions available in the Root Cause Analysis Toolkit • Plan for completing the Data Narrative using the Planning Root Cause Analysis note catcher (Toolkit, p. 7). • Select a recorder and facilitator for your group.

14. Section III, Step Three: Determine Root Causes • Read Step Four: Determine Root Causes (in UIP Handbook, p. 18-20) • Consider the following questions: • Why is it important to identify root causes? • How will you know when you have identified a root cause?

15. The Role of Root Cause Analysis Root Cause Analysis Action Plan Priority Performance Challenges

16. Root Causes are. . . • Statements describing the deepest underlying cause, or causes, of performance challenges. • Causes that if dissolved would result in elimination, or substantial reduction of the performance challenge(s). • Why we see our current pattern of performance. • Things we can change and need to change • The focus of our major improvement strategies. • About adult action.

17. Non-examples of Root Causes • What is NOT a root cause? • Student attributes (poverty level) • Student motivation • With your table, brainstorm a few ideas of explanations that might appear to be root causes but don’t qualify.

18. Background: Root Cause Analysis • Why root causes? • Getting past the “symptoms” to an explanation that is worth taking action on. • Read: “Root Cause Basics” (Root Cause Analysis Tools, p. 11-16) • With your team, create a 1 sentence description of “Root Cause” that you will use.

19. Building Background on Root Cause Analysis • Turn to the Planning for Root Cause Analysis note catcher (Toolkit, p. 7) • Make notes about how you will build background knowledge of your team on root cause analysis. • Who will help with this? • When will this occur? • What materials will you use?

20. Agenda Role of Root Cause Analysis Priority Performance Challenges More than Performance Data Identify Root Causes Finalize the Data Narrative

21. Root Causes and Priority Performance Challenges • Root causes are always focused on at least one priority performance challenge. • They explain why we see the patterns of performance that we see. • So. . . priority performance challenges must be clearly identified first.

22. What are “high quality” Notable Trends and Priority Performance Challenges • Work with a partner. Take out UIP Quality Criteria. Consider: • Data Narrative Criteria (definition and first seven bullets) and for Title I schools (last bullet) p. 3 • Trends (definition and all bullets) p. 4 • Priority Performance Challenges (definition and all bullets) p. 4 • Discuss: What should our Notable Trends and Priority Performance Challenges include?

23. Effective Feedback Clear, descriptive, criterion-based, and indicate: √ how their response differed from that reflected in the UIP quality criteria, and √ how they can move forward (what they might do next).

24. Provide feedback about Priority Performance Challenges • Choose a partner table team or individual. • Exchange trend statements and priority performance challenges. • Consider: • To what degree does the partial draft of their data narrative (trend statements and priority performance challenges) meet the relevant quality criteria? • How could they improve their plan so far? • Provide feedback to your partner.

25. Responding to Feedback • Consider the feedback you received • How will you respond to the feedback you received? What will you do next to incorporate this feedback into your data narrative? • How did it go providing feedback about another district’s plan? • What did you learn? • Large-group share out

26. Agenda Role of Root Cause Analysis Priority Performance Challenges More than Performance Data Identify Root Causes Finalize the Data Narrative

27. If you’re only looking at Student Learning, you’re missing 65% of the data. – Victoria Bernhardt

28. Multiple measures must be considered and used to understand the multifaceted world of learning from the perspective of everyone involved.-Victoria Bernhardt

29. Demographics Perceptions Student Learning/ Performance School Processes What types of data do we have?

30. Demographics Enrollment, Attendance, Race/Ethnicity, Gender, Drop-Out Rate, Language Proficiency, Mobility Victoria Bernhardt

31. Perceptions Perceptions of Learning Environment, Values and Beliefs, Attitudes, Observations Victoria Bernhardt

32. Student Learning (performance) State-Administered Tests, District Interim Tests, Common cross-classroom assessment, End of unit assessment Victoria Bernhardt

33. School Processes Records of School Programs and Processes, Classroom Records, Classroom Observations Victoria Bernhardt

34. Demographics Provides information that allows for the identification of actions, processes, programs that best meet the needs of all students. School Processes Perceptions Student Learning Victoria Bernhardt

35. Activity: Data Questions • Refer to the Data Intersection Questions worksheet. • Consider the questions listed. For each, identify what different types of data would be needed. • Identify 2 additional questions at your table. As a group identify what data would be needed.

36. Activity: Data Intersections • Refer to the Data Intersections Activityworksheet • Could you use this activity with your staff to get them thinking about what data they need for their root cause analysis work?

37. Using “other types” of data • Why is it important to use data other than student performance data in planning? • For what step(s) in unified improvement planning would data other than student learning/performance data be used?

38. Looking Backward LOOKING FORWARD

39. Consider Other Resources Available from CDE • Turn to “Resources Available from CDE to Support Root Cause Analysis” (Toolkit p. 33) • Have you used any of these resources in your district or school? • Do any of these tools address topics likely to be important in your district or school? • Discuss: How could you use these resources to collect data to support your root cause analysis?

40. More details on other data provided by CDE • Equitable Distribution of Teachers • TELL Survey Data • School Site Team Reviews

41. CDE’s History on Equitable Distribution of Teachers Analysis • All districts expected to update Equitable Distribution of Teachers analysis through consolidated application. Moving to UIP now • CDE would prepare reports for districts with higher gaps. All required data is now available on SchoolView.org.

42. ESEA Definition of Equitable Distribution of Teachers ESEA requires that LEAs “ensure…that low-income students and minority students are not taught at higher rates than other students by unqualified, out-of-field, or inexperienced teachers.” (NCLB, Sec 1112(c)(1)(L))

43. Colorado’s Approach to EDT

45. SchoolView: Understanding the EDT Graph The y-axis represents percentage of novice teachers, those less than three years of total teaching experience. The horizontal blue line represents the state’s mean percentage of novice teachers. The red line represents the average percentage of novice teachers within your district. The x-axis represents percentage of free and reduced lunch students, a proxy for poverty. The vertical red line represents the top quartile for poverty for secondary schools. The dots represent schools. The colors represent the overall growth rating on SPF. 1 1 5 6 2 3 3 2 4 4 5 6

46. Understanding Each Quadrant Schools within this quadrant have a high percentage of novice teachers and are serving a high percentage of FRL students. The graph focuses attention on this quadrant. Schools within this quadrant have a high percentage of novice teachers and are serving a lower percentage of FRL students. 4 1 2 Schools within this quadrant have a low percentage of novice teachers and are serving a high percentage of FRL students. 3 Schools within this quadrant have a low percentage of novice teachers and are serving a lower percentage of FRL students.

49. Equitable Distribution of Teachers Practice • Turn to: Interpreting Equitable Distribution of Teachers Data Practice (Toolkit, p. 23) • The first two pages are a reminder about how to interpret this data. • Work with a partner to complete the Practice questions. • The graphic will be on the next slide (in color), so wait for it 