Using Data to Problem Solve

1. Using Data to Problem Solve Susan Barrett www.pbis.org www.pbismaryland.org

2. Thanks to… • Center on PBIS • Steve Goodman- Michigan • Tim Lewis • Rob Horner • George Sugai • Catherine Bradshaw • Don Kincaid

3. Adopt systems perspective at a Building Level Systems Perspective • Organization do not “behave” …individuals behave • “Organization is group of individuals who behave together to achieve a common goal” • “Systems are needed to support collective use of best practices by individuals in an organization” (Horner, 2001) Schools as Systems Goal to create communities that for all its members have common • Vision • Language, & • Experience Biglan, 1995; Horner, 2002

4. What a Leadership Team does… • Communicates common vision for schoolwide supports • Works collaboratively to establish building capacity to support all students • Commits resources to establish procedures for support • Develops methods for evaluating progress towards measureable outcomes • Action planning based on data

5. Should get easier for your school over time • Handbook • Describes core features • Expectations and teaching matrix (rules for settings) • Teaching plans and teaching schedule • Acknowledgement system • Continuum of consequences for problem behavior • Building Leadership Team • Regular meeting schedule and process • Regular schedule for annual planning and training • Annual Calendar of Activities • On-going support for staff

6. Use the PBIS Maryland website as reference www.pbismaryland.org

7. Team Time Your Leadership Team • Does your team understand the leadership function in managing and coordinating implementation? • On a scale of 1 (low) to 5 (high), how well is your team doing with this responsibility?

8. Standards and Protocols • Available on www.pbismaryland.org • What is a “PBIS trained” team? • What happens if a school does not meet criteria? • What is required to be a PBIS implementing school? • How does a school become inactive? • What is required for a school to be eligible for PBIS Maryland Recognition?

9. Purpose of Systems Measures • Benchmarks of Quality Checklist • Evaluates status of Tier I Positive Behavior Supports • Completed Annually, Required for Recognition • Submitted to Jerry electronically on April 10 • Submitted online at www.pbssurveys.org • Implementation Phase Inventory • Required for Recognition • Due November 10 and April 10 to Jerry • Self-Assessment Survey- Not required but… • Evaluates status of Schoolwide, Nonclassroom, Classroom and Individual Student Supports • Submitted online at www.pbssurveys.org

10. What is the BOQ? • Lists components of PBIS programs that address the critical elements of PBIS implementation • Completed by school teams on a yearly basis to assess how they score on a 100 point scale with regard to developing and implementing school-wide PBIS • Useful in developing action plans for following year • One of the measures used by MSDE-SPHS-JHU to determine schools achieving Exemplar Status

11. Three Components of Benchmarks of Quality • Team Member Rating Form • Completed by team members independently • Returned to coach • Scoring Form • Completed by coach using Scoring Guide • Used for reporting back to team • Scoring Guide • Describes procedure for completing BOQ • Includes a rubric for scoring each item

12. BOQ Will Provide: • Summary of team members’ perceptions of PBIS implementation (scored: ++ in place, + needs improvement, and - not in place) • Objective assessment of school’s implementation based on criteria described in a rubric (100 point scale) • Comparison between the above factors which will encourage discussion of strengths and weaknesses and provides ideas for action planning

13. What is the IPI? • Implementation Phases Inventory • Two times/year • Due November 10, April 10 • Coach completes with Team • Four Phases • Preparation • Initiation • Implementation • Maintenance

14. What is the SET? School-wide Evaluation Tool One of several methods to evaluate Tier 1 Required if school is seeking Recognition Status External certified SET Assessor will conduct site visit Should you be a SET Assessor?

15. Measures the level of implementation of SWPBIS (not intended to measure everything!) The Critical Features Expectations Defined Expectations Taught System for Encouraging Expected Behaviors System for Discouraging Problem Behaviors Monitoring and Decision Making Management District Level Support What does it measure?

16. Why use it? The results help PBIS teams: Assess the features of PBIS in place Determine annual goals for school-wide effective behavior support evaluate on-going efforts toward school-wide behavior support design and revise procedures as needed compare efforts toward school-wide effective behavior support from year to year

17. What does it look like? Permanent product review Office Discipline Referral (ODR) form Current Action Plan Discipline Handbook/Plan School Improvement Plan Lesson Plans & Schedule 2 to 3 hour school visit: Observations Classroom and Non-classroom settings Interviews Administrator, Staff, PBIS Team Members, and Students

18. What is the Self-Assessment Survey? • Self-assessment survey to assess the extent to which Positive Behavior Support practices and systems are in place within a school • School-wide (15 items) • Non-classroom (Specific Setting) (9 items) • Classroom (11 items) • Individual Student (8 items)

19. Who Completes the Self-Assessment Survey? • Initially, the entire staff in a school completes the Survey. • In subsequent years and as an on-going assessment and planning tool, the Survey can be completed in several ways: • All staff at a staff meeting. • Individuals from a representative group. • Team member-led focus group.

20. Using the Self-Assessment Information for Decision Making • Is a system in place? • “in place” > 66% • Is there a need to focus on a system? • Current status of “in place” is < 66% and • Priority for improvement is “High” for > 50% • Which system should receive focus first? • Always establish schoolwide as first priority • Which features of the system need attention? • Combine survey outcomes with information on office referrals, attendance, suspensions, vandalism, perceptions of staff/faculty

21. Individual Summary Charts • Charts are provided for each system (school-wide, nonclassroom, classroom, and individual) • Current status Charts • Percentage of respondents who answered "In Place", "Partially In Place", and "Not In Place" • Improvement Priority Charts • Percentage of respondents who answered "High", "Medium", and "Low”

22. Example of PBS Self Assessment Survey Individual Summaries Chart

23. Analysis of Schoolwide System Chart Shows a chart with bars for components of the schoolwide system • Expectations defined (question 1) • Expectations taught (question 2) • Reward system (question 3) • Violations system (question 4-8) • Monitoring (question 10-12) • Management (question 9, 14-16) • District support (question 17-18)

24. Analysis of Schoolwide System Chart

25. Example of PBS Self Assessment Survey Individual Item Score Schoolwide Component White = In Place Yellow = Partial In Place Red = Not In Place

26. Why conduct Self-Assessment Survey in addition to Checklists? • Checklists are conducted by team, all/most staff complete survey • Look for areas of convergence across tools • Increases confidence of data • Look for areas of divergence across tools • Decrease confidence of data? • Possible reasons for disparity… • Lack of understanding of questions • Staff not fully aware of work of Building Leadership Team • Support component not fully “In Place”

27. Differences between the BOQ action plan form and the Self-Assessment Survey

28. To Do List • Review results from… • BOQ Checklist • Determine if you would like your school staff to complete Self-Assessment Survey • Review School’s Action plan- What is the link to overall School Improvement Plan? • Based on this information complete action plan for you! • What celebrations can you share with your school community before this year is over? • What is your plan to strengthen your schools’ behavior support for the next school Year? Please take a moment to complete the appropriate section of the Follow-Up Activity Worksheet to document the work yet to be done

29. Student Measures • How do we know Implementation of Tier 1 PBIS is making an impact? • What data should our team be reviewing? • How do we build that into the agenda so we it is standard practice? • Do we have a core group on our team that reviews that data prior to the monthly team meeting?

30. Data-Based Decision Making • Determine what questions you want to answer • Determine what data will help to answer questions • Determine the simplest way to get data • Put system in place to collect data • Analyze data to answer questions Focus on both Academic and Social Outcomes

32. 1. Determine what questions you want to answer Examples Can we predict problems/success? When/where/who? Possible “function” of problem behavior? Who needs targeted or intensive academic supports? What environmental changes/supports are needed?

35. 2. Determine what data will help to answer questions • Existing data set(s) • Current data collection • Additional / new data • Confidence in accuracy? • Complete picture?

38. 3. Determine the simplest way to get data • Agreement on definitions • Standard forms / process • Frequency of collection • Target “Multi-purpose” data/use Train ALL staff on use & provide on-going TA

41. 4. Put system in place to collect data • Build on existing systems • Add components over time • Central entry point • Electronic

44. 5. Analyze data to answer questions • Trends • Instruction & supports in place/not in-place • Pre/post “big outcomes” • Comparisons (norm / local) • Relative growth • Absolute growth

47. By Location

48. By Behavior

49. By Student

50. By # of Referrals