Individual Student Systems Evaluation Tool ISSET

1. Individual Student Systems Evaluation ToolISSET Educational & Community Supports University of Oregon January 2013

2. Agenda • Overview of ISSET • Preparing for and Conducting the ISSET • ISSET Interviews • ISSET Scoring Guide • Summarizing ISSET Results • MATT

3. School-Wide Positive Behavior Support Intensive Interventions Specialized Individualized Systems for Students with High-Risk Behavior ~5% Targeted Interventions Specialized Group Systems for Students with At-Risk Behavior Universal Interventions School-/Classroom- Wide Systems for All Students, Staff, & Settings ~15% ~80% of Students

4. Important Outcomes • Effects of the system • Student behavior • Staff behavior • Overall system • Process (fidelity) • Fidelity of specific interventions • Fidelity of the system

5. Tools for Assessing Fidelity • Self Assessment • Monitoring progress over time • Developing an action plan • External Evaluation • Monitoring progress over time • Useful when outside opinion is warranted • Research • High-Level Evaluation

6. Existing Tools for Assessing Process • Universal Component of SWPBS • External • School-wide Evaluation Tool (SET) • Self Assessment • Team implementation Checklist (TIC) • Benchmarks of Quality (BoQ) • Self-Assessment Survey (SAS) • Targeted & Intensive Components of SWPBS • External • Individual Student Systems Evaluation Tool (ISSET) • Self Assessment • Self-Assessment Survey (SAS) • Benchmarks of Advanced Tiers (BAT) • Monitoring Advanced Tiers Tool

7. https://www.pbisassessment.org/ Overview of Tools/Instruments

8. ISSET & MATT SYSTEMS Data Practices • For each feature: • What practices are implemented? • What systems are used? • What outcomes are assessed? • Key Features • Foundations: What needs to be in place? • Targeted interventions • Intensive interventions

9. ISSET Steps • Step 1 Administrator Interview • Step 2 Tier 2 & 3 Team Leader Interview • Possibly meet with most commonly used Tier 2 Coordinator • Step 3 Staff Interviews • Step 4 Summarize Interview Responses • Step 5 Review Permanent Products • Step 6 Complete Scoring Guide

10. Scheduling the ISSET • Contact the principal and team leader(s) to schedule the ISSET (2-3 hours) • Email confirmation of time and a list of required materials needed for review at the interview (Form email will be provided) • Need 10-30 minutes with Administrator and 30-1 hour with Team Leader • Two data sources used to score: • Interviews • Review of permanent products/documented procedures

11. Administrator Interview School _________________________ Date of Completion ______/______/______ ADMINISTRATOR INTERVIEW QUESTIONS First let’s talk about your team. 1. Do you have a team of two or more people at your school that receives requests for assistance, develops support plans and monitors intervention results? (B1) Yes No if no, skip to question 2 How often does this team meet? (B2) _______________ How often are you able to attend those meetings? (A1) _____________ (0= 0-24%, 1= 25%-49% of meetings, 2= 50% or more) 2. Is there someone identified as the coordinator of function-based support for all the students in your school? (A4) Yes No If yes, is that person on your school staff? Yes No If yes, who is it? ______________________ 3. Is there a person(s) on, or available to, the school behavior support team who is trained to conduct functional behavioral assessments and lead a team in the use of functional behavioral assessment information to develop behavior support plans? (A5) Yes No 4. Is the number of students receiving targeted or intensive interventions and their overall progress reported to faculty? (D3) Yes No If yes, how often? _____________ Before ending the administrator interview check the list of permanent products on page 4 and collect and/or review any that he/she has available.

12. Team Leader Interview BEHAVIOR SUPPORT TEAM LEADER INTERVIEW QUESTIONS Repeat the introduction if needed. Let’s talk about your Behavior Support Team and the processes associated with the team. 1) How many people are on the team and what are their roles in your school? 2) Is there a process for requesting assistance from the team for a student that may need extra behavior support? (C3) Yes No If yes, what is the process? __________________________________________________________ (Get copy of Request for Assistance forms) 3) Have at least 4 of 5 of the most recent requests for assistance received support (meeting held or had an FBA conducted/started and/or intervention planned) within 10 school days of the request? (C4) Yes No 4) Do any BST team members attend annual professional development training in targeted and intensive interventions? (B3) Yes No If so, how many? 5) When the team meets to complete an FBA, does the team include individuals with knowledge about: The student? Yes No The setting in which the problem behavior usually occurs? Yes No An understanding of functional behavior assessment and building behavior support plans linked to the functional behavior assessment? (G3) Yes No What about when the team meets to complete a BSP, does the team include individuals with knowledge about: The student? Yes No The setting in which the problem behavior usually occurs? Yes No An understanding of functional behavior assessment and building behavior support plans linked to the functional behavior assessment? (H3) Yes No

13. Team Leader Interview Now let’s talk about Targeted and Intensive Interventions: A targeted intervention is defined as an intervention designed for students who are not responding to universal interventions. These interventions are implemented in a similar manner for all students receiving it. Eligible students might need additional social, academic, and/or organizational support. An intensive individual intervention is defined as an intervention for students who need individualized behavior support. 6) Does your school have any targeted interventions available to students who might need them? (E1) Yes No If yes, what are they? __________________________________________________ Do you have anything written down that describes these interventions? Yes No (If no, skip targeted interventions questions 7, 9, 15-19 ) Which one is most commonly used? ____________________________ Use their name for the program for the interview. 7) In reference to (the most commonly used targeted intervention) (E1) Get written information about this intervention to score for documentation. Is the intervention organized around SW expectations &/or academic goals? Yes No Is the intervention available for students to enter at any time? Yes No When students are identified for the intervention do they begin the intervention within 3 school days of determination? Yes No Do student assessment/progress data prompt the need for modification? Yes No Does the intervention include structured prompts for what to do during the day (relevant situations)? Yes No Do students on the intervention receive positive feedback from staff on a daily basis? Yes No Does the intervention include a weekly check-in with the student’s family? Yes No Are orientation materials available for students entering the intervention?? Yes No Are orientation materials available for staff/substitutes/volunteers who have students using the intervention? Yes No Do students using the intervention have daily opportunities to use their new skills? Yes No If yes, tell me about that process. Give me an example.

14. Team Leader Interview 8) Who manages, coordinates and monitors the (most commonly used targeted intervention, their program name)? (E3) Does this person have formally allocated time apart from their other job responsibilities for this coordination? Yes No Is that documented in their job description? Yes No If yes, ask to see the job description. (If yes, ask for a copy) 9) Are office discipline referral data reviewed and used to identify students who currently are not receiving an intervention but who might benefit from a targeted or intensive intervention? (C1) Yes No If yes, how often? ______________ 10) Do you have a process for determining if a student begins a targeted intervention? (C5) Yes No If yes, tell me about the process. How about a process for determining if a student begins an intensive intervention? Yes No If yes, tell me about the process. 11) Do you monitor the outcomes for students receiving targeted or intensive behavior support? (D1) Yes No If yes, how often? ____________ 12) If a behavior support plan focuses on a problem behavior that places a student or others at physical risk, does the team have a strategy for identifying risk and implementing a safety plan if needed? (H6) Yes No (see BSPs for documentation) 13) Does a team (school-wide or BST) review data across all students across the school (e.g., ODR patterns, formative evaluation) to determine whether the school needs to develop or modify their existing targeted and intensive interventions? (D2) Yes No If yes, how often? _________

15. Team Leader Interview 14) Does the school-wide team monitor the number of students on targeted and/or individual supports? (A3) Yes No If yes, how often? ____________ 15) Once it has been determined that a student needs targeted and intensive support, is there a process for notifying and including family members? (D5) Yes No If yes, is the process written down? ____________ Once an intervention has been implemented, do parents continue to receive information about their child’s progress? Yes No 16) Do you notify staff members involved with those students? (D4) Yes No If yes, is the process written down? Yes No Once an intervention has been implemented, does staff continue to receive information about their student’s progress? Yes No 17) How are decisions made about how to monitor and when to modify or discontinue a targeted intervention? (F1) _______________________________________________________________ Are there written decision rules or a process for how this happens? Yes No (ask to review them) 18) How much time does the (most commonly used targeted intervention) require per day for instructional staff to implement/monitor (this refers to instructional staff who implement the program with students during the school day)? (E4) ______________________ 19) How do you train or orient staff, volunteers, substitutes, students and families about the (most commonly used targeted intervention)? (E2) _______________________________ Do you have anything written down that describes this orientation or training? (E2) Yes No (ask to review it) 20) Does the team collect and review data at least once a year to evaluate that the (most commonly used targeted intervention) is being implemented as planned (with fidelity)? (F2) Yes No Before ending the Behavior Support Team Leader interview check the list of permanent products on page 4 and collect and/or review any that he/she has available.

16. Permanent Products • 19 of the 35 questions require a review of permanent products • Recommend having the school copy all materials in advance and remove any identifying, confidential data by blacking it out.

17. Foundations • Practices • Is there a team that meets regularly and uses data? • Are there multiple systems for identifying students? • Do students and teachers receive assistance quickly? • Systems • Commitment • Universal level of SWPBS in place? • Does team have access to expertise in function-based supports? • Does administrator attend team meetings? • Are new faculty and staff trained? • Does the behavior support team consist of key roles? • Data • Are outcomes monitored for and across students? • Are data used for identifying interventions? • Do teachers see data? • Are family members involved and do they see outcomes?

25. Targeted Interventions • Practices • Is there at least one targeted intervention? • Are interventions efficient? • Systems • Are interventions documented? • Are all faculty and staff familiar with intervention? • Is there a coordinator? • Data • Are there data-based decision rules for all group interventions? • Is fidelity monitored?

27. Targeted Intervention Implementation Feature Checklist ISSET Evaluation Question E1. (pg. 19)

30. Intensive Interventions • Practices • Are FBAs technically adequate? • Are interventions linked to the FBA? • Do interventions contain key components? • Systems • Are relevant people involved in developing intervention? • Data • Are outcomes assessed? • Is fidelity monitored?

32. FBA Includes Intensive Individualized Interventions Features Checklist Use for scoring ISSET Part III, Questions G 1-2, H 1-2 & 4-6, I 1-2.

36. BSP Includes

37. Scoring Summary

38. Summarizing at School Level

39. Summarizing at School Level C D E F G

40. Action Planning

41. Annual ISSET Results

42. Next Steps • Score 4 FBA/BIPs and bring to follow-up training • 1-2 hour training before shadowing ISSET in actual building • When shadowing everyone will score the questions which will be followed up with debrief by person who led the ISSET

43. Wrap-Up • Cost • Expertise • Time • Training • IOA • Returns • External review that validates (or not) the self-assessment • Highly detailed comprehensive picture of fidelity • What is school doing well and what do we need to work on • Can be used for action planning