Facilitating PS/ RtI Capacity: Tools , Skills, and Strategies for Practitioners

1. Facilitating PS/RtI Capacity: Tools, Skills, and Strategies for Practitioners NASP 2013 Mini-Skills Presentation February 14th, 2013 Amanda March Amber Brundage Clark Dorman Jose Castillo Kevin Stockslager University of South Florida

2. Purpose • To enhance practitioners’ understanding of empirically supported systems-change procedures, detailing systems-theory and principles identified as critical for success of PS/RtI initiatives.

3. Learning Objectives • Discussion of critical components of PS-RtIimplementation &scale-up • Review of a comprehensive three-phase systems-change model • Consensus • Infrastructure • Implementation • Presentation, discussion, &practice of various tools to evaluate &progress monitor PS/RtI practices • Participants will leave with knowledge, skills, &tools to facilitate PS/RtI implementation &evaluation in their local settings

4. Advanced Organizer • PS/RtI& MTSS • MTSS and Program Evaluation in the Schools • Example Tools & Practice Activities • Beliefs on RtI Scale • Perceptions of RtI Skills Survey – Revised • Tier I and II Critical Components Checklist • Discussion • Resources & References

5. Introductions • Who are we? • Who are you? • Clicker Activities

6. PS/RtI& MTSS

7. PS/RtI Model of Service Delivery • RtI is the practice of providing high quality instruction matched to student needs and utilizing data to make educational decisions about students (Batsche et al., 2005) • Components of a PS/RtI Model • Integrated multi-tier model of service delivery • Problem-solving method • Integrated data collection and assessment system

8. MTSS Defined • Evidence-basedmodel of educating students that uses data-based problem-solving to integrate academic and behavioral instruction and interventions; • Integrated instruction/interventions are delivered to students in varying intensities (multiple tiers) based on student need; • Need-driven decision making seeks to ensure the allocation of resources (district, school, classroom) are based on student need at the appropriate levels to accelerate performance of allstudents to achieve and/or exceed proficiency.

9. Context Florida’s MTSS + ________________________________________________________________ =

10. Changing Systems

11. Multi-Tiered System of Support Model in Education Systems Approach: • System - “The orderly combination of two or more individuals whose interaction is intended to produce a desired outcome.”(Curtis, Castillo, & Cohen, 2008) • Principles of Systems Change: • Shared Mission, Beliefs, and Values • Key Stakeholder Involvement • Effective, Committed Leadership • Systems Perspective – “Big” Picture • Structured Planning and Problem-Solving • Data-based decision making • Knowledge and skills to build capacity through professional development

12. Three-Phase Change Model

13. Florida’s Change Model Consensus Infrastructure Implementation

14. Three Phase Change Model Consensus Belief is shared Vision is agreed upon Implementation requirements understood Infrastructure Development Regulations Training/Technical Assistance, Professional Development Model (e.g., Standard Protocol) Tier I and II intervention systems E.g., K-3 Academic & Behavioral Support Plan Data Management Technology support Decision-making criteria established Implementation

15. MTSS and Program Evaluation in the Schools

16. PS/RtI Evaluation Tool Technical Assistance Manual - Revised www.floridarti.usf.edu

17. Florida PS/RtI Evaluation Tools • Tools for Progress Monitoring PS/RtI • Self-Assessment of Problem-Solving Implementation (SAPSI) • Tools for Examining Consensus Development • Beliefs on RtI Scale • Perceptions of Practice Survey • Tools for Examining Infrastructure Development • Perceptions of RtI Skills Survey –Revised • Coaching Evaluation Survey – Revised • Tools for Examining Integrity of PS/RtI • Tier I and II Observation Checklist • Tier I and II Critical Components Checklist • Problem-Solving Team Meeting Checklists – Initial & Follow-Up • Tier III Critical Components Checklist

18. Today’s Featured Tools • Tools for Examining Consensus Development • Beliefs on RtI Scale • Tools for Examining Infrastructure Development • Perceptions of RtI Skills Survey –Revised • Tools for Examining Integrity of PS/RtI • Tier I and II Critical Components Checklist

19. MTSS Evaluation Issues • The data you collect should be driven by the evaluation questionsyou want to answer • Are we implementing MTSS with fidelity? • Do we have the capacity to implement successfully? • Do staff buy into implementing MTSS?

20. Answering Evaluation Questions • Use data to inform evaluation questions • Use data to answer broad/specific questions • Align analysis and data display with evaluation questions • Consider available technology resources to facilitate analyses of data—online administration, automatic analysis, knowledge and skill of personnel

21. Assessing Beliefs Integral to PS/RtI Practices Beliefs on RtI Scale

22. Beliefs of RtI Scale Description and purpose

23. The Beliefs of Educators Research suggests: • Educators beliefs about student learning styles, styles of teaching, and instructional strategies impact their willingness to implement new practices (Fang, 1996; Sparks, 2002) • Educational reform occurs when educators understand the need for change and embrace this need as a moral imperative (Fullan, 2010; Sharratt & Fullan, 2009) • Beliefs of leaders communicated to stakeholders influence the climate for successful implementation of new practices (Sharratt & Fullan, 2009)

24. Description of Survey • Assesses educators beliefs regarding: • Student learning • The role of data in decision-making • Expectations for the effectiveness of instruction • 19 items; 5-point Likert scale • 1= Strongly Disagree …5= Strongly Agree

25. Purpose of Instrument Purpose of the Beliefs on RtI Scale is to measure and inform consensus development in two ways: • Assess impact of professional development on educator beliefs about PS/RtI • Identify commonly held beliefs that will likely facilitate or hinder implementation efforts

26. Beliefs on RtI Scale Administration procedures & scoring

27. Administration procedures-Intended Audience • Who should complete? • SBLT members • Instructional staff • Who should use results? • SBLTs • DBLTs

28. Directions for Administration • Methods for administration/dissemination • Completed individually • Anonymity • Opportunity for questions • Role of school principal—explain the “why” • Role of RtI coach/coordinator/SBLT member • Frequency of use: resources, rationale, recommendations

29. Scoring Two techniques to analyze survey responses: • Mean rating for each item calculated to determine average perceived belief level • Frequency of each response option selected calculated for each item

30. Calculating Item Mean • Overall assessment of reported beliefs of educators within a school/district • Can be done at domain (factor) and/or individual item level • Domain level: examine patterns in reported beliefs regarding • academic ability and performance of SWD • data-based decision-making • functions of core and supplemental instruction • Item level: identify specific beliefs staff report v. beliefs in need of exploration and support

31. Calculating Frequency of Response Options • Provides information on range of belief levels • Can be used to determine what percentage of staff may require little, some, or high levels of support to implement PS/RtI • Informs professional development decisions

32. Beliefs on RtI Scale Interpretation and use of data

35. Interpretation & Use of Data (cont.) • Sharing data with stakeholders: • DBLTs, SBLTs, instructional staff • Use data to: • Develop/adjust consensus-building goals • Design training/coaching activities • Facilitate consensus-building discussions re: rationale for PS/RtI, patterns and changes in beliefs over time

36. Beliefs on RtI Scale Practice Activity

37. Assessing Perceptions of Skills Integral to PS/RtI Practices Perceptions of RtI Skills Survey

38. Perceptions of Skills The likelihood of embracing new practices increases when: • Educators understand the need for the practice • Educators perceive they either have the skills to implement the practice or will be supported in developing required skills

39. Perceptions of RtI Skills Survey Description and purpose

40. Perceptions of Skills—Description and Purpose • Theoretical Background: • Assess educators’ perceptions of skills they possess to implement PS/RtI • Understand perceptions of skills and how perceptions change as function of professional development to facilitate PS/RtI implementation

41. Description of Survey • Assesses skills/amount of support needed for: • Applying PS/RtI practices to academic content • Applying PS/RtI practices to behavior content • Data manipulation and technology use • 50items; 5-point Likert scale • 1= I do not have the skill at all (NS)…5= I am highly skilled in this area and could teach others (VHS)

42. Purpose of Instrument Purpose of the Perceptions of RtI Skills Survey: • Assess impact of professional development • Identify “comfort level” with PS/RtI practices to inform PD; allocate resources

43. Perceptions of RtI Skills Survey Administration procedures & scoring

44. Administration & Scoring • Same procedures as Beliefs on RtI Scale

45. Perceptions of RtI Skills Survey Interpretation and use of data

46. Interpretation & Use of Data • Three domains: • Perceptions of skills applied to academic content • Perceptions of skills applied to behavior content • Perceptions of data manipulation and technology use skills • Three methodologies: • Calculate mean at domain level • Calculate mean at item level • Frequency/percentage of who selected each response option • Identify specific skills/skills sets for PS/support

49. Perceptions of RtI Skills Survey - Revised Practice Activity

50. Implementation Integrity