Workshop Agenda

1. Classroom Management: Identifying & Teaching Student Replacement Behaviors Jim Wrightwww.interventioncentral.org

2. RIOT/ICEL Framework to Organize Student Data • 5-Step Process for Defining Problem Behaviors • Common Pitfalls in Behavior Interventions • Internet Resources Workshop Agenda

3. RIOT/ICEL Framework: Organizing Information to Better Identify Student Behavioral & Academic Problems

4. Evaluation. “the process of using information collected through assessment to make decisions or reach conclusions.” (Hosp, 2008; p. 364). Example: A student can be evaluated for ability to ‘comply with teacher requests’ by collecting information using various sources (e.g., direct observation, teacher and student interview, teacher behavior log, Daily Behavior Report Card, etc.), comparing those results to peer norms or developmental expectations and making a decision about whether the student’s current performance is acceptable. Data Collection: Defining Terms Assessment. “the process of collecting information about the characteristics of persons or objects by measuring them. ” (Hosp, 2008; p. 364). Example: The construct ‘complying with teacher requests’ can be assessed using various measurements, including direct observation, teacher and student interview, teacher behavior log, Daily Behavior Report Card, etc. Measurement. “the process of applying numbers to the characteristics of objects or people in a systematic way” (Hosp, 2008; p. 364). Example: Frequency counts can be used to measure the rate of student behaviors that are brief in duration and have a clear onset and end point.

6. RIOT/ICEL Framework Sources of Information • Review (of records) • Interview • Observation • Test Focus of Assessment • Instruction • Curriculum • Environment • Learner

7. RIOT/ICEL Definition • The RIOT/ICEL matrix is an assessment guide to help schools efficiently to decide what relevant information to collect on student academic performance and behavior—and also how to organize that information to identify probable reasons why the student is not experiencing academic or behavioral success.   • The RIOT/ICEL matrix is not itself a data collection instrument. Instead, it is an organizing framework, or heuristic, that increases schools’ confidence both in the quality of the data that they collect and the findings that emerge from the data.

8. RIOT: Sources of Information • Select Multiple Sources of Information: RIOT (Review, Interview, Observation, Test). The top horizontal row of the RIOT/ICEL table includes four potential sources of student information: Review, Interview, Observation, and Test (RIOT). Schools should attempt to collect information from a range of sources to control for potential bias from any one source.

9. Select Multiple Sources of Information: RIOT (Review, Interview, Observation, Test) • Review. This category consists of past or present records collected on the student. Obvious examples include report cards, office disciplinary referral data, state test results, and attendance records. Less obvious examples include student work samples, physical products of teacher interventions (e.g., a sticker chart used to reward positive student behaviors), and emails sent by a teacher to a parent detailing concerns about a student’s study and organizational skills.

10. Select Multiple Sources of Information: RIOT (Review, Interview, Observation, Test) • Interview. Interviews can be conducted face-to-face, via telephone, or even through email correspondence. Interviews can also be structured (that is, using a pre-determined series of questions) or follow an open-ended format, with questions guided by information supplied by the respondent. Interview targets can include those teachers, paraprofessionals, administrators, and support staff in the school setting who have worked with or had interactions with the student in the present or past. Prospective interview candidates can also consist of parents and other relatives of the student as well as the student himself or herself.

11. Select Multiple Sources of Information: RIOT (Review, Interview, Observation, Test) • Observation. Direct observation of the student’s academic skills, study and organizational strategies, degree of attentional focus, and general conduct can be a useful channel of information. Observations can be more structured (e.g., tallying the frequency of call-outs or calculating the percentage of on-task intervals during a class period) or less structured (e.g., observing a student and writing a running narrative of the observed events).

12. Select Multiple Sources of Information: RIOT (Review, Interview, Observation, Test) • Test. Testing can be thought of as a structured and standardized observation of the student that is intended to test certain hypotheses about why the student might be struggling and what school supports would logically benefit the student (Christ, 2008). An example of testing may be a student being administered a math computation CBM probe or an Early Math Fluency probe.

13. Formal Tests: Only One Source of Student Assessment Information “Tests are often overused and misunderstood in and out of the field of school psychology. When necessary, analog [i.e., test] observations can be used to test relevant hypotheses within controlled conditions. Testing is a highly standardized form of observation. ….The only reason to administer a test is to answer well-specified questions and examine well-specified hypotheses. It is best practice to identify and make explicit the most relevant questions before assessment begins. …The process of assessment should follow these questions. The questions should not follow assessment. “ p.170 Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 159-176). Bethesda, MD: National Association of School Psychologists.

14. ICEL: Factors Impacting Student Learning • Investigate Multiple Factors Affecting Student Learning: ICEL (Instruction, Curriculum, Environment, Learner). The leftmost vertical column of the RIO/ICEL table includes four key domains of learning to be assessed: Instruction, Curriculum, Environment, and Learner (ICEL). A common mistake that schools often make is to assume that student learning problems exist primarily in the learner and to underestimate the degree to which teacher instructional strategies, curriculum demands, and environmental influences impact the learner’s academic performance. The ICEL elements ensure that a full range of relevant explanations for student problems are examined.

15. Investigate Multiple Factors Affecting Student Learning: ICEL (Instruction, Curriculum, Environment, Learner) • Instruction. The purpose of investigating the ‘instruction’ domain is to uncover any instructional practices that either help the student to learn more effectively or interfere with that student’s learning. More obvious instructional questions to investigate would be whether specific teaching strategies for activating prior knowledge better prepare the student to master new information or whether a student benefits optimally from the large-group lecture format that is often used in a classroom. A less obvious example of an instructional question would be whether a particular student learns better through teacher-delivered or self-directed, computer-administered instruction.

16. Investigate Multiple Factors Affecting Student Learning: ICEL (Instruction, Curriculum, Environment, Learner) • Curriculum. ‘Curriculum’ represents the full set of academic skills that a student is expected to have mastered in a specific academic area at a given point in time. To adequately evaluate a student’s acquisition of academic skills, of course, the educator must (1) know the school’s curriculum (and related state academic performance standards), (2) be able to inventory the specific academic skills that the student currently possesses, and then (3) identify gaps between curriculum expectations and actual student skills. (This process of uncovering student academic skill gaps is sometimes referred to as ‘instructional’ or ‘analytic’ assessment.)

17. Investigate Multiple Factors Affecting Student Learning: ICEL (Instruction, Curriculum, Environment, Learner) • Environment. The ‘environment’ includes any factors in the student’s school, community, or home surroundings that can directly enable their academic success or hinder that success. Obvious questions about environmental factors that impact learning include whether a student’s educational performance is better or worse in the presence of certain peers and whether having additional adult supervision during a study hall results in higher student work productivity. Less obvious questions about the learning environment include whether a student has a setting at home that is conducive to completing homework or whether chaotic hallway conditions are delaying that student’s transitioning between classes and therefore reducing available learning time.

18. Investigate Multiple Factors Affecting Student Learning: ICEL (Instruction, Curriculum, Environment, Learner) • Learner. While the student is at the center of any questions of instruction, curriculum, and [learning] environment, the ‘learner’ domain includes those qualities of the student that represent their unique capacities and traits. More obvious examples of questions that relate to the learner include investigating whether a student has stable and high rates of inattention across different classrooms or evaluating the efficiency of a student’s study habits and test-taking skills. A less obvious example of a question that relates to the learner is whether a student harbors a low sense of self-efficacy in mathematics that is interfering with that learner’s willingness to put appropriate effort into math courses.

20. The teacher collects several student math computation worksheet samples to document work completion and accuracy. • Data Source: Review • Focus Areas: Curriculum

21. The student’s parent tells the teacher that her son’s reading grades and attitude toward reading dropped suddenly in Gr 4. • Data Source: Interview • Focus: Curriculum, Learner

22. An observer monitors the student’s attention on an independent writing assignment—and later analyzes the work’s quality and completeness. • Data Sources: Observation, Review • Focus Areas: Curriculum, Environment, Learner

23. A student is given a timed math worksheet to complete. She is then given another timed worksheet & offered a reward if she improves. • Data Source: Review, Test • Focus Areas: Curriculum, Learner

24. Comments from several past report cards describe the student as preferring to socialize rather than work during small-group activities. • Data Source: Review • Focus Areas: Instruction, Environment

25. Activity: Use the RIOT/ICEL Framework • Discussion: How can this framework be used in your school to improve the quality of the data that you collect on student?

26. Defining Student Problem Behaviors: A Key to Identifying Effective Interventions Jim Wrightwww.interventioncentral.org

27. Defining Problem Student Behaviors… • Define the problem behavior in clear, observable, measurable terms (Batsche et al., 2008; Upah, 2008). Write a clear description of the problem behavior. Avoid vague problem identification statements such as “The student is disruptive.” A well-written problem definition should include three parts: • Conditions. The condition(s) under which the problem is likely to occur • Problem Description. A specific description of the problem behavior • Contextual information. Information about the frequency, intensity, duration, or other dimension(s) of the behavior that provide a context for estimating the degree to which the behavior presents a problem in the setting(s) in which it occurs.

29. Defining Problem Student Behaviors… • Develop examples and non-examples of the problem behavior (Upah, 2008). Writing both examples and non-examples of the problem behavior helps to resolve uncertainty about when the student’s conduct should be classified as a problem behavior. Examples should include the most frequent or typical instances of the student problem behavior. Non-examples should include any behaviors that are acceptable conduct but might possibly be confused with the problem behavior.

31. Defining Problem Student Behaviors… • Write a behavior hypothesis statement (Batsche et al., 2008; Upah, 2008). The next step in problem-solving is to develop a hypothesis about why the student is engaging in an undesirable behavior or not engaging in a desired behavior. Teachers can gain information to develop a hypothesis through direct observation, student interview, review of student work products, and other sources. The behavior hypothesis statement is important because (a) it can be tested, and (b) it provides guidance on the type(s) of interventions that might benefit the student.

33. Defining Problem Student Behaviors… • Select a replacement behavior (Batsche et al., 2008). Behavioral interventions should be focused on increasing student skills and capacities, not simply on suppressing problem behaviors. By selecting a positive behavioral goal that is an appropriate replacement for the student’s original problem behavior, the teacher reframes the student concern in a manner that allows for more effective intervention planning.

35. Defining Problem Student Behaviors… • Write a prediction statement (Batsche et al., 2008; Upah, 2008). The prediction statement proposes a strategy (intervention) that is predicted to improve the problem behavior. The importance of the prediction statement is that it spells out specifically the expected outcome if the strategy is successful. The formula for writing a prediction statement is to state that if the proposed strategy (‘Specific Action’) is adopted, then the rate of problem behavior is expected to decrease or increase in the desired direction.

38. Defining Academic Problems: Get It Right and Interventions Are More Likely to Be EffectiveJim Wrightwww.interventioncentral.org

39. Defining Academic Problems: Recommended Steps • Be knowledgeable of the school academic curriculum and key student academic skills that are taught. The teacher should have a good survey-level knowledge of the key academic skills outlined in the school’s curriculum—for the grade level of their classroom as well as earlier grade levels. If the curriculum alone is not adequate for describing a student’s academic deficit, the instructor can make use of research-based definitions or complete a task analysis to further define the academic problem area. Here are guidelines for consulting curriculum and research-based definitions and for conducting a task analysis for more global skills.

40. Defining Academic Problems: Recommended Steps Curriculum. The teacher can review the school’s curriculum and related documents (e.g., score-and-sequence charts; curriculum maps) to select specific academic skill or performance goals. First, determine the approximate grade or level in the curriculum that matches the student’s skills. Then, review the curriculum at that alternate grade level to find appropriate descriptions of the student‘s relevant academic deficit. For example, a second-grade student had limited phonemic awareness. The student was not able accurately to deconstruct a spoken word into its component sound-units, or phonemes. In the school’s curriculum, children were expected to attain proficiency in phonemic awareness by the close of grade 1. The teacher went ‘off level’ to review the grade 1 curriculum and found a specific description of phonemic awareness that she could use as a starting point in defining the student’s skill deficit.

41. Defining Academic Problems: Recommended Steps Research-Based Skill Definitions. Even when a school’s curriculum identifies key skills, schools may find it useful to corroborate or elaborate those skill definitions by reviewing alternative definitions published in research journals or other trusted sources. For example, a student had delays in solving quadratic equations. The math instructor found that the school’s math curriculum did not provide a detailed description of the skills required to successfully complete quadratic equations. So the teacher reviewed the National Mathematics Advisory Panel report (Fennell et al., 2008) and found a detailed description of component skills for solving quadratic equations. By combining the skill definitions from the school curriculum with the more detailed descriptions taken from the research-based document, the teacher could better pinpoint the student’s academic deficit in specific terms.

42. Defining Academic Problems: Recommended Steps Task Analysis. Students may possess deficits in more global ‘academic enabling’ skills that are essential for academic success. Teachers can complete an task analysis of the relevant skill by breaking it down into a checklist of constituent subskills. An instructor can use the resulting checklist to verify that the student can or cannot perform each of the subskills that make up the global ‘academic enabling’ skill.For example, teachers at a middle school noted that many of their students seemed to have poor ‘organization’ skills. Those instructors conducted a task analysis and determined that--in their classrooms--the essential subskills of ‘student organization’ included (a) arriving to class on time; (b) bringing work materials to class; (c) following teacher directions in a timely manner; (d) knowing how to request teacher assistance when needed; and (e) having an uncluttered desk with only essential work materials.

43. Defining Academic Problems: Recommended Steps • Describe the academic problem in specific, skill-based terms (Batsche et al., 2008; Upah, 2008). Write a clear, brief description of the academic skill or performance deficit that focuses on a specific skill or performance area. Here are sample problem-identification statements: • John reads aloud from grade-appropriate text much more slowly than his classmates. • Ann lacks proficiency with multiplication math problems (double-digit times double-digit with no regrouping). • Tye does not turn in homework assignments. • Angela produces limited text on in-class writing assignments.

44. Defining Academic Problems: Recommended Steps • Develop a fuller description of the academic problem to provide a meaningful instructional context. When the teacher has described the student’s academic problem, the next step is to expand the problem definition to put it into a meaningful context. This expanded definition includes information about the conditions under which the academic problem is observed and typical or expected level of performance. • Conditions. Describe the environmental conditions or task demands in place when the academic problem is observed. • Problem Description. Describe the actual observable academic behavior in which the student is engaged. Include rate, accuracy, or other quantitative information of student performance. • Typical or Expected Level of Performance. Provide a typical or expected performance criterion for this skill or behavior. Typical or expected academic performance can be calculated using a variety of sources,

46. Defining Academic Problems: Recommended Steps • Develop a hypothesis statement to explain the academic skill or performance problem. The hypothesis states the assumed reason(s) or cause(s) for the student’s academic problems. Once it has been developed, the hypothesis statement acts as a compass needle, pointing toward interventions that most logically address the student academic problems.

48. Defining Student Problem Behaviors: Team Activity • As a team: • Consider the 4-step process described in this workshop for defining student academic concerns. • How would your school incorporate this approach into your RTI problem-solving model?