Understand and Explore Assessment Concepts Relative to UDL

1. Understand and Explore Assessment Concepts Relative to UDL UDL Institute CAST Tracey E. Hall 2013

2. Operational Definition of Assessment The gathering of information about a learner from his or her performance in a variety of tasks, subjects, and learning contexts to determine abilities and knowledge for the purpose of making educational decisions (Salvia & Ysseldyke, 2009).

3. Types of Assessment: • SummativeEvaluation: Measurement to determine subject performance at the end of a specific grade level, unit, year, or instructional episode. Mastery Measurement • Formative Evaluation:Assessment of progress toward a long-term goal or major objective, used primarily to diagnose what students have learned in order to plan further instruction (i.e., the on-going assessment of progress toward an objective).

4. Types of Assessment: • SummativeEvaluation: Measurement to determine subject performance at the end of a specific grade level, unit, year, or instructional episode. Mastery Measurement • Formative Evaluation:Assessment of progress toward a long-term goal or major objective, used primarily to diagnose what students have learned in order to plan further instruction(i.e., the on-going assessment of progress toward an objective).

5. CAST’s latest definition Formative assessment is an iterative process embedded throughout instruction that is used by educators and learners.   Educators (1) collect and analyze multiple measures of learner performance, and (2) use these data to inform and adjust instruction.   Learners (1) interact with measures that approximate classroom instruction; and (2) with guidance, use assessment data to better self-regulate learning.   Formative assessment is used to address the variability of learners, improve the achievement of intended instructional goals by all, and build educator and learner expertise.

6. What is a test supposed to do? Educational tests are: • Indirect measure of a construct or a set of closely related constructs. • Constructs are not observable • A test provides a sample of observable behaviors believed to be the product of the intended construct • Observable behaviors are used to make an inference about the construct

7. Accessing Assessment 2 perspectives • Access to the test • Access to the construct

8. Accessing the Construct

9. Test construct • Test construct refers to the concept or the characteristic that a test is designed to measure. e.g., In a mathematics assessment, an item designed to evaluate students' ability to show equivalence has a test construct of numerical equivalency. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999).

10. Components of an assessment item Tests are designed to: • Present information to stimulate the construct. • Interact with the information as the construct is operating • Produce a response which becomes a physical product

11. Components of an assessment item Tests are designed to: • Present information to stimulate the construct. • Interact with the information as the construct is operating • Produce a response which becomes a physical product

12. How Items Function A Quantita- tive Score Inference to Construct Interact with Construct Present Information Produce Response Interact with Apply View Product Measure of Stimuli Construct of Construct Construct If a challenge or barrier exists at any of these levels, inference about the construct will fail. Michael Russell 2011

14. When there are construct irrelevant impediments, distractions, or barriers in the assessment methods it is essential to provide scaffolds, supports or accommodations in order to improve accuracy and validity, especially for individuals with • disabilities.

15. Keep in mind, using those same scaffolds, supports or accommodations inappropriately – where they affect the construct relevant demands of the assessment – is likely to invalidate the measure.

16. Construct relevant • Construct relevant refers to the factors (e.g., mode of presentation or response) that are relevant (related) to the construct that the test is intended to measure. • Identifying the symbols for a math calculation problem is a construct relevant change to the assessment item.

17. Construct Irrelevant • Construct irrelevance is the extent to which test scores are influenced by factors (e.g., mode of presentation or response) that are irrelevant (not related) to the construct that the test is intended to measure. • For Elisa, a student with severe cognitive disabilities, changing the font size and increasing sizing of images helped her to see the assessment item for counting objects and doing so was construct irrelevant to assessing ability to count.

18. Accommodations, scaffolds, and supports should not be exclusive to the instructional environment but should be a part of the whole instructional episode including assessment. • The preservation of the distinction between what is construct relevant and what is construct irrelevant is essential when making decisions about accommodations, scaffolds, and supports in the assessment environment.

20. Envision taking a test:

21. Envision taking a test: Ability to concentrate on the problem is one of the constituent parts of the successful mental work

22. Envision taking a test: Many students in the United States struggle in the areas of reading and writing, with difficulties emerging early and continuing into the secondary school years and beyond (National Center for Education Statistics, 2002, 2003, 2005, 2007). As student performance levels diverge, it becomes increasingly necessary to provide greater differentiation of instruction. Two approaches are particularly suited for helping teachers and practitioners to differentiate instruction.

23. Many students in the United States struggle in the areas of reading and writing, with difficulties emerging early and continuing into the secondary school years and beyond (National Center for Education Statistics, 2002, 2003, 2005, 2007). As student performance levels diverge, it becomes increasingly necessary to provide greater differentiation of instruction. Two approaches are particularly suited for helping teachers and practitioners to differentiate instruction.

24. Adapted presentation • Flexible font size (magnification) • Color contrasts • Format, White space Easy in a digital environment, no item interference. No impact on measuring the construct. Only change is how the item is being displayed. “These are no brainers in a computer environment” Mike Russell 09. On paper, more difficult to make changes such as these.., for administration not student – lots of different papers

25. Alternate representations: in digital environment – relatively simple water H2O agua

26. Multiple Representations of Information

27. Alternate means for Expression and Action: also easily accomplished in a digital environment

28. Engagement Example: Options for to read and respond. Students are engaged by choice (b) often select based on interest or prior knowledge (c) perform better

29. Limitations of Accommodationsas Retrofits • Marginal efficacyConsider: • Extended time • Read-aloud accommodation vs. independent reading and strategic test-taking • May invalidate measurementsConsider: • Use of read-aloud for decoding task • Use of calculator for arithmetic task Thus it is often difficult to level the playing field with accommodations, especially if the playing fields was initially un-level for all students.

30. Some Solutions: Consider Student Diversity During Test Development& Delivery • During Test Development • Ensure narrowly-defined constructs • Conduct bias review early • Ensure items are suitable for appropriate delivery to all students • During Test Delivery • Provide multiple means of recognition, expression & engagement • Ensure matching of classroom and assessment supports

31. Broadly-Define Constructs / Bias Review: Assumptions of Prior Knowledge The cartoon above depicts United States frustration with A)the Good Neighbor policy B)Dollar Diplomacy C)the Spanish-American War D)the Bay of Pigs invasion • 12th Grade NAEPU.S. History Item

32. Broadly-Defined Constructs / Bias Review:Culture and Native Language Assumptions Circle the picture that starts with “B” • 1st Grade Phonological Awareness Tasks Circle the picture that starts with “T”

33. Ensuring Suitable Delivery: Invalidation “Sarah must determine the diameter of a circle that she has drawn, but doesn’t know the value of П. Can she still do it?” Consider effect of read-aloud accommodation on the following test question: • 8th Grade Geometry: Word Problem vs. Symbol Recognition

34. MCA-IIGrade 10 Reading Sample Item

36. UDL Considerations for Assessment Development Structure . UDL Considerations for Assessment Development Structure: • Based on the UDL Principles • Research on working memory • Research on eye movement *handout

37. UDL Considerations for Assessment Development • Linguistic Complexity • Information Density • Self-Regulationand Visio-Spatial information

38. Linguistic Complexity Linguistic Complexity (LC) The relationship between the sentence processing mechanism and the available computational resources (Gibson, 1998) • LC-1 Syntax – • LC-2 Simplifying vocabulary • LC-3 Reduced sentence length • LC-4 Language translation • LC-5 Clarify anaphoric references

39. Information Density Information Density (ID) The two critical limitations on handling information in working memory are the (1) small number of pieces of information an individual’s memory can handle and (2) the short duration of time during which information can remain in memory (Sylwester & Choo, 1992). The following modification structures help to reduce working memory load and maximize use of information read when used carefully. • ID-1 Question placement • ID-2 Emphasize key information • ID-3 Chunking or combining tasks (questions or response options) • ID-4 Guide information processing— • ID-5 Contextualizing skills –skill icons • ID-6 Hint • ID-7 Line numbering • ID-8 Passage primer

40.  Self-Regulation and Visio-Spatial information (SR) • Self-regulation is seen by many cognitive researchers as a pivot upon which students’ achievement turns. The structures noted here when used in a non-construct relevant application help to support self-regulation and attention to tasks. • SR-1 Progress map • SR-2 Self check • SR-3 Optional Workspace • SR-4 Skill Icon Preview • SR-5Reduce Reflexive Eye Movement

41. Example Item 1: A Time-Distance Graph

42. Example Item 2: Features of Plant and Animal Cells

43. Example Item 3: The Boiling Points of Two Beakers of Water

44. UDL - Assessment revisions/considerations • Discussion –UDL considerations • Small groups • Start with the task directions • Think UDL access to executive functioning