Alignment of Alternate Assessments-Alternate-Achievement Standards and Academic Content Standards

1. Alignment of Alternate Assessments-Alternate-Achievement Standards and Academic Content Standards Claudia Flowers, Diane Browder, & Shawnee Wakeman UNC Charlotte

2. Assessments From presentation by Sue Rigney National Alternate Assessment Center

3. Types of Alternate Assessments • Alternate Assessments • Based on grade-level academic achievement standards • Full range of content, same degree and pattern, full range of cognitive complexity • Based on alternate achievement standards** • Clear link to the grade-level content standards, but may be reduced in complexity National Alternate Assessment Center

4. Alternate Academic Achievement Standards • Must be aligned to content standards (include knowledge and skills that link to grade-level expectations) • Promote access to the general curriculum • Reflect professional judgment of the highest learning standards possible National Alternate Assessment Center

5. Alternate Academic Achievement Standards (cont.) • Grade-level content may be reduced in complexity • For each grade level, define one or more alternate achievement standards for proficiency • Should be defined in a way that supports individual growth because of their linkage to different content across grades National Alternate Assessment Center

6. Content Standards Assessments State B Classroom Instruction Assessments Teacher Student Learning Student Standards-Based System A National Alternate Assessment Center

7. Alignment Procedures • Some holistic methods • Quantitative methods* • Webb’s Method • Surveys of Enacted Curriculum (SEC) • Andrew Porter, John Smithson, and other researchers National Alternate Assessment Center

8. Webb’s Alignment Method • Categorical concurrence • Range-of-knowledge • Balance of representation • Depth-of-knowledge • Sources of Challenge (Bias)*** Content Standards Alternate Assessment National Alternate Assessment Center

9. Math Measurement Content Standard 2 Content Standard n ..... .... .... Obj 1 Obj 2 Obj 3 Obj n Obj 1 Obj 2 Obj 3 Obj n .... Obj 1 Obj 2 Obj 3 Obj n Telling Time Categorical Concurrence Categorical Concurrence At least 6 per category (content standard) Our findings— 77-94% of AA items/task hit 0-67% Categorical Concurrence National Alternate Assessment Center

10. Math Measurement Content Standard 2 Content Standard n ..... .... .... Obj 1 Obj 2 Obj 3 Obj n Obj 1 Obj 2 Obj 3 Obj n .... Obj 1 Obj 2 Obj 3 Obj n Telling Time Range-of-Knowledge Our findings— 0-36% Range-of-Knowledge At least 50% of objectives aligned to item National Alternate Assessment Center

11. Math Measurement Content Standard 2 Content Standard n ..... .... .... Obj 1 Obj 2 Obj 3 Obj n Obj 1 Obj 2 Obj 3 Obj n .... Obj 1 Obj 2 Obj 3 Obj n Telling Time Balance of Representation Our findings— 0-33% Distribution Across Standard National Alternate Assessment Center

12. Depth of Knowledge • Consistency between the cognitive demands of the standards and cognitive demands of assessments • Recall (Level 1), Skill or Concept (Level 2), Strategic Thinking (Level 3) and Extended Thinking (Level 4) • We had a level 0 for prerequisite skills National Alternate Assessment Center

13. Depth of Knowledge • Consistency between standards and assessments indicates alignment of what is elicited from the students is demanding cognitively as what is stated in the standard. • (Webb, p. 5) • Our findings • Most AA had at least one item/task @ 3 or 4 • One of the AA had 33% of items @ or above standard National Alternate Assessment Center

14. How is AA alignment different from General Education? • Interplay between the number of items/tasks on an assessment and the number of content standards (strands) to be covered (performance assessments) • Diverse group to evaluate the alignment—curriculum folks are a most • What do you align? • New dimensions of alignment (e.g., fairness)? • New criteria for each dimension? • What about teacher designed AA? National Alternate Assessment Center

15. Surveys of Enacted Curriculum • Alignment of standards, assessments, instruction, and more • Use a two dimensional common content matrix (Content by Cognitive Demand) National Alternate Assessment Center

16. Example of Matrix Complexity National Alternate Assessment Center

17. Content Mapping National Alternate Assessment Center

18. Alignment Index National Alternate Assessment Center

19. UNC Charlotte Research on Alternate Assessment Alignment • What curricular domains are used? • Are the performance indicators within reading and math aligned with standards for this content? • What type of tasks and contexts are used in alternate assessments that are clearly aligned? • To what extent are states with strong general curriculum focus aligned with grade level content standards? National Alternate Assessment Center

20. Method Obtained alternate assessment information and materials from 41 states in 2001 Used 31 states’ materials that included information on “performance indicators” (assessment items; sample tasks for standards; extended standards) Coded information to find patterns Reference Browder, D., Ahlgrim-Delzell, L., Flowers, C., Karvonen, M. Spooner, F., & Algozzine, R. (2005). How states define alternate assessments. Research and Policy in Developmental Disabilities, 15 (4). Alignment ResearchWhat curricular domains are present in states’ alternate assessments? National Alternate Assessment Center

21. FINDINGSPrior to NCLB, most states’ alternate assessments included academic domains National Alternate Assessment Center

22. Method Selected a representative sample of performance indicators from each of the 31 states Reviewed by researchers in reading and math education (general education) Reference Browder, D., Flowers, C., Ahlgrim-Delzell, L. Karvonen, M. Spooner, F. , & Algozzine, R. (2004). The alignment of alternate assessment content to academic and functional curricula. Journal of Special Education, 37, 211-224. Alignment ResearchDo the performance indicators in states’ AA align with reading and math standards? National Alternate Assessment Center

23. Examples from strongly aligned states Math Compare volumes of more and less Use strategies such as counting, measuring, to determine possible outcomes in problem solving Reading Answer questions related to story Identify pattern in familiar story Examples from weakly aligned states Math Replace rollers in beauty parlor Measure growth of fingernails Reading Show anticipation on roller coaster Attend to visual stimuli Findings Mixed:Some states had strong alignment to academic content; some weak alignment National Alternate Assessment Center

24. Method Content analysis 31 states from 2001 States with clear alignment to academic content compared with states with weak alignment to determine curricular focus Reference Browder, D., Spooner, F., Ahlgrim-Delzell, L., Flowers, C., Karvonen, M., & Algozzine, R. (2004). A content analysis of curricular philosophies in states’ alternate assessment performance indicators. Research and Practice in Severe Disabilities, 28, 165-181. Alignment ResearchWhat type of curriculum is reflected in states’ alternate assessments? National Alternate Assessment Center

25. Frequency of Each Philosophy • Across all 6 states for TASK • 54% academic • 18% functional • 11% social • 4% early childhood • Across all 6 states for CONTEXTS • 63% functional • 25% academic • 9% social • 1% early childhood National Alternate Assessment Center

26. FindingsStates with clear alignment used more academic tasks and contexts National Alternate Assessment Center

27. Method Obtained sample alternate assessments from three states with strong links to academic content Applied criteria for alignment developed by Norman Webb for general education assessments Reviewed using each state’s grade level content standards (reviewed one grade per state) Reference Flowers, C. Browder, D., & Ahlgrim-Delzell, L. (In press). An analysis of three states’ alignment between language arts and mathematics standards and alternate assessments. Exceptional Children. Alignment Research:To what extent do alternate assessments align with grade level content standards? National Alternate Assessment Center

28. FindingsStrong match with grade level standards, but selective use of standards • Overall alignment strong • 78-94% of alternate assessment items in three states could be directly linked with one of their grade level academic content standards for reading & math • Less breadth and depth than recommended for general education assessments • Fewer objectives sampled; fewer items per standard; less balance across objectives than recommended for general education • Depth of knowledge at all levels, but skewed to more basic levels of knowledge National Alternate Assessment Center

29. Research Conclusions • Most states are focused on aligning their alternate assessments with academic content standards • Some of these alternate assessments have clear alignment with academic content; others weak alignment • Even states with strong alignment with grade level content standards face challenges in determining breadth and depth of the state standards to sample National Alternate Assessment Center

30. References • Council of Chief State School Officers (n.d.). Alignment Models. Retrieved April 29, 2005, from http:// www.ccsso.org/Projects/alignment_analysis/models/418.cfm • Porter, A. C. & Smithson, J. L. (2001). Defining, developing, and using curriculum indicators (CPRE Research Report Series RR-048). University of Pennsylvania: Consortium for Policy Research in Education. • Webb, N. L. (1997). Research Monograph No. 6: Criteria for alignment of expectations and assessments in mathematics and science education. Washington, DC: Council of Chief State School Officers. National Alternate Assessment Center

31. More information • http://education.uncc.edu/cpflower National Alternate Assessment Center