1 / 48

Designing Validity Into an Alternate Assessment

Designing Validity into an Alternate Assessment . Designing Validity Into an Alternate Assessment. Neal Kingston, PhD Angela Broaddus , PhD, Meagan Karvonen , PhD Center for Educational Testing and Evaluation, University of Kansas Karen Erickson, PhD

misha
Télécharger la présentation

Designing Validity Into an Alternate Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Designing Validity into an Alternate Assessment Designing Validity Into an Alternate Assessment Neal Kingston, PhD Angela Broaddus, PhD, Meagan Karvonen, PhD Center for Educational Testing and Evaluation, University of Kansas Karen Erickson, PhD Center for Literacy & Disability Studies, University of North Carolina at Chapel Hill Moderator: Susan Weigert, PhD, OSEP The present publication was developed under grant 84.373X100001 from the U.S. Department of Education, Office of Special Education Programs. The views expressed herein are solely those of the author(s), and no official endorsement by the U.S. Department should be inferred.

  2. Overview of Evidence Centered Design

  3. Five Layers of ECD 3

  4. Domain Analysis • Descriptions of the knowledge domain being assessed • Situations or activities that evoke the relevant concepts and skills • Common misconceptions • Instructional strategies 4

  5. Domain Modeling • Identify focal knowledge and skills • Identify potential tasks for assessing focal knowledge and skills • Describe characteristics and variable features of assessment tasks 5

  6. Conceptual Assessment Framework • Student models • proficiencies the assessment aims to address • Task models • how students will respond in an assessment situation to indicate their knowledge • Evidence models • how observations collected from tasks offer evidence about what students know 6

  7. Assessment Implementation • Task/testlet development • Scoring rules • Measurement models 7

  8. Assessment Delivery • Tests administered to students • Tests scored • Reports generated and interpreted 8

  9. Overview of the DLM Maps, Claims, Conceptual Areas, and Essential Elements

  10. Overview of the DLM Maps, Claims, Conceptual Areas, and Essential Elements

  11. Learning Map

  12. Models need to specify claims… (Gong, 2012)

  13. DLM Claims English Language Arts – Claim 1 Students will comprehend text in increasingly complex ways Mathematics – Claim 1 Students will demonstrate increasingly complex understanding of number sense.

  14. Conceptual Areas • Conceptual areas are comprised of nodes that represent the development of related cognitive processes. • Conceptual areas contain: • nodes that have been identified as the targets for Essential Element, and • nodes preceding and extending beyond the targets.

  15. ELA.C1.1 Determine Critical Elements of Text • ELA.C1.1 Determine Critical Elements of Text

  16. English Language Arts Claims and Conceptual Areas

  17. English Language Arts Claims ELA.C1 Students can comprehend text in increasingly complex ways. ELA.C2 Students can produce writing for a range of purposes and audiences. ELA.C3 Students can communicate for a range of purposes and audiences. ELA.C4 Students can engage in research/inquiry to investigate topics and present information.

  18. ELA.C1Students can comprehend text in increasingly complex ways. ELA.C1.1 Determine Critical Elements of Text ELA.C1.2 Construct Understandings of Text ELA.C1.3 Integrate Ideas and Information from Text

  19. ELA.C2 Students can produce writing for a range of purposes and audiences. ELA.C2.1Use Writing to Communicate ELA.C2.2 Integrate Ideas and Information in Writing

  20. ELA.C3 Students can communicate for a range of purposes and audiences. ELA.C3.1 Use Language to Communicate with Others ELA.C3.2 Clarify and Contribute in Discussion

  21. ELA.C4Students can engage in research/inquiry to investigate topics and present information. ELA.C4.1 Use Sources and Information ELA.C4.2 Collaborate and Present Ideas

  22. Mathematics Claims

  23. Claim M.C1 – Number Sense Students demonstrate increasingly complex understanding of number sense.

  24. Claim M.C2 - Geometry Students demonstrate increasingly complex spatial reasoning and understanding of geometric principles.

  25. M.C3 – Measurement, Data, and Analysis Students demonstrate increasingly complex understanding of measurement, data, and analytic procedures.

  26. M.C4 – Algebraic and Functional Reasoning Students solve increasingly complex mathematical problems, making productive use of algebra and functions.

  27. EE Concept Maps take content to assessment design • Identify claim, conceptual area, CCSS, and Essential Element • Identify key vocabulary • Describe and define a range of skill development • Describe and define misconceptions • Identify prerequisite and requisite skills • Identify sensitivity and bias barriers • Identify accessibility issues

  28. Essential Element Concept Map 28

  29. Feelings of CharactersELA.EE.RL.3.3 Identify the feelings of the characters in a story Embedded and/or Conclusion Both Embedded Both Embedded

  30. FractionsM.EE.3.NF1-3 Differentiate a fractional part from a whole

  31. Dissecting the EECM

  32. Framework • Claim • Conceptual Area • Common Core State Standard • Essential Element • Essential Question • “Does” versus “Can” • Use “Does the student…?” when asking a comprehension question. • Use “Can the student…?” when asking a performance question. • Questions focus on EE, address linkage node

  33. Vocabulary • Concepts • Words

  34. Nodes • Initial (a) • Distal (b) • Proximal (c) • Target (d) • Successor (e) • Supporting Nodes

  35. Questions/ Misconceptions • Questions to Ask • What additional skills does the student need to reach the next node? • Misconceptions and Errors • What is preventing the student from reaching the next node?

  36. Next EE/Accessibility/ Barriers • Next EE • Accessibility • Barriers • Testlet Access

  37. Instructionally Relevant Testlets

  38. Testlet Development

  39. Testlet Development Process • Follows a rolling sequence in conceptual areas, driven by EECM development • One item writer develops a set of testlets

  40. Testlet Sets Testlet a Behavior Connect the map… …to the items developed. Initial Precursor Testlet b Distal Precursor Behavior Proximal Precursor Testlet c Behavior Testlet d Target Behavior Testlet e Successors Behavior

  41. DLM Testlet Development Process

  42. Dynamic Learning Maps

  43. Structure of a Testlet • Begins with engagement activity • ELA: Text presented in segments with questions embedded and at conclusion • Math: series of questions or problems related to single topic

  44. Sample Testlets

  45. Testlet Administration

  46. Discussion

  47. Questions • What features must be in place for alternate assessments to be instructionally relevant and not just embedded? • What factors might support teachers in making full use of an assessment like DLM? • What are the important research questions to be asked about instructionally relevant alternate assessments?

  48. Thank you! THANK YOU! For more information, please go to: www.dynamiclearningmaps.org

More Related