Webb’s Depth of knowledge

1. Webb’s Depth of knowledge a scale of cognitive demand

2. DOK Introduction? + ! • Code with a ?to indicate that you have never heard of this, • Code with a +to indicate that you know something about this, • Code with an !to indicate that you know this so well that you could teach this!

3. Origin • In 1997, Norman Webb, of the Wisconsin Center of Educational Research ; University of Wisconsin-Madison, developed a process and criteria for systematically analyzing the alignment between standards and standardized assessments.

4. No Child Left Behind (NCLB) requires assessments to “measure the depth and breadth of the state academic content standards for a given grade level” (U.S. Department of Education, 2003, p. 12) Mechanism to ensure that the intent of the standard and the level of student demonstration required by that standard matches the assessment items.

5. Since then, the process and criteria have demonstrated application to reviewing curricular alignment as well. Webb’s Depth of Knowledge (DOK) model may be employed to analyze the cognitive expectation demanded by standards, curricular activities and assessment tasks (Webb, 1997).

6. DOK Level 1Recall • Basic recall of concepts, definitions, facts, and processes • Answering a Level 1 item can involve following a simple, well-known procedure or formula • Simple skills and abilities or recall characterize DOK 1

7. Examples of DOK Level 1 • Locate or recall facts found in text • Apply a well known formula • Use a dictionary to find the meaning of words • Represent math relationships in words, pictures, or symbols • Perform a simple science process or a set of procedures

8. DOK Level 2Skill/Concept • DOK 2 includes the engagement of some mental processing beyond recalling or reproducing a response. • Items require students to make some decisions as to how to approach the question or problem. • These actions imply more than one mental or cognitive process/step.

9. Examples of DOK Level 2 • Identify and summarize the major events, problem, solution, conflicts in literary text • Explain the cause-effect of historical events • Predict a logical outcome based on information in a reading selection • Retrieve information from a table, graph, or figure and use it to solve a problem requiring multiple steps.

10. DOK Level 3Strategic Thinking • DOK 3 requires deep understanding as exhibited through planning, using evidence, and more demanding cognitive reasoning. • The cognitive demands at Level 3 are complex and abstract. • An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.

11. Examples of DOK Level 3 • Justify a response when more than one answer is possible. • Analyze or evaluate the effectiveness of literary elements (e.g. characterization, setting, point of view, conflict and resolution, plot structures). • Solve a multiple-step problem and provide support with a mathematical explanation that justifies the answer.

12. DOK Level 4Extended Thinking • DOK 4 requires high cognitive demand and is very complex. • An investigation or application that requires time to research, think or process multiple conditions of the problem. • Students are expected to make non-routine manipulations or connections across disciplines/content areas/multiple sources. • Due to the complexity of cognitive demand, DOK 4 requires an extended period of time.

13. Examples of DOK Level 4 • Analyze and synthesize information from multiple sources. • Design and conduct experiments and projects. • Conduct a study that requires taking into consideration a number of variables.

14. DOK Tips It’s not just the verbs: • DOK 1- Describe three characteristics of metamorphic rocks. (Requires simple recall) • DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types) • DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of the rock cycle and a determination of how best to represent it)

15. DOK Tips It’s not the difficulty of the task, it’s the complexity. • What is the definition of exaggerate? DOK 1— recall (NOT difficult!) • What is the definition of glabella? DOK 1— recall (DIFFICULT!)

16. DOK Tips • If there is only one correct answer, it is probably level DOK 1 or DOK 2   • DOK 1: you either know it (can recall it, locate it, do it) or you don’t • DOK 2 (conceptual): apply one concept, then make a decision before going on applying a second concept

17. DOK Tips •  If there is more than one solution/approach, requiring evidence, it is DOK 3 or 4 • DOK 3: Must provide supporting evidence and reasoning (not just HOW solved, but WHY – explain reasoning) • DOK 4: all of “3” + use of multiple sources or texts

18. Let’s review • Turn and talk to your neighbor. Discuss how looking at DOK levels of activities in your lessons might increase rigor in your classroom. • In 3 minutes we will share. • New York City Department of Education DOK Video

19. OK, what do you think? Grade 4 Assessment : Think carefully about the following question. Write a complete answer. You may use drawings, words, and numbers to explain your answer. Be sure to show all of your work. Laura wanted to enter the number 8375 into her calculator. By mistake, she entered the number 8275. without clearing the calculator, how could she correct her mistake? Without clearing the calculator, how could she correct her mistake another way? Level 3.  “An activity that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.” Since there are multiple possible approaches to this problem, the student must make strategic decisions about how to proceed, which is more cognitively complex than simply applying a set procedure or skill.

20. Here’s another: Level 1. Students only need to be able to recognize even numbers.

21. Now try this one. • Look at the drawing. • 1. 23 • 2. 25 • 3. 2 23. 3. 28 • 4. 30 • 5. 32 Grade 8 Assessment Look at the drawing. The numbers alongside each column and row are the total of the values of the symbols within each column and row. What should replace the question mark? Level 3. This item can be approached through a number of viable strategies: pattern recognition, guess-and-check, algebra, etc. This freedom means that the student must make choices and assumptions. Furthermore, no matter what strategy she employs, she must keep track of a complex logical chain. The multiple choices provided do not make this task any less complex.

22. How about this one? Grade 11 Assessment Level 3. This item gives the student a new definition and asks her to use it as a basis for reasoning. In order to ascertain whether the student really understands the asymptotic behavior that makes a 6% effective rate impossible, this item must be open-ended. This is why most Level 3 items are open-response items, because the complexity of thinking they require the students to display could not be displayed using the multiple choice option.

23. And one more? Level 2. This item is not Level 1 because it is not routine, nor does it focus on a memorized definition or procedure. In fact, it involves numerous steps, because it requires students to compare several different pairs of shapes before arriving at the correct answer. For these reasons, many spatial reasoning items are Level 2. Note that this may be a coded as a source-of-challenge item, because choice C seems to be drawn in a misleading way.

24. More to think about . . . • Karin Hess, from the The National Center for the Improvement of Educational Assessment, Inc. has done extensive research regarding rigor in the classroom and she applies Webb’s DOK to Bloom’s Cognitive Process Dimensions.

25. More to think about . . . Hess’ Cognitive Rigor Matrix and Curricular Examples Applying Webb’s Depth-of-Knowledge Levels to Bloom’s Cognitive Process Dimensions – Math/Science

26. Today’s Task • Look at your current lessons and code activities. • Enhance lessons to increase opportunities for rigor. • Collaborate! • DOK objectives in specific subject areas are available, just ask: • Math • Language Arts • Science • Social Studies

27. So many choices: • Differentiation • I/E times • Opportunities for: • Research • Deeper understanding • Self-selection • Long-term projects • More complex problems • Integrating subjects

28. We all know. . . • This work WON’T be finished today. • You have choices! • You know your students better than anyone. • Collaboration among colleagues will greatly help your efforts. • There are resources for level 3 and 4 activities! • This work will change the way your classroom looks. • Your students’ expectations and outcomes will increase.

29. Thank you!