Finding Time for Deeper Thinking: It’s All about Balance and Intent
In my October 2015 blog post, I addressed the question of “balance” when designing summative mathematics assessments. In that post, and in my work with schools and testing companies, I generally suggest that about half of any summative assessment’s score points be devoted to conceptual understanding – that means DOK 2-type questions. In mathematics, this is usually achieved through routine word problems where students are asked to apply more than one concept and make decisions as to an appropriate strategy, procedure, or use of a model (graph, labeled diagram, etc.) that will lead to a solution. These DOK 2 test items need to be balanced with some DOK 3 or 4 problems that are not routine, problems that require explanations of WHY my reasoning or solution works, not simply HOW the problem was solved. This does not mean you need to include an equal number of test questions at DOK 3-4; however, it does mean that enough of the test score points are given for deeper understanding. And while all students may not always be able to solve the more complex problems in an elegant way, they will increase in their ability to do so the more they have opportunities to experience true problem solving during class. I call these the practice scrimmages before getting on the bus to go play the game.
In an ELA assessment my rule still holds true about half of the assessment being DOK 2-type questions, which entail students demonstrating the ability to “get the gist” of a text by making basic inferences, sequencing or summarizing key events, comparing characters, and identifying the central idea of a text. This month, I share some of my thoughts on how to balance instructional time, so that deeper thinking does not get pushed to the end of the week, or worse…to the end of the unit or even the semester.
Nicole W., from CA wrote me to ask: What percentage of our instructional day should be spent in each DOK level?
Dr. DOK: I began my response to Nicole with my favorite answer - IT DEPENDS! While I do not believe that instructional time in any given day can be broken down in this way - by DOK Levels - I think there are some good ways to think about how to move away from “Random Acts of Rigor” to planned balance.
First of all, about half of the score points in a summative assessment should be at DOK 2 – focusing on conceptual understanding. That means a reading assessment should be asking students about central ideas, sequence of events, cause-effect, comparing-contrasting information presented in the text, and/or summarizing key ideas in the text. A math assessment needs to include routine/familiar word problems where students must decide what strategy (measure, add, divide, etc.) will work and then perform it, organize or graph data, compare-contrast information, create a mathematical diagram that shows math relationships, etc.
So what makes up the rest of the summative assessment? YOU NEED BALANCE! I recommend at least one or two open-ended questions, such as what is the theme or author's message? What is the author's perspective on the topic? Is there any potential bias in how the author presents his/her thinking and supporting evidence? For all of these questions, students must analyze text evidence in order to support their interpretations (and all of these are DOK 3-type questions). In math, you want to include some non-routine problem-solving opportunities: Look at how this student solved this problem. Is her answer correct? Prove how you know using calculations, diagrams, equations, and words to explain your thinking. (These also are DOK 3-type problems.) To achieve balance in testing, the open-ended questions should get more score points than the DOK 1 or 2 questions, and probably be scored on a 4- or even 8-point scale that include points not only for correct answers, but use of an appropriate approach/strategy, broader conceptual understanding, and evidence that makes a convincing argument.
Following the line of thinking about assessments, I'd say that EVERY Lesson needs to have a balance of DOK 2 + DOK 3 --- not necessarily in a planned sequence, with DOK 1 and DOK 2 questions always coming before a DOK 3 question; but a mix of questions such as, "can you provide some evidence to support your thinking? What makes you say that? How do you know the author believes this? Those are DOK 3-type questions, requiring students to uncover their thinking and reasoning, using credible evidence.
Performance tasks, such as composing multi-paragraph writing, conducting science investigations, and justifying solutions in math problem solving provide opportunities to use DOK 1 skills to complete more complex tasks...students have to locate details in text (DOK 1) in order to provide supporting evidence, for example. Performance tasks are the "scrimmages" before the games (summative assessments) that allow students to practice transfer of skills and concepts to new or novel situations. I recommend that each day, students are engaged in some kind of performance task - some days it will be in math, some days in writing or reading, some days in social studies, science, or even physical education where they are asked to evaluate demonstration of skills against criteria for excellence. These types of short and longer instructional tasks are the most engaging kinds of learning when done with peers. Products can be oral, visual, digital, by demonstration, or … if you insist, in writing.
DOK 4-type tasks require more time and more than one resource to delve in to (e.g., analyze, evaluate, construct alternative), such as what might be required in a unit project, researching and presenting on a topic, or comparing themes or author styles in two or more texts. Generally speaking, you can get to DOK 4 thinking with print and non-print texts fairly easily in most content areas. I'd expect a teacher to get there with literacy skills (reading/writing/speaking/listening) each week, at minimum.