University of Akureyri

1. University of Akureyri Teachers’ Professional Development Conference April 2007

2. Assessment for Learning Workshop: Part 2 The quality of questioning Val Brooks

3. Why is questioning so important? • Identified by Black and Wiliam (1998) as one of 4 key areas for further research & development • OFSTED (1996) described questions as “the single most important factor in students’ achievement of high standards where questions were used to assess pupils’ knowledge and challenge their thinking” • “Judicious questioning is nearly half the learning” (Jerome Bruner) • “our most important intellectual tool” (Postman)

4. BUT • On average teachers ask 300+ questions each day • “Teachers do not generally review the assessment questions that they use and do not discuss them critically with peers, so there is little reflection on what is being assessed” (Black & Wiliam 1998: 17)

5. Key issues • Questioning is a major feature of classroom life, occupying high proportion of time • A powerful strategy: potential to enhance teaching, learning and assessment • Questioning is employed in an intuitive and naturalistic manner and its true potential is not harnessed

6. Objectives for Workshop: Part 2 To extend your awareness of key research findings: • The problems and pitfalls associated with conventional approaches • Strategies which are being developed to improve the quality of questioning • To identify strategies with which you may wish to experiment

7. Components of Questioning The question itself Posing the question Providing an answer Providing feedback Issues to Consider What kinds of questions Who does the asking? How? Who answers? How? Who provides feedback? How? Ways of conceptualising this session

8. Task • Describe the approach (i.e. identify its component parts) • In what ways is this approach helpful to teachers? • What are the potential pitfalls with this approach?

9. The Approach T broadcasts question to entire class T accepts answer from a pupil, usually one who responds quickly T responds (i. e. repeats answer or modifies it) The lesson moves on Research Findings IRF cycle: one of most commonly used forms of classroom discourse Initiation (T’s question) Response (Student’s answer) Feedback (T’s evaluation of answer) Commonly used approach

10. Benefits • Enables teachers to involve pupils & keep them engaged • Gives teachers conversational control - who speaks, when, about what, for how long etc • Teachers able to accomplish their plans but without relying on a didactic style of teaching

11. Consequences • “[some] pupils don’t even try to think out a response – if you know that the answer, or another question, will come along in a few seconds, there is no point in trying. It is also common that only a few pupils in a class answer teachers’ questions. The rest then leave it to those few, knowing that they cannot respond as quickly and being unwilling to risk making mistakes in public. So the teacher… by accepting answers from a few can keep the lesson going but is out of touch with the understanding of most of the class – the question-answer dialogue becomes a ritual, one in which all connive and thoughtful involvement suffers.” (Black and Wiliams 1998, pp. 11-12)

12. Potential Pitfalls • Feedback is obtained from a small, possibly unrepresentative, sample of the class (e.g. John’s teacher missed a subtle form of feedback conveying confusion) • Lesson moves on but how many are left behind??? • When an answer is not provided quickly teachers either • Re-phrase the question • Answer it themselves! • Teachers provide insufficient wait time for students to process their ideas and formulate thoughtful answers • “too often teachers seem scared of silence, so that they fail to allow pupils sufficient time to think” (OFSTED 1998: 80)

13. Consequences • Lessons move at the pace of quickest-thinking children • Others are left behind • Those who fail to keep pace are liable to become disengaged and are more likely to become disruptive • Feedback obtained from only a small group of children so the teacher loses touch with learning of many in the class • T dominates the ‘talk’, talking far more than pupils

14. Closed Limited number of permissible responses Answers usually short Require knowledge and recall Accelerates the pace of a lesson E.g. What is your name? Open A wider range of possible responses Elaborated answers Require pupils to explain or justify their thinking Encourage slower, more reflective pace E.g. How do you feel about your name? Categorising Questions

15. Closed questions are useful for Checking factual accuracy • Establishing how much pupils already know about a topic before starting to teach • Reviewing learning at the end of a lesson • Testing recall of prior learning at a later stage • Formal tests and examinations

16. Benjamin Bloom’s Taxonomy: A Hierarchy of Thinking Skills Higher order • Evaluation • Synthesis • Analysis Lower order • Application • Comprehension • Knowledge

17. Research findings • Closed/recall questions used more frequently than open questions • Over-exposure to closed questions promotes lower order thinking • Teachers are asking too many questions • Over-questioning encourages a less thoughtful, more passive and dependent response in students • Attainment is raised where pupils are given regular practice in higher order thinking (Black and Wiliam 1998)

18. Ways of categorising questions Closed Open Recall Thought Response Information -seeking -seeking

19. Response-seeking T: Can anyone tell me why the litmus paper turned blue? P1: Because it’s wet. T: (Doubtfully) Ummm … not quite right. P2: Because it was put into acid. T: (Looks doubtful) P3: Because it was put into alkaline. T: Good! It turned blue when it was exposed to alkaline. Now, can someone tell me something you might find at home that is alkaline?

20. Response-seeking T: What is a frog? Silence T: It’s an am … Silence T: It’s an amph … Silence T: It’s an amphib … Silence T: An amphibian!

21. “Assessment must therefore extend beyond the • simple determination of the extent to which • (students) have learned as intended to the • discovery of what they have actually learned, • right or wrong. Teachers, like marksmen, may • have clear objectives, but if they are to improve • the quality of their performance then - like • marksmen - they will want to know where all • their shots went, and not merely how many find • their target. Indeed, the patterns of ‘wrong’ • learning, like the distribution of off-target shots, • will provide the clearest cues to improvement.“ • (Mary Simpson, 1990)

22. Information-seeking questions are useful for: • Eliciting extended answers in which pupils have to explain or justify their thinking • Discouraging guessing • Promoting higher order thinking • Close monitoring of progress during a lesson • Diagnostic assessment

23. Giky Martables The protruberances of the giky martables are perfunctory and enable them to search and easily locate their talibands. When the talibands are disard, they perform this ritual more often, sometimes in groups but more often in pairs or singly. They have even been known to excalibate the twigs and plants in their path, leaving chaos in their wake. In the early spring, if the temperature rises, they bimimer their dresarwars, sometimes travelling large distances before they macoca and cafuffle in woods or hedges. (Black et al, Kings College, London)

24. Pupils focus on regurgitating the required response Shift of focus away from learning and on to winning teacher approval Can often work out answers with little or no understanding of meaning Resort to rote/surface approaches to learning Resort to guesswork Teachers Are misled (These questions “do not necessarily tell teachers what they need to know about their students’ learning” (B+W, 1998)) - May think that pupils really understand when they don’t - May think that pupils can’t do something when they can Response-seeking

25. Feedback: Handling Inappropriate Answers • Wragg (1997): over a third of answers received a non-verbal response or no reply • OFSTED (1998:80) noted a tendency to be: ‘undemanding or over-sensitive to pupils’ feelings, accepting a wrong or partial answer rather than pressing the pupil to reflect and refine or go a stage further’. • Torrance and Pryor (1998:18) also found that teachers, anxious to utilise infants’ contributions, accepted partial or irrelevant answers and avoided explicit correction of errors

26. Task • Analyse this transcript from Torrance and Pryor’s investigation into formative assessment at KS1. (The appendix explains the transcribing conventions.) • The transcript: a Year 2 whole-class Mathematics lesson. The teacher is revising work on tens and units before extending the work by introducing the concept of ‘rounding up’ to the nearest appropriate number. • Evaluate the teacher’s questioning. How effective is it? • How many of the pitfalls that we have identified can you detect?

27. Look out for • Too many questions/ unthinking use of questioning • Over-use of closed/response-seeking and insufficient use of open/information-seeking questions • Pupils become passive/dependent/resort to guessing • Tendency to ignore/provide inadequate feedback on inappropriate responses • T dominating discourse • IRF pattern • T re-phrasing question or answering it • Insufficient wait time

28. Review of Session Objectives To extend your awareness of: • Some key research findings • The problems and pitfalls associated with conventional approaches • Strategies which are being developed to improve the quality of questioning (continued in next session) • To identify strategies you may wish to experiment with

29. Assumptions underlying innovations in questioning • Techniques are best suited to • open questions requiring higher order thinking • An information-seeking approach • Being used in conjunction with one another rather than in isolation • Techniques not well-suited to • closed questions/lower order thinking • a response-seeking approach

30. Benjamin Bloom’s Taxonomy: A Hierarchy of Thinking Skills Higher order • Evaluation • Synthesis • Analysis Lower order • Application • Comprehension • Knowledge

31. Strategy 1: Applying Bloom’s Taxonomy • KNOWLEDGE ( requires accurate recall – remembering information learnt previously ) • COMPREHENSION ( requires understanding of material – expressing ideas in their own words) • APPLICATION ( requires transfer of ideas to a new context – understanding a general principle and applying it in a new situation) • ANALYSIS (requires breaking a subject down into its parts - exploring the nature of the parts and their relationships with one another) • SYNTHESIS (requires bringing ideas together for a new purpose – to create a theory, plan, experiment, forecast) • EVALUATION (requires making judgements and explaining reasons for them – weighing the evidence and reaching conclusions)

32. Task • Use Bloom’s taxonomy to categorise these questions If you have time • Pick a topic and devise 6 questions on it, one for each of the thinking skills

33. Strategy 2: Encouraging Thoughtful Behaviour • Studies have found an average wait time of between 0.9 and 3 seconds • Requiring an almost instant response only works well with knowledge/recall questions • Some teachers are increasing their wait time, to allow students to formulate more thoughtful answers

34. Effects of increasing the wait time after a question: • In one study the average wait time was about a second but: ‘where a longer silence was left – even as short as three seconds – the quality and extent of pupils’ responses improved dramatically… not only longer but also more thoughtful’ (Woods 1998:176). • More pupils offer answers • The number of ‘I don’t know’ responses decreases • The number of hypothetical answers increases • The frequency of answers from less able students increases • Students more likely to challenge and/or improve each other’s answers

35. One teacher’s observation on his former approach to questioning • “There must have been times (still are?) where an outside observer would see my lessons as a small discussion group surrounded by many sleepy onlookers” • (James, Two Bishops School)

36. Strategy 3: Involving more learners No hands’ policy • Pupils are trained not to raise their hands to answer a question • This strategy works well with increased thinking time/questions requiring more thought • Everyone is expected to provide an answer if called upon

37. Requirements • Well-thought out, clearly worded questions • Sensitivity/attention to differentiation • Classroom climate where admitting to uncertainty and making mistakes are seen as a natural part of learning

38. Other Strategies for Obtaining fuller feedback • Pupil votes • Answers written on hand-held dry-wipe boards • True/false cards • Traffic lights • Thumbs • Response partners • P-I-P (Private-Intimate-Public)

39. Who does the asking? • One study found that teachers asked between 6 and 12 times as many questions as pupils • Pupils’ questions are often about trivial or procedural issues • Pupils ask surprisingly few questions and surprisingly few of their questions are about learning!

40. Encouraging Learners to become Questioners • B+W (1998) report an experimental study involving students in HE who were studying Physics. • Treatment group: trained to generate their own thought-provoking questions and then try to answer them. • Control group: relied on tutors’ questions. • Which group performed best? Why?

41. Encouraging Learners to become Questioners • Study of 2 outstanding teachers (B+W, 1998) • Lessons characterised by unusually high frequency of questioning • Over 60% of questions asked by pupils Implications?

42. Developing a spirit of enquiry in the classroom • Planning opportunities for pupil questioning into lessons (e.g. by ‘chunking’) • Use of response groups • Mehrabian looked at features of communication (7% - words, 38% - tone of voice, 55% - non-verbal communication) • Welcoming body language (nodding, leaning forward, good eye contact, active listening) • Making use of pupil questions in discussion, tests, homework etc

43. Developing a spirit of enquiry in the classroom Eyre(1997) (Able Children in Ordinary Schools) • In one school - at the end of a topic pupils were asked to write a list of questions about the topic to which they did not know the answer • Some pupils’ questions went way beyond the level expected for their age • Strategy capable of identifying the very able as well as those who are struggling

44. Task • Contrast this example with the Torrance and Pryor example • How does the questioning differ? • How does the discourse differ?

45. Review of Session Objectives To extend your awareness of: • Some key research findings • The problems and pitfalls associated with conventional approaches • Strategies which are being developed to improve the quality of questioning • To identify strategies with which you may wish to experiment