Tools & Tests

1. Tools & Tests Dr Paul Drijvers Freudenthal Institute for Science and Mathematics Education Faculty of Science, Utrecht University www.fi.uu.nl/~pauld

2. Tools & Tests: Outline • Introduction • Tests with Tools • Tools for Tests

3. 1 Introduction • What is the Freudenthal Institute? • Key concepts: • Realistic Mathematics Education • Design research • Sites: www.fi.uu.nl/enwww.fi.uu.nl/tooluse/enwww.fi.uu.nl/~pauld (this presentation)

4. FI and Tests (1) • PISA’s Assessment Framework

5. FI and Tests (2) • Not only summative, but also formative assessment, or tests for diagnostic purpose • New formats, new goals: • Math Alympiadwww.fi.uu.nl/alympiade/en • Math B-daywww.fi.uu.nl/wisbdag

6. FI and Tools (1) Applet and game design: • www.fi.uu.nl/rekenweb/en/ (primary) • www.fi.uu.nl/wisweb/en/(secondary)

7. FI and Tools (2) Tools for upper secondary level: • Graphing calculators • Computer Algebra • Dynamic Geometry

8. has to be reflected in Teaching / learning Assessment drives Tools & tests: why?

9. Tools & tests: a hot issue • What are we assessing, tool skills or mathematical skills? • Technological development:How do we control the capacities of the tool? • Emotion: “students should be able to do this by hand”

10. 2 Tests with Tools Why tests with ICT tools? • ICT is important in today’s society, and therefore should have a place in education. • ICT allows for different types of questions, for assessment of other kinds of skills. • Assessment should reflect teaching. • (National) tests are means to change teaching practice.

11. Looking at our neighbours (feb 2008)

12. France, BacS, 2007, CASCalc allowed

13. Germany (NRW), 2007, GC allowed

14. Germany (NRW), 2007, GC allowed

15. Germany (NRW), 2007, CAS allowed

16. UK, 2007, GraphCalc allowed Traditional task, no influence of / role for the technology available

17. UK, 2007, GraphCalc allowed

18. Denmark, 2006, technology allowed

19. Denmark, 2007, no technology allowed

20. Denmark, 2006, technology allowed A high cylinder-shaped container has a whole in the bottom. When there is water in the container, it leaks out through the hole. At every moment, the rate of change of the water level is in direct ratio to the square root of the height of the water level. a) Set up a differential equation which describes this situation

21. Netherlands, 2002, GraphCalc allowed

22. Netherlands, 2002, GraphCalc allowed

23. The GC-intersect technique solution of an equation = intersection of graphs x instead of h, decimal point instead of comma, line editing instead of pretty print window settings choose graphs solution on an interval;there may be more choose start value approximate versus exact no use in case of parameters

24. Solve an equation graphically / numerically Mental scheme Type of tasks Technique Artefact Instrument Task – Technique – Theory interaction

25. Theoretical sidestep: instrumentation • Tools are not neutral, tools matter!(Noss & Hoyles, 2003) • The thinking shapes the tool use and the tool possibilities and constraints shape the thinking and the development of schemes and techniques • Problem solving strategies, conceptual understanding and thinking schemes co-emerge with machine techniques

26. Artefact and instrument Artefact: • The object or ‘thing’ in use Instrument: • The artefact ánd the schemes and techniques the user develops to use it for an intended purpose:Instrument = Artefact + Scheme/Technique

27. Instrumental genesis • The instrumental genesis of a meaningful instrument requires the development of appropriate techniques and mental schemes. • In such a scheme conceptual and technical aspects co-develop and are interrelated.

28. Conclusions Tests with Tools (1) • Assessment with technological tools is an issue in many countries, subject to discussion and change. • In most cases, more is asked than just ICT-output, and is reasoning / interpretation / explanation required. • In many countries, (also) by-hand results are required. In some cases in a non-technology part of examination, in other cases through a specific phrasing of the assessment item.

29. Conclusions Tests with Tools (1) Different ways to deal with technology at examinations. Trends: • Neglect, i.e. ask always for exact by-hand results. This does not avoid the opportunity for students to explore situations or verify by-hand solutions with the ICT tool. • Use ICT for more complex situations / applications / modeling tasks / contexts. Interpretation and modeling are often central issues in this approach. • Mixed format, in which ICT can be used for exploration and approximation, but at the end exact by-hand algebra is required. • Cf Brown: required / optional / neutral / excluded • A lot of work to be done!

30. 3 Tools for Tests Why digital assessment? • Flexible in time • Randomization of items • Constant quality • Automatic grading • Feedback

31. Tools for Tests Difficulties with tools for digital assessment • Specific tools required, such as equation editor and graphing tool • Computer algebra needed for interpretation of (equivalent) answers • Adequate feedback hard to implement

32. Criteria for assessment tools Some criteria from Bokhove’s research: • Central data storage, student registration • Math facilities (equations, graphs, representations) • Integrated CAS for evaluation of student answers • Different feedback levels (process level, level of the answer, global level), allowing both formative and summative assessment • Authoring facilities for the teacher • …

33. Examples of digital assessment tools • Maple Testing and Assessment (Maple T.A.) • Digital Mathematics Environment (DME) • System for Teaching and Assessment using a Computer algebra Kernel (Stack) • WirisOnline • WWW Interactive Multipurpose Server (WIMS) • (Activemath) • …

34. Example: DME on linear equations See www.fi.uu.nl/dwo/en Example: module on linear equations Line from formative to summative: • Strategy development • Practice • Self assessment • Final assessment

35. 1. Strategy development

36. 2. Practice

37. 3. Self assessment

38. 4. Final assessment

39. Conclusion Tools for Tests Temporary conclusions on digital assessment: • Relevant development with growing importance for the future • At present: limitations to overcome • Take care of too narrow views on assessment due to technological limitations!

40. Regulations Germany: • NRW: Bei der Darstellung der Lösungen müssen für alle Teilaufgaben grundsätzlich der Lösungsansatz (je nach Aufgabenstellung die Sachaussage und/oder die mathematische Formel) notiert und die Wahl begründet werden. Darüber hinaus sind wesentliche Entscheidungen bei der Aufgabenlösung zu erläutern bzw. zu begründen und wesentliche Rechenschritte zu dokumentieren. Die ausschließliche Angabe des richtigen Rechenergebnisses einer Teilaufgabe führt nicht zu Bewertungspunkten.

41. Regulations Denmark: The assessment of the single answers, and of the overall impression, will weighten whether the students line of thought is clearly expressed, including whether the answer encompasses: • a linking text from start to the end which clearly presents the idea of the single task and the single sub-question • Appropriate structure of the answer in accordance with good mathematical custom • documentation in the form of a sufficient number of calculations • report of the chosen strategy, including use of different facilities offered by the computer tool • use of figures and illustrations • a clear connection between text and figures • report of the use of technical mathematical terms introduced by the student, in the case of non-standard knowledge • Conclusion of the questions with clearly stated results, presented in clear language with the use of common technical mathematical terms