Pepite : software to help teachers diagnose students ’ algebra competencies

1. 1 1 Déc. 2001 Pepite : software to help teachers diagnose students’ algebra competencies Élisabeth DELOZANNE Brigitte GRUGEON IUFM de Créteil IUFM d’Amiens LIUM (Le Mans) DIDIREM (Paris) France

2. Overview • Introduction • Main ongoing project : Lingot • Pepite • Educational foundations (Grugeon 95) • Pepite software (Jean 2000) • Pepite testings • Results and questions

3. Pedagogical Basic Assumptions • Students’ answers show coherences in their reasoning • It is possible to analyse their answers to identify the competences the students have built (correct, partial, inappropriate) • Identifying those competences would help teachers to give appropriate tasks to destabilize them (if inappropriate) and to make them evolve

4. Design assumptions • Computers can help teachers both • in identifying competences • “Pépites” (nuggets) • in giving appropriate tasks to enhance learning and understanding • building “Lingot”, (ingot) • Two classes of end users • Students • usual user-centred design techniques (Human Computer Interaction) • task analysis based on educational research • Teachers • new usage patterns to be invented • participatory design (researchers and teachers)

5. Lingot project • Objectives • To instrument teacher’s activity with systems to enhance algebra learning and teaching • Pepite : Assessing assistant • Stephanie Jean • Free download : • Pepite.univ-lemans.fr • Demo in English • Amico : Learning environment • Dorothée Rasseneur • Interaction with virtual peers • To promote algebra as a proving tool • To develop links between different representation modes • Natural language, algebraic, graphical, numerical • Demo in French

6. Pépite • Objective • To help teachers diagnose student's competence in algebra • Multidisciplinary project • Educational research in mathematics (DIDIREM) • basis: a multidimensional model of algebraic competence • result: a paper and pencil diagnostic tool • Computer science research (LIUM) • (partial) automation of this paper and pencil diagnostic tool • Together • Developing awareness about diagnosis algebra competencies • Modifying teaching approach to algebra • Watching teachers using Pépite in classrooms • Spreading research results

7. A diagnosis tool (Brigitte Grugeon 1995) • A multidimensional model of competence • Mastered skills, meaning of letters, processing algebraic expressions, translation between representations, type of rationality • A set of Paper & pencil exercises and an analysis grid • Technical (generational and transformational), modelling, interpreting skills • Analysis grid derived from the model of competence • Students’ profiles • quantitative description • Success rates on mastered skills • qualitative description • Meaning of letters, processing algebraic expressions, translation between representations, type of rationality • description of flexibility between representations • Diagram

8. PÉPITEST PÉPIPROFIL PÉPIDIAG Transversal analysis Interpretation of the data Pépite software End-users: Students Teachers Students ?

9. PÉPITEST: the students’ software • proposes 22 exercises • Large sample of algebraic activities • gathers the student's answers • data collector for the diagnosis • quality of data and quality of the diagnosis are linked • HCI issues to obtain better results

10. PÉPITEST: Main Issues • Can PÉPITEST collect informative enough data to build a student profile ? • To what extent does the software change students' answers ? • Does PÉPITEST reduce the variety of proposed answers? • What are the differences between answers with paper & pencil and answers with PÉPITEST (especially for algebraic expressions)?

11. PÉPIDIAG: the analysis module • interprets and codes the student's answers • data collected by PÉPITEST • applies the analysis grid • MCQ • Open answers • algebraic expressions, natural language, mixed (“mathural” language) • fills a matrix for each student • 55 questions • 35 items in the model of competence • gives very fine description of the student's behaviour

12. PÉPIPROFIL: the teachers’ software • Two tasks for the teacher • To scrutinize student’s profile • To understand it, to complete it, to correct it • To ways • By student’s answers • By profile items • To have an overview on student’s profile

13. PÉPIPROFIL: the teacher software Two tasks for the system • to establish the profiles • by transversal analysis of the filled matrix • to highlight consistency in the student's way of functioning • to present them to teachers • with different levels of description • overall view of the profile / all available information • with different modes of representation • Graphical, textual, values

14. Pepite testings • PépiTest • Lab testing and 2 hours videotape • 100 students (grade 9 and 10) in 4 differents schools • PepiDiag • Comparing the matrix filled by Pepite and by teachers or researchers • PepiProfil • 10 Case studies in personalised help sessions • Uses of Pepite • 40 downloading • Training sessions for math teachers • 5 sessions : Teachers trainers inside Pepite team • Outside ? • Pilot sessions • personalised help for students with difficulties • auto-assessment and revision before examination • introduction of a notion from a debate with data show • learning activities

15. Results PÉPITEST • Limits : • difficulties to enter algebraic expressions • Supplies reliable data • finding every kind of answers expected by the didactical analysis • showing competences or fragilities that usual tests do not • corresponding to students' usual functioning ? • Cases studies show that teachers recognize their students • Used by some teachers • Used by teachers trainers • Larger sample of exercises, more different competences involved

16. Results PÉPIDIAG • Analyses 75% of the answers • simple methods to analyse answers • Does not analyze • expressions on several lines • mixed language • On average • experts correct 10 % of Pepite’s analysis

17. Results PÉPIPROFIL • The teacher completes and corrects PepiDiag’s diagnostic • when trained : ½ H • first time : 1 H 1/2 • PepiProfil automatically builds • detailled profiles • abstract of the profiles • Evaluation • profiles correspond to experts ones • to teachers’ ones ? • Teachers involved in the project • Experienced teachers, teacher trainers • Depends on how they consider teaching algebra • Beginners (pre-service) • matching student's knowledge ?

18. Usable by teachers? • What they like: • shows competences that they were not aware of • makes them more confident with their poor students • helps them to install new relationship with their students’ errors • helps them to understand the new official program in mathematics • What they dislike: • difficulties with using computers • didactical jargon • time consuming (1 H1/2 test, ½ H analysis) • distorting mirror of their teaching • mistrust of their understanding of algebra teaching • What they ask for: • remediation proposals according to student’s profile • a feedback for the students given by the software • possibility to modify the test • training in using the model of competence

19. Ongoing and future work • Correcting bugs and “jargon” • Suggesting activities adapted to each profile • Making a diagnostic for students and discuss it with them Later • Enhancing the diagnosis • Make a more reusable software in an other domain (statistics)

20. Key elements in Pépite • An iterative methodology which enables us • to work in a pluridisciplinary context and in classrooms • to build an “instrument” for teachers (Rabardel) • Intrumentalisation (artefact evolution) • Instrumentation (teacher evolution) • to spread research results • A diagnostic assistant that can help teachers improving their teaching of algebra

21. Thanks for attention • Questions?