1 / 33

Using the CEF R in Catalonia

This document outlines the implementation of the Common European Framework of Reference for Languages (CEFR) in Catalonia's state-funded language schools. With over 40,000 students and 16,000 certificate exams annually across 13 languages, key challenges include ensuring examination difficulty consistency and comparability of different certificates. The project from 2003-2007 focused on developing level specifications, revising curricula, and linking certificates to CEFR levels. Lessons learned emphasize the need for continued teacher training and empirical checks in assessment practices.

mira
Télécharger la présentation

Using the CEF R in Catalonia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using the CEFR in Catalonia Neus Figueras nfiguera@xtec.cat

  2. The EOI system • State funded language schools (+16) • Two levels defined (aimed at B1 and B2) • Published curricula • 13 different languages • 40,000 students • 16,000 certificate exams every year • Standardised certificates since 1995

  3. Issues to be solved in 2002-1 • Revise certificate examinations. • Is examination difficulty equivalent across time? • Is the lower certificate consistently easier than the higher certificate? • Are the different certificates in the different languages comparable?

  4. Issues to be solved in 2002-2 • Develop level specifications related to the CEFR. • Revise existing curricula in relation to the CEFR. • Link certificates to CEFR levels.

  5. Project Overview 2003-2007 • Project design • Empirical scale development • Item banking (English) • Manual procedures for linkage, (Specification Standardisation, Empirical validation) • Defining and exemplifying A2 • Developing curriculum objectives for A2, B1 and B2 • Developing test specifications for A2, B1 and B2

  6. CEFR

  7. Challenges • Where to start? • Involve teachers (and item writers). • Improve existing practice. • Bring in the ELP onto the project. • How? Need to count on experts. • Limited resources.

  8. Scale development (all languages)

  9. CEFR Methodology Step 1: selecting level descriptors. Step 2: translation into Catalan. Step 3: mapping descriptors onto levels. Step 4: developing and validating new scales.

  10. Teachers involved DescriptorsTeachers* Languages • Reading 40 103 10 • Listening 46 99 10 • Speaking 80 92 12 • Writing 53 89 12 • Grammar 34 81 12 • Vocabulary 61 73 12 * Arabic, Basque, Catalan, Dutch, English (>40), French (>20), German (>15), Italian, Japanese, Portuguese, Russian, Spanish

  11. Lessons learnt from scale development • Continued training/familiarisation is necessary. • Appearances need to be checked empirically. • Exact correspondence may not be possible, but is it desirable?. • Linkage does not mean equivalence.

  12. Item Banking (English)

  13. Methodology Step 1 • Booklet development

  14. S1 S2 S3 S4 S5 L1 L2 L3 A1 A2 A3 V1 X X X V2 X X X V3 X X X V4 X X X V5 X X X V6 X X X X V7 X X X X V8 X X X X Booklet design for anchoring items (2003)

  15. Methodology Step 2 • Data collection (260-784 students per item) • Analyses: CTT and IRT (Total surviving items :301) Step 3 • Standard setting procedures: - test centered - examinee centered - annual average pass rate • Setting (provisional) cut off scores at Elemental (B1) and Aptitud (B2)

  16. TCC of the eight versions(2003)

  17. Using the Manual for linking exams (English, French, German)

  18. Challenges in Specification • How to map the examination? Different versions of the test? Specifications? • How to tackle differences of coverage (subskills)? • Who does what? • Who checks it?

  19. Challenges in Standardisation • Reference “r” materials not ready until 2005. • Differences across skills, across languages. • Assessing task vs. item levels. • No “linguistic competence” reference materials.

  20. The proposal from Dutch CEFR project • Training. • Describing texts and items according to set parameters (reading and listening). • Estimating their CEFR levels. • Pretesting the items thus labelled. • Calibrating the items. • Standard-setting on the scale coming from the calibration. • Assigning a psychometric level to the items. • Assigning a definitive level to the items.

  21. Using the Council of Europe Item CD (German)

  22. The German booklet

  23. Inter-judge Consistency in assessing items L22 R15

  24. Judges: Consistency with the Pre-Estimation

  25. Consistency pre-estimation with Empirical Results Reading Listening

  26. Developing CEFR based curricula

  27. Developing CEFR-based curricula • Focusing on what students can do. • Drafting objectives. • Defining content. • Defining assessment criteria. + • Methodological guidelines.

  28. Conclusions so far

  29. The CEFR IS a Bible ( but only in the widest sense of the word)

  30. The Manual is no book of spells

  31. There is no ONE holy Grail

  32. We learnt much more about our exams. • It has been a competence building process. • We have become less dogmatic. • We know there is further work to do and room for improvement. • Combining highly technical work with enthusiasm is crucial.

  33. How do I know if my B1 is your B1? This is my B1. What’s your B1 like?

More Related