1 / 13

Cross-language comparability of language tests: Year 12 examinations in Estonia

The Third EALTA Conference Krakow, 19–21 May 2006. Cross-language comparability of language tests: Year 12 examinations in Estonia. Ülle Türk ulle.turk@ut.ee. Research question. How to ensure the comparability of different foreign language Year 12 examinations?

gema
Télécharger la présentation

Cross-language comparability of language tests: Year 12 examinations in Estonia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Third EALTA Conference Krakow, 19–21 May 2006 Cross-language comparability of language tests: Year 12 examinations in Estonia Ülle Türk ulle.turk@ut.ee

  2. Research question • How to ensure the comparability of different foreign language Year 12 examinations? • Why would the examinations need to be comparable? • What does comparability mean in this particular case? ulle.turk@ut.ee

  3. Overview • Background to Year 12 examinations • Year 12 examinations in foreign languages • Year 12 examinations in English and German • Reading • Conclusions and way forward ulle.turk@ut.ee

  4. Year 12 examinations = national school-leaving examinations established in 1997 • centrally developed, administered and marked • test the learning outcomes of the National Curriculum (2002) • contain tasks at different levels of difficulty • results on a 100-point scale • offered in 13 subjects • four foreign languages:English, French, German, Russian • students required to take three examinations:mother tongue + two others ulle.turk@ut.ee

  5. Foreign language examinations • National Curriculum for foreign languages • two compulsory foreign languages • FL A: starting in grades 1–3; 3 hrs per week/ 2 hrs per week • FL B: starting in grades 4–6; 3 hrs per week/ 2 hrs per week • level B2 in ONE foreign language by the end of Year 12 • Ministry regulations on the development, administration and grading of examinations and reporting results • five papers: listening and reading comprehension, speaking, writing, language structures • equally weighted (20 points each) • High-stakes examinations • results used for university entrance • foreign language requirement: English, French or German ulle.turk@ut.ee

  6. Questions • Does 93 in English reflect a significantly higher level of competence than 92? • Does 93 in the examination of 2006 reflect the same level of competence as 93 in the examination of 2004 (or 2001)? • Is 63 in English the same as 63 in German, French or Russian? • What does ‘the same’ mean? • The same level of competence?  B2 • The level of competence reached after the same amount of work/ same number of classroom hours? • US Foreign Service Institute (Jackson & Kaplan, 1999: 78) • US Defense Language Institute Foreign Language Center (MacWhinney 1995: 294) • Threshold Level (Trim) ulle.turk@ut.ee

  7. Year 12 examinations in FLs 2001 2004 Mean % No No Mean% English 8488 64.1 9099 66.6 1408 1076 German 66.0 66.7 445 509 69.4 72.3 Russian 75.4 79.6 French 67 63 ulle.turk@ut.ee

  8. German & English: reading paper • 50 minutes • Three texts with tasks • Length of texts: 1500 words • No of tasks: • German: one task per text • English: one or two tasks per text • No of items • German: 20 • English: 40 ulle.turk@ut.ee

  9. Reading: mean scores Lang Year Paper Text 1 Text 2 Text 2 English 2001 61% 71% 72%/66% 69%/35% 2004 68% 78% 75% 57% German 2001 61% 67% 67% 55% 2004 60% 84% 62% 46% ulle.turk@ut.ee

  10. Text types German English 2001 & 2004 2001 2004 Short adverts Text 1: Newspaper interview Opinion article Text 2: Short news/ opinion articles Short book reviews Magazine article Text 3: Newspaper/magazine article (600–700 words) Newspaper article Newspaper article ulle.turk@ut.ee

  11. Task types German English 2001 & 2004 2001 2004 Task 1: matching (person + advert) matching (question + answer) 1) inserting sentences into text 2) multiple choice: 3 Task 2: matching (text + heading) 1) identifying important information 2) matching (definition + word) identifying important information Task 3: multiple choice (4 options) 1) matching (text + title) 2) summary cloze 1) T/F/NI 2) matching (definition + word) ulle.turk@ut.ee

  12. Conclusions and way forward • The reading paper in English seems more difficult than that in German (B2/C1?  B1/B2?) • Perhaps this should be the case? • What about the National Curriculum? ?Relate examinations of each language to the CEFR levels ?Study the rate of acquisition of different foreign languages ulle.turk@ut.ee

  13. Sources • Hardcastle, Peter. 2004. Test Equivalence and Construct Compatibility across Languages. University of Cambridge ESOL Examinations Research Notes, 17, August, 6 − 11. • Jackson, Frederick H. & Marsha A. Kaplan. 1999. Lessons learned from fifty years of theory and practice in government language teaching. In: Georgetown University Round Table on Languages and Linguistics. Washington, DC: Georgetown University Press, 71−87. • MacWhinney, Brian. 1995. Language-Specific Prediction in Foreign Language Learning. Language Testing, 12, 292−320. ulle.turk@ut.ee

More Related