An Empirical Study of the Reliability and Validity of Proofreading and Error Correction items of Old and New Versions

1. An Empirical Study of the Reliability and Validity ofProofreading and Error Correction itemsof Old and New Versions in TEM-8 娄 喜 祥 福建省漳州师范学院外语系 2007.5.4

2. Contents • The necessities for such a research • Research questions • The methodology of the research • The presentation of the data in the research • The conclusion of the research • Bibliography

3. The necessities for such a research • Purpose for the research in the field of language testing • The relationship between the reliability and validity of a test • The commonness of Proofreading Item in language testing papers • The changes in the format of such items in TEM-8

4. Research questions • What are the reliability coefficients of proofreading items of new and old versions in TEM-8 ? • What are the validity coefficients of proofreading items of new and old versions in TEM-8 ? • Are there any differences between new and old version TEM-8 test in their reliability and validity?

5. Methodology(1) • Experiment design: Using the two versions of Proofreading and error correction items to test the same group of students, then calculate their reliability and validity coefficients respectively. • Subject: • 67 examinees are from Grade 2002, DLF of the writer’s school, 12 males, 55 females. They are in the first half school year of the fourth grade. And the experiment was carried out at the end of such a period.

6. Methodology(2) • Materials and instruments 1) Two versions of model MET-8 Proofreading and Error Correction tests: The old version is composed of two passages from the book edited by Liu Yuzhen (刘玉珍,1995) and the new-version one is comprised of three passages from the book edited by Zhang Chong (张冲, 2003). 2) The objective items of the final examination of that semester for Advanced English, including the items testing on vocabulary, rhetoric knowledge, reading comprehension, etc.is taken as the criterion to measure the criterion-related validity of those two tests. 3) The statistical software: SPSS 13.0

7. The descriptive statistics of the results of the students in these two tests are listed here.

8. The Paired Sample T-test on the difference between the scoring ratios of the students in these two different version tests

9. Parallel form reliability of these two version tests • Of the Old version test • Of the New version test

10. The correlation coefficients between these two tests and the final exam * Correlation is significant at the 0.05 level (2-tailed). ** Correlation is significant at the 0.01 level (2-tailed).

11. Conclusion of the research • The experiment has shown that the reliability of old version TEM-8 Proofreading and Error Correction before 1996 is a little higher than the new version one since 1996. • The validity of the new version one since 1996 is much higher than the old version one before 1996.

12. Shortcomings of the research • ． 本文虽发现了新旧短文改错题的差异，但本实验仍存在着不足：一、本实验为了避开学生熟悉往年真题而采用了模拟题，有可能与真题有差异；二、本实验是在随堂测试的条件下实施的，这与实际的考试也可能有差异；三、本实验的被试由于有少数学生缺考，也会造成差异。 所以，本文的结论仍须经进一步的验证。

13. Bibliography • 1.Wood, R. (2001). 评估与测试:研究综述[M]. 北京: 外语教学与研究出版社. • 2.舒运祥. (2001). 外语测试的理论与方法[M]. 上海: 上海世界图书出版公司. • 3.刘玉珍. (1995). 高校英语专业八级统考模拟试题集[M]. 天津: 南开大学出版社. • 4.刘润清, & 胡壮麟. (1991). 语言测试和它的方法[M]. 北京: 外语教学与研究出版社. • 5.张厚粲, & 徐建平. (2003). 心理与教育统计学[M]. 北京: 北京师范大学出版社. • 6.张冲. (2003). 英语专业8级水平测试习题集[M]. 上海: 上海外语教育出版社. • 7.马广惠. (2003). 外国语言学及应用语言学统计方法[M]. 杨凌: 西北农林科技大学出版社.

14. Thank you for your attention! Welcome any questions!