Questions and Answers from ETS about the TOEFL iBT

1. IEP/HE SIG Forum The TOEFL iBT: Three Years Later CoTESOL “Welcoming the World” October 23, 2008 Questions and Answers from ETS about the TOEFL iBT

2. The TOEFL iBT

3. The TOEFL iBT • Reading

4. The TOEFL iBT • Reading • Listening

5. The TOEFL iBT • Reading • Listening • Break

6. The TOEFL iBT • Reading • Listening • Break • Speaking

7. The TOEFL iBT • Reading • Listening • Break • Speaking • Writing

8. The TOEFL iBT • Reading • Listening • Break • Speaking • Writing Total test time: 4 hours

9. The TOEFL iBT • Reading: 3-5 passages, 20 minutes each • Listening • Break • Speaking • Writing

10. The TOEFL iBT • Reading: 3-5 passages, 20 minutes each • Listening: 6-9 passages, 60-90 minutes total • Break • Speaking • Writing

11. The TOEFL iBT • Reading: 3-5 passages, 20 minutes each • Listening: 6-9 passages, 60-90 minutes total • Break: 10 minutes • Speaking • Writing

12. The TOEFL iBT • Reading: 3-5 passages, 20 minutes each • Listening: 6-9 passages, 60-90 minutes total • Break: 10 minutes • Speaking: 6 questions, 45-60 second responses • Writing

13. The TOEFL iBT • Reading: 3-5 passages, 20 minutes each • Listening: 6-9 passages, 60-90 minutes total • Break: 10 minutes • Speaking: 6 questions, 45-60 second responses • Writing: 2 questions, 50 minutes total

14. TOEFL Score Comparisons CBT 0-40 97 173 213 250 300 iBT 0-9 32 61 79-80 100 120 PBT 310 400-403 500 550 600-603 677

15. General Questions

16. Question 1 • Does ETS recommend minimum TOEFL iBT scores for students entering undergraduate and graduate programs? If not, how do universities choose minimum entry scores for the TOEFL iBT?

17. Answer 1 No, we don’t. Because the needs of different universities and programs are so varied, ETS does not recommend entry scores, but provides information to institutions in setting scores for their particular circumstances.

18. Answer 1 Here is a link to the information on the ETS website: http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=39282275562cb110VgnVCM10000022f95190RCRD&vgnextchannel=ee76d0b941abb110VgnVCM10000022f95190RCRD

19. Answer 1 At the highest level, ETS offers institutions 3 alternative methods of setting scores:

20. Answer 1 • Use the TOEFL iBT standard-setting CD-ROM. E-mail TOEFLNews@ets.orgor call +1-609-683-2008 to obtain the CD.

21. Answer 1 • Use the TOEFL Score Comparison Tables to find which TOEFL iBT scores are comparable to your score requirements on TOEFL PBT. We also encourage universities to look at the annual TOEFL test and Score Data Summary which includes student performance on the test by native language, native country, gender and purpose for taking the test as well as percentile data. This can be very helpful when comparing iBT score requirements with previous CBT or PBT requirements.

22. Answer 1 Score Data Summary: http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=20beaf5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRD

23. Answer 1 OR • Go to  www.ets.org/toefl • Select "TOEFL Research" on the left hand side of the screen

24. Answer 1 • Consider the score requirements set by peer institutions.

25. Question 2 • Are universities satisfied with the score setting or do they think it needs some adjustment with consideration given to the different sections?

26. Answer 2 We are encouraging universities to review their score requirements now that students have been admitted with TOEFL iBTscores, and ETS had carried out academic studies, so there is actual test taker performance data in the annual Test and Score Data Summary. This is good score use practice and we support it.

27. Question 3 • What type of feedback is ETS receiving from ESL professionals, universities, and students on the TOEFL iBT?

28. Answer 3 Overall, the feedback is quite positive. In the beginning, we had some problems getting enough seats to meet the demand, but that problem is easing now. We get positive feedback about some of the changes we have made in TOEFL iBT, such as adding the speaking measure, adding integrated items, eliminating discrete grammar items, and allowing note taking.

29. Answer 3 Here is a link to information on our website, with further links to news stories about TOEFL iBT: http://www.ets.org/portal/site/ets/menuitem.3c27603a8b4042d69a77b13bc3921509/?vgnextoid=811ed898c84f4010VgnVCM10000022f95190RCRD Some market research we recently conducted brought us the complaint from a student that you actually had to learn English to pass TOEFL, while with other tests you could just do test prep. We took that as a compliment!

30. Question 4 • What percentage of test takers challenge their scores? Does ETS have a percentage for successful challenges?

31. Answer 4 Sorry, this is proprietary information that we don’t publish.

32. Answer 4 Here is information on how to apply for a rescore: http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=1b01fa5947946110VgnVCM10000022f95190RCRD&vgnextchannel=c025a1e13bf36110VgnVCM10000022f95190RCRD

33. Answer 4 OR • Go to www.ets.org/toefl • Select "Test Scores" • Select "Internet-based Test" • Pick the tab that says "Cancel, Reinstate, Rescore"

34. Question 5 • When are the PBT and CBT going to be phased out in the U.S? Are there plans to phase out these tests abroad?

35. Answer 5 The CBT test ended in July of 2006 and because scores are valid for 2 years, institutions should not be receiving CBT scores any longer. So CBT for TOEFL is now history. 95% of all TOEFL testing today is Internet-based.

36. Answer 5 The PBT test is currently offered in countries without the infrastructure to support the iBT and for the foreseeable future we will need to continue with it. We monitor this situation regularly because our goal is to bring TOEFL iBT to those countries as quickly as possible. Until we can, we’ll continue to offer the paper-based TOEFL, in order to serve students in those locations.

37. Answer 5 In addition, we used the paper-based TOEFL in a very limited way to support the TOEFL iBT test center network. Now that the TOEFL iBT testing network has matured, 3 years into the process, we don’t use it very much since 95% of all our testing is Internet-based. However, students can still be admitted to a university with a PBT score.

38. Answer 5 The PBT is still offered in the U.S. 6 times per year in cities such as San Francisco, New York, and Boston. You can go to ETS’ website and search for PBT sites by clicking on each state individually and searching for “paper-based test” on the list of testing centers: http://www.ets.org/bin/getprogram.cgi?test=toefl

39. Question 6 • Have you had native English speakers take the TOEFL iBT? If so, does ETS have any data how native speakers perform on the test?

40. Answer 6 Yes, we have administered TOEFL to native speakers, but the report has not been published yet. There is a published research report on the performance of native speakers on CBT, available in PDF format at: http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=43b1af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRD

41. Answer 6 Some high level findings in the research on the iBT and native speakers (though not published yet) were: • Overall, native speakers performed better on the TOEFL than non-native speakers, but did not “ace” it as a group; • The Reading and Writing Sections showed moderate differences between native speakers and non-native speakers; • The Listening section showed greater differences between native speakers and non-native speakers; • The Speaking section showed the greatest differences between the native speakers and non-native speakers. The lowest score on the rubric for Speaking was rarely given to non-native speakers.

42. Answer 6 The results of this study are still being analyzed and the report being written, but research reports are put on the ETS website in PDF format when they are published, so those who want more detail could check there periodically for this report: www.ets.org.

43. Answer 6 If you are interested in ETS’ ongoing research for TOEFL, here’s a link for a framework of recent research. You’ll see it’s on a variety of topics to ensure test quality. http://www.ets.org/Media/Research/pdf/Framework_Recent_TOEFL_Research.pdf

44. Question 7 • How successful does ETS consider the TOEFL iBT? What are some challenges ETS faces with this new format?

45. Answer 7 We’re very pleased with the TOEFL iBT test. Overall test reliability ranges between .94 and .95 and the new integrated questions are performing as we expected in terms of discriminating those with greater or lesser proficiency. The new integrated questions and the way TOEFL does scoring for the Speaking and Writing sections are what we are probably most proud of.

46. Answer 7 All test results, including the spoken and written responses, are sent to ETS via the Internet after the test and are then distributed to our online scoring network for scoring. Each written response is scored by two raters and spoken responses are distributed to between 3-6 different scorers. ETS separates the testing and scoring environments to maintain security, validity and reliability. Thus, there is no possibility the scorer may know the test taker, and scorers evaluate worldwide responses.

47. Answer 7 Some of the challenges we have faced with TOEFL iBT are:

48. Answer 7 • Building a test center network for TOEFL iBT. For CBT we had 617 test centers. Today we have grown to more than 4000. Recruiting, training and supporting that number of secure test centers to ensure we had sufficient test center space required great effort. Three years on, the network is now mature and we’ve managed to move beyond the initial learning curves.

49. Answer 7 • Ensuring we have sufficient raters. On TOEFL CBT, test takers were asked to write one essay and there was no speaking section. Now we need scores for 2 essays and 6 spoken responses for each student.

50. Answer 7 • Reliable electricity supply. As mentioned earlier, we need to have a reliable electrical supply in-country and at least intermittent Internet supply to offer the iBT test.