David G. Payne, PhD Executive Director, GRE Program Higher Education Division

1. The GRE Program A brief overview and a preview of changes to come David G. Payne, PhDExecutive Director, GRE Program Higher Education Division

2. Topics for today • GRE Board and GRE Program • GRE Testing Program • General Test • Subject Tests • Services • Changes in the GRE General Test • Verbal Reasoning • Quantitative Reasoning • Analytical Writing • GRE Subject Tests • GRE Services • Questions from you

4. Biochemistry, Cell & Molecular Biology Biology Chemistry Computer Science Literature in English Mathematics Physics Psychology The GRE Testing ProgramI. The General TestVerbal, Quantitative and Analytical WritingII. Subject Tests

5. How has the GRE General Test changed in recent years? • Before October 2002 – Verbal Reasoning, Quantitative Reasoning, Analytical Reasoning • After October 2002 – Verbal Reasoning, Quantitative Reasoning, Analytical Writing • October 2006 – New GRE!

6. Skills Measured in General Test:Verbal Section • Analyze and evaluate written material and synthesize information obtained from it • Analyze relationships among component parts of sentences • Recognize relationships between words and concepts

7. Item Types in General Test:Verbal Section • Antonyms • Analogies • Sentence Completions • Reading Comprehension

8. Skills Measured in General Test:Quantitative Section • Understand basic concepts of arithmetic, algebra, geometry, and data analysis • Reason quantitatively • Solve problems in a quantitative setting

9. Item Types in General Test:Quantitative Section • Arithmetic Questions • Algebra Questions • Geometry Questions • No Construction of Proof Questions • Data Analysis Questions

10. BUILDING AN ADAPTIVE TEST TEST DESIGN

11. Test Delivery and Test Preparation • Verbal and Quantitative Sections are Computer Adaptive Tests • Test Preparation Materials are available at the GRE Website: • http://gre.org

12. Analytical Reasoning Tasks • Deductive logic: sets of 3 – 7 related conditions describing a structure of relationships upon which a series of test questions is based (e.g., conditions under which a sequence of events can be scheduled) • Inductive logic: test questions that ask the examinee to understand, analyze, and evaluate arguments (e.g., identification of an assumption)

13. Expressed desire by graduate schools for information regarding applicants Critical Thinking Skills Writing Skills High correlation of Analytical Reasoning with V and Q measures (Most of the variance in AR is shared with V and Q.) Reasons for Including Analytical Writing

14. Skills Measured in General Test:Analytical Writing Section • Articulate complex ideas clearly and effectively • Examine claims and accompanying evidence • Support ideas with relevant reasons and examples • Sustain a well-focused, coherent discussion • Control the elements of standard written English

15. Analytical Writing Tasks • Present Your Views on an Issue (45 minutes, choice of 2 topics) • Analyze an Argument (30 minutes) • Each essay is scored on a 0-6 scale using holistic scoring • Two scores for each essay • GRE Website presents directions, actual topics, scoring guide, and sample essays for both the Issue and Argument tasks (www.gre.org/gentest.html)

16. A Sample Issue Task Directions: Present your perspective on the issue below, using relevant reasons and/or examples to support your views. It is more important to allocate money for immediate, existing social problems than to spend it on long-term research that might help future generations.

17. A Sample Argument Task Discuss how well reasoned you find this argument.The University of Claria is generally considered one of the best universities in the world because of its instructors' reputation, which is based primarily on the extensive research and publishing record of certain faculty members. Several faculty members are internationally renowned as leaders in their fields. Many of the faculty from the English department are regularly invited to teach at universities in other countries. Two recent graduates of the physics department have gone on to become candidates for the Nobel Prize in Physics. And 75 percent of the students are able to find employment after graduating. Therefore, because of the reputation of its faculty, the University of Claria should be the obvious choice for anyone seeking quality education.

18. Score Scales • Analytical Writing: • Scores from 0 – 6, in half-point increments • Overall score is for the two essays • Scores are not provided for individual essays • Analytical Reasoning: • Scores from 200 – 800, in 10 point increments

19. 6 – Insightful analyses of complex ideas, logically compelling reasons, well organized, skillful sentence variety, few or no usage errors 5 – Generally thoughtful analysis of ideas, logically sound reasons, generally well organized, sentence variety conveys meaning, minor usage errors 4 – Competent analysis of ideas, relevant reasons, adequately organized, satisfactory control of sentence structure, some usage errors 3 – Some competence though flawed in one or more of the following: limited analysis or development, weak organization or control of sentence structure, usage errors that result in vagueness 2 – Serious weakness in at least one of the following: lack of analysis, development, or organization, serious problems in sentence structure, usage errors obscure meaning 1 – Fundamental deficiencies: content that is confusing or irrelevant, little or no development, pervasive errors that result in incoherence Score Level Descriptors

20. Analytical Writing Scores Score reports also contain percentile ranks for each score.The percentile rank shows the relative standing of the score compared to a reference group. For GRE, the reference group is always data from the most recent three-year period.

21. Scores and Percentiles from Different Testing Years Since the norm group changes every year, a single score (e.g., 560) may have a slightly different percentile rank in 2003 (PR 76) than in 2004 (PR 75). The reason is that the reference group for 2003 scores is based on examinees from 1998-2001 and the reference group for 2004 scores is based on examinees from 1999-2002.

23. Analytical Writing Scores for First Year of Testing

24. Analytical Writing Scores for First Year of Testing

25. Analytical Writing Scores for First Year of Testing

26. Difference Between Analytical Reasoning and Analytical Writing • Item Type • Analytical Reasoning: Multiple-choice questions administered as a computer-adaptive test • Analytical Writing: Two essays on assigned topics • Skills Assessed • Deductive and inductive logic • Critical writing and thinking skills

27. Score Interpretation of Analytical Reasoning and Analytical Writing • Multiple-choice Analytical Reasoning test • cannotbe compared with Analytical • Writing test. • Interpret each test using percentile ranks • on score reports • Score distributions for AW are available • on GRE Web site • www.gre.org/interpret.html)

28. Changes Planned for GRE General Test • Goals of the Revision • Increase the validity of the test • Evidence Centered Design • New Item Types and Tasks • Decrease Effects of Memorization • Address security concerns • Provide faculty with better information on applicants performance • Essay Distribution • Make better use of computer-enabled items • New GRE will be launched October 2006 • Revisions to Verbal, Quantitative and Analytical Writing Sections

29. Changes Planned for the Verbal Section • Less reliance on vocabulary, more on higher cognitive skills • More text-based materials such as reading passages • A broader selection of reading passages • Emphasis on skills related to graduate work such as complex reasoning • Expansion of computer-enabled tasks (e.g., highlighting similar ideas in two passages)

30. Changes Planned for the Quantitative Section • More depth of quantitative reasoning skills • More breadth of quantitative reasoning skills • Question types that require production, rather than selection, of answers • Expansion of computer-enabled tasks (e.g., producing an equation)

31. New, more focused prompts Prompts will reduce possibility of reliance on memorized materials Prompts may be used only once (still TBD) Both Argument and Issue tasks will be 30 minutes GRE Program and Board will examine feasibility of Essay Distribution to Designated Institutions Changes Planned for the Analytical Writing Section

32. Test Delivery • From now until October 2006 • Verbal and Quantitative sections will continue to be computer-adaptive tests worldwide, except • China, (including Hong Kong), Korea, Taiwan • Analytical Writing will continue to be a computer-based test worldwide • Starting in October 2006 • Verbal and Quantitative sections will be computer-based linear tests worldwide • Analytical Writing will continue to be a computer-based test worldwide

33. GRE Services for Institutions • Search Service – a recruitment tool to • locate prospective students • Institutional Summary Statistics – a • summary of performance data on groups • of examinees • Test preparation workshops for educators • – free materials and advice about setting • up workshops for students on campus

34. GRE Services for Students • Diagnostic Service – feedback for examinees on their performance on skills measured in the General Test • Fee Waiver Program – fee waivers for tests, publications, and services • Question and Answer Review Service – examinees can review questions answered incorrectly

35. Current GRE Subject Tests • Biochemistry, Cell and Molecular Biology • Biology • Chemistry • Computer Science • Literature in English • Mathematics • Physics • Psychology

36. Subject Tests • Most Subject Tests are offered three times a year (November, December, April) • Free test preparation is available at test registration. • GRE Web site contains details about the content of each test (www.gre.org/subjtest.html)

37. Test Predictors of GGPA U S VQA SU VQAU VQASU Biology .33 .37 .27 .43 .38 .45 Chemistry .36 .51 .30 .58 .43 .63 Lit. in English .34 .32 .29 .47 .41 .54 Physics .28 .27 .20 .45 .31 .49 Psychology .37 .37 .33 .46 .44 .50 Predictive Validity of Subject Tests

38. “A Comprehensive Meta-Analysis of the Predictive Validity of the Graduate Record Examinations: Implications for Graduate Student Selection and Performance” Nathan R. Kuncel Sarah A. Hazlett Deniz S. Ones Psychological Bulletin 2001, Vol. 127, No. 1, 162-181 “The results indicated that the GRE and UGPA are generalizably valid predictors of graduate grade point average, 1st-year graduate grade point average, comprehensive examination scores, publication citation counts, and faculty ratings.” “Subject Tests tended to be better predictors than the Verbal, Quantitative, and Analytical tests.”

39. GRE New Product Ideas • Pre-GRE • Standalone Analytical Writing • Masters Admission Test • Critical Thinking • Analytical Reasoning • Standardized Letter of Recommendation

40. We welcome feedback! • Questions/Comments from audience OR • E-mail me at: Dpayne@ets.org