Oklahoma Core Curriculum Tests

1. Oklahoma Core Curriculum Tests General Item Writer Training Session February 14, 2008

2. Meeting Goal To provide information about the DRC item writing process and our guidelines for writing sound items that assess the construct being taught.

3. Item Writers • Selection criteria • Contracts and Confidentiality statements

4. Oklahoma PASS Oklahoma Test and Item Specifications Oklahoma Style Guide DRC Item Writer Training Manual DRC Fairness in Testing Manual DRC Universal Design Guide Item/Passage Writing Templates Item Writer Training Materials

5. Parts of an Item • Multiple-choice items • Stem • Response Options • Correct Answer • Distractors • Graphics/Artwork (if applicable) • Distractor Rationales (mathematics)

6. Components of an Item The lengths of some of the biggest ships ever built are shown in the table below. Graphic Stem Question What is the range of the lengths of the ships in the table? (A) 105 [last length – first length] (B) 263 (C) 775.5 [mean] (D) 795.5 [median] Options Rationales Distractors

7. Stems • Present one clearly stated question • Include information that avoids a repeat in each option • Are clear and concise • Use the active voice • Use grade-level appropriate vocabulary • Include only what is necessary • Do not cue the answer or miscue a distractor

8. Distractors and Answer • Distractors (Options) Should • include only what is necessary • be parallel in structure • not contradict or negate the stem • be as brief as possible • fit grammatically and syntactically with the stem • Answers • There should only be one correct answer • All distractors should be plausible

9. Graphics/Artwork • Graphics/Artwork • Stimuli are necessary and functional • Stimuli are clear, simple, and considerate of students with special needs • Stimuli are properly labeled • Stimuli are complete • Content is current and accurate • When used as options, are parallel in style and layout

10. Good Items Should • Measure the intended skill or standard • Be consistent with item writer guidelines and specifications • Be of appropriate difficulty for the grade level • Be grammatically correct • Assess meaningful concepts • Assess comprehension of the concept

11. Content of Items Should • Allow students to show their understanding of the curriculum being tested • Test knowledge and skills that are important • Provide passage-dependent connections for reading test.

12. Style • Follow the Oklahoma style guide • Be consistent in tone

13. Ensuring Fairness for All Students Standard 7.4: Test developers should strive to identify and eliminate language, symbols, words, phrases, and content that are generally regarded as offensive by members of racial, ethnic, gender, or other groups, except when judged to be necessary for adequate representation of the domain. Standards for Educational and Psychological Testing (AERA, APA, NCME, 1999)

14. Definition of Bias Bias is the presence of some characteristic of an item that results in differential performance for two individuals of the same ability but from different sub groups.

15. Stereotyping • Depicting members of either gender • Experiencing stereotyped emotions • Participating in stereotyped activities • Working in stereotyped occupations • Portraying stereotyped characteristics • Portraying stereotyped situations • Using demeaning labels • Showing stereotyping through pictures

16. Gender Bias • Titles and specific terms referring to humanity in general (manpower) • Gender-specific terms for occupations (fireman) • Use of pronouns that imply a stereotype • The nurse went to the hospital, and she was able to talk with the patient. • Use of phrases that identify genders in terms of their roles or occupations • The travelers took their wives and children with them.

17. Gender Bias (continued) • Use of words and phrases that are not parallel • The men’s department is on the right; the ladies’ department is on the left. • Use of figures of speech • Old wives' tale • Man versus nature • Use of gender-specific terms • Career girl

18. Regional • Terms not commonly used nationwide or used only within a particular region or state • Davenport • Toboggan • Bubbler • Pike • Soda

19. Ethnic or Cultural • Terms that are demeaning and/or offensive to a particular ethnic group or culture • Terms that depict a particular group as poor or uneducated

20. Class or Socioeconomic • Activities, possessions, or ideas that are not common to all students within a given strata • Country club • Snow skiing • Town home or loft • European vacations • Private swimming pool

21. Religious or Cultural • Terms that are demeaning and/or offensive to a particular religious group • Halloween • Rock-and-roll dance • Terms that portray a particular belief or practice as the norm or as inferior or superior to any other

22. Other Types of Bias • Ageism (bias against a particular age group) • Irresponsible teens • Bias against persons with disabilities • Despite disability, she was able to . . . • Content should show people in everyday situations, and groups should be depicted as fully integrated in the society

23. Experiential • The questions and activities reflected in the questions should be relevant to the life experiences of the persons responding to the items. • For a student to respond sensibly to test questions, he/she must know what the question is talking about.

24. Experiential examples • 1 under par • Tossed the hat in the ring • Subway turnstile • Took the skyway • Running neck-and-neck

25. Sensitivity Concerns • Sensitive and offensive topics should be avoided • Also to be avoided are • Highly charged or controversial topics • Topics and/or items that appear to promote or defend a particular set of values • References to tragedies or hardships, such as floods or earthquakes, tornados, dust storms

26. Special Circumstances • Historical Contexts • Literary Contexts

27. Universal Design Universally designed assessments are based on the premise that each student in school is a part of the population to be tested and that testing results should not be affected by disability, gender, race, or English language ability.

28. Universal Design • Test questions have concise and readable text. • Simple, clear, commonly used words are used whenever possible. • Unnecessary words and extraneous text are omitted. • Vocabulary and sentence complexity is appropriate for the grade level assessed. • Technical terms and abbreviations are used only if they are related to the content being measured. • Definitions and examples are clear and understandable.

29. IDEAS Item Banking System Item Development and Educational Assessment System (IDEAS) • Houses OCCT items • Web-based, secure server • Maintains “history” of an item • Item editing tool • Forms construction tool • Delivery system to our online partner CAL

30. IDEAS Item Banking System • Information placed on item writer template • Coded electronically in IDEAS • Editing occurs within system • Document specific information and/or consider certain criteria as each item is developed.

31. Content Presentations • Mathematics • Reading • Science • Social Studies

32. Questions

33. Oklahoma Core Curriculum Tests