Test Construction Processes

1. Test Construction Processes • 1- Determining the function and the form • 2- Planning( Content: table of specification) • 3- Preparing( Knowledge and experience) • 4- Reviewing( possible problems) • 5- Pretesting( item and test analysis ) • 3- Validating( reliability and validity) • 4-Reviewing • 1- To find possible problems after writing • 2-better to be done by the same test constructor after some time interval • 3- or to be reviewed by another test constructor specially a native educator

2. 5- Pretesting Definition: Administering the newly developed test to a group of examinees having the same characteristics as of the target examinees. Purposes: 1- to find some characteristics of items such as Item facility and item difficulty Item discrimination Choice distribution( choices ) 2- to find some characteristics of a test such as reliability validity Practicality Item facility 1- Definition: How easily an item is answered correctly by testees. 2- Formula: IF= ∑correct responses /the number of testees. IF= 0 : No correct responses. Very difficult question. IF= 1 : very easy question. Accepted range: 0.37< IF < 0.8 Experts recommend that the average level of difficulty for a four-option multiple choice test should be between 60% and 80%. Item difficulty is the percentage of students who answered a test item wrongly( Item difficulty: 1- IF ) or ∑wrong answer/ n

3. Note: Very easy items and very difficult items don’t do a good job of discriminating between students who know the content and those who do not. They should be modified or discarded. However, you may have very good reason for putting either type of  question on your exam. For example, some instructors deliberately start their exam with an easy question or two to settle down anxious test takers or to help students feel some early success with the exam. Item discrimination: 1- definition: The degree to which an item discriminates between high achieving students and low achieving students. In other words, how well the item serves to discriminate between students with higher and lower levels of knowledge. A basic consideration in evaluating the performance of a normative test item is the degree to which the item discriminates between high achieving students and low achieving students. It is often referred to as Item Effect, since it is an index of an item’s effectiveness at discriminating those who know the content from those who do not.

4. 2- Formula: ID= HC- LC / ½ n Procedures to calculate ID: a. rank scores in descending. b. group the subjects into two equally numbered groups( high and low groups). c. use the formula. ID: item discrimination HC: correct responses of the high group LC: correct reponses of the low group n: all respondents. 3- Interpretation of calculation 1- range: -1 ≤ ID ≤ 1 2- Acceptable range: 0.4 ≤ ID 3- 0.6 ≤ ID ≤ 0.7 discriminates well. 4- ID= 1 discriminates perfectly. 5- ID= -1 discriminates perfectly in the wrong direction( question discarded or modified). 6- ID= 0 doesn’t discriminates.

5. Distractors A multiple choice question skeleton: A traditional multiple choice question (or MCQ) is one in which a student chooses one answer from a number of choices supplied. A multiple choice question consists of • Stem - the text of the question • Options - the choices provided after the stem • Key - the correct answer in the list of options • Distractors - the incorrect answers in the list of options. The function of distractors Distractors play a vital role for the process of multiple choice testing, in that good quality distractors ensure that the outcome of the tests provides more credible and objective picture of the knowledge of the testees involved. On the other hand, poor distracters would not contribute much to the accuracy of the assessment as obvious or too easy distracters will pose no challenge to the students and as a result, will not be able to distinguish high performing from low performing learners.

6. Item analysis and choice distribution: Test results can be used to identify and remove non-functioning distractors from tests. If properly constructed, MCQs are able to test higher levels of cognitive reasoning and can accurately discriminate between high- and low-achieving students. One aspect where many MCQs fail is in having effective distractors. Teachers often spend a great deal of time constructing the stem and much less time on developing plausible options to the correct answer. Nonfunctioning distractors are options that are selected infrequently (<5%) by examinees. Thus , the two factors that affect the ability of an exam to discriminate between levels of student ability are: (1) the quality of individual test items, and (2) the number of test items. The parameter that is useful in analyzing the quality of an individual test question is the proportion of the students who choose a particular answer to the question. This parameter is called choice distribution and is grouped under the title item analysis.  Therefore choice distribution in MCH questions refers to the frequency with which a choice is selected by testees. Each choice should attract reasonable number of students, not too many or limited number.

7. Interpretation of choice distribution: • An item even with acceptable amount of facility and item discrimination may not be considered a good item since some of its choices may not attract an acceptable number of testees. • Item#1 • They've  got  ………….. money  that they don't know what to do with it . • a)enough                 b)so many               •     c)too much                       d)so much • Item#2 • Air ……...is one of the most important problems in big cities • a) extinction b) pollution • c) temperature d) action

8. Interpretation of choice distribution: A choice is ineffective or nonfunctioning, if it attracts no testees or doesn’t differentiate among testees. A choice is effective or functioning, if it attracts less knowledgeable more than knowledgeable ones. A distractor is malfunctioning, if the number of hard working students outnumbers less knowledgeable ones. Apply Logic 1- principles( item analysis and procedures) are all suggestive rather than prescriptive. 2- Decide to include or exclude an item within a test or administer a special type of test based on your class and students circumstances Thanks for listening