Objectives:
E N D
Presentation Transcript
Objectives: • Introduction to Item Analysis • To begin to understand how to identify items that should be improved or eliminated.
Item Analysis • No item is perfect. • An item might be ambiguous, too simple, too difficult, or non-discriminating. • Non-discriminating means the item can not be used to measure individual differences on the trait that is measured by the test.
Achievement Tests • Item analysis can help diagnose student errors. • It can help improve the quality of tests. • It can lead to instructional improvements.
Achievement Tests • Item analysis can identify problems with the answer key on a teacher-made test, or problems with the machine scoring on a standardized test. • It can help isolate items where the students “guessed” a lot.
The Basic Indexes of Item Analysis • Difficulty – What percentage of respondents got the item “right” or indicated that they possess the trait being measured? • Discrimination – The extent to which the item differentiates between persons with high and low scores on the test.
The Basic Indexes of Item Analysis • Difficulty – Measured by a simple percentage. • Discrimination – Measured by the difference between high and low scoring groups on the proportion answering the “right” answer.
Discrimination • Items that are poor discriminators should be eliminated or modified. • Balance content validity with construct validity. • High discrimination tends to increase reliability.
Discrimination and Difficulty • Discrimination and difficulty are related. • With very difficult items it is harder to show high discrimination. • Balance purpose of assessment with the range of difficulty of items. • Generally .2 - .8 difficulty is desired.
Discrimination • For educational achievment tests, you want to look at the discrimination for the “distractors” or wrong options on a multiple choice item. • Ideally, you want them to be selected mostly by the low scoring respondents.
Discrimination • For Educational tests – • Form a 2 x 2 matrix that crosses “Right” vs. “Wrong” on the item by “High” vs. “Low” on the total score of the test. • High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile.
Discrimination • For Psychological tests – • Form a 2 x 2 matrix that crosses “High” vs. “Low” on the item by “High” vs. “Low” on the total score of the test. • High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile.
An Example from the PRI • “I am able to ask for emotional support.” • Part of the Social Resourcefulness Factor • Part of the Assistance in Relationships subscale.
