QUALITY ASSURANCE IN OBJECTIVE TESTING

1. QUALITY ASSURANCE IN OBJECTIVE TESTING Isaac Olakanmi Abimbola Department of Science Education University of Ilorin Ilorin, Nigeria abimbola@unilorin.edu.ng

2. OUTLINE • Introduction • Types of Test • Essay • Objective • Forms of Objective Test • Quality Assurance • Ethical Principles in Evaluation • Conclusion

3. INTRODUCTION • Lesson objectives guide instruction • Whatever is implemented in instruction should guide testing • Teachers should only test what was intended for learning and what was actually taught • Lesson objectives Instruction Test • A Table of Specification/Test Blue-print ensures correspondence

4. TYPES OF TEST • Essay Type • Objective Type

5. TYPES OF TEST (CONTD.) • Essay Tests • Require long, detailed answers • Contain more than one question • Answered in writing only

6. TYPES OF TEST (CONTD.) Objective Tests Objectively scored/ layperson/machine scored Reliably scored Adequate content sampling It can be biased!

7. TYPES OF TEST (CONTD.) • Forms of Objective Test • 1. Two-choice items • (i) True/False items • (ii) Completion items • 2. More than two-choice items • (i) Matching items • (ii) Multiple-choice items

8. TYPES OF TEST (CONTD.) • Forms of Objective Test (Contd.) • 1.Two-choice items • (i)True/False items Shooting questions Statements to be judged and identified by examinees as True or False

9. Forms of Objective Test (Contd.) • 1. Two-choice items • (i) True/False items (Contd.) • Example: • The name of the first spacecraft launched in 1957 by the former Soviet Union is Sputnik I _____ (True/False)

10. Forms of Objective Test (Contd.) • 1. Two-choice items • (i) True/False items (Contd.) • Quality Assurance • Ensure balance between true and false items • Each statement should be clearly true or false, not partially true or false

11. Forms of Objective Test (Contd.) • 1. Two-choice items • (i) True/False items (Contd.) • Quality Assurance (Contd.) • Avoid double negative items • Avoid long and complex statements or propositions • Measure only one idea in one statement—no double-barreled item

12. Forms of Objective Test (Contd.) • 1. Two-choice items • (i) True/False items (Contd.) • Quality Assurance (Contd.) • Use one judgement mode at a time—True or False, Yes or No, Correct/Incorrect • Avoid direct quotations from books or other materials • Do not give clues

13. Forms of Objective Test (Contd.) • 1. Two-choice items • (ii) Completion Type (Two choices against each blank) • Example: • Human beings breathe in ____. (Air/Oxygen)

14. Forms of Objective Test (Contd.) • 2. More than two-choice items • (i) Matching items • A two-column format containing premises and responses • The response column is shuffled against the premise column • Examinee matches each premise with a corresponding response • Measures understanding of concepts, principles and classification schemes

15. Forms of Objective Test (Contd.) • 2. More than two-choice items • (i) Matching items (Contd.) • Example: • Match the weather instruments in Column 1 with their uses in Column 2 by writing the appropriate letters (A-F) in the spaces provided:

16. Forms of Objective Test (Contd.) • 2. More than two-choice items • (i) Matching items (Contd.) • Example (Contd.):

17. Forms of Objective Test (Contd.) • 2. More than two-choice items (Contd.) • (i) Matching items (Contd.) • Quality Assurance • Premise should be on the left-hand column with numbers and responses on the right-hand column with letters • Items should be homogeneous • All items should be on the same page • Responses should be more than premises • Provide no clue in the two columns

18. Forms of Objective Test (Contd.) • More than two-choice items (Contd.) • (ii) Multiple-choice Items • Contain two parts—Stem and alternatives/options • Stem could be a question or statement • 3-5 alternatives/options are available with only one correct or appropriate answer (The Key) • Incorrect or inappropriate answers are called distractors/distracters/foils

19. Forms of Objective Test (Contd.) • More than two-choice items (Contd.) • (ii) Multiple-choice Items(Contd.) • Extensive content coverage is possible • Measures several levels of knowledge • Scoring is highly objective—lay person/machine • Chance for guessing is minimal

20. Forms of Objective Test (Contd.) • More than two-choice items (Contd.) • (ii) Multiple-choice Items(Contd.) • Examples: • 1) When human beings breathe out, what do they breathe out? • A. air B. oxygen C. nitrogen D. carbon dioxide • 2) An example of a living thing is _________. • A. water B. fire C. egg D. wind

21. Forms of Objective Test (Contd.) • More than two-choice items (Contd.) • (ii) Multiple-choice Items(Contd.) • Quality Assurance • Items should be constructed after every lecture • There should be only one correct option • All the options should appear plausible • Letters of the correct options should vary • Avoid “None of the above” and “All of the above “ as options • Do not start a stem with a dash

22. QUALITY ASSURANCE IN OBJECTIVE TESTING • 1. Statement of lesson objectives • 2. Table of Specification/Test Blueprint to ensure fair representation of course content • Objectives/Contents can be stated as percentages, number of items or item numbers.

23. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.)

24. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 3. Determine the item format to be used • 4. Item Writing • (i) Use terms whose definitions are precise from examinees’ point of view • (ii) Stem should not start with a dash or blank space • (iii) Stem should not contain negatives, especially, double negatives • (iv) Use students’ misconceptions and alternative conceptions as distractors

25. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 4. Item Writing (Contd.) • Example (Misconceptions/Alternative Conceptions): • Energy is an abstract idea that is not easy to define. Which of the following statements best represents what energy is? • A. energy is force B. Energy is a quantity • C. energy behaves as a fluid D. energy is fuel

26. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 4. Item Writing (Contd.) • Example (Misconceptions/Alternative Conceptions—Multiple levels): • Identify which of the following are characteristic of the vertebrate respiratory surface: • 1. moist 2. vascularized 3.semi-permeable 4. freely • permeable 5. dry

27. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 4. Item Writing (Contd.) • Example (Misconceptions/Alternative Conceptions—Multiple levels) Contd.: • A. 1,2,3. B. 1,2,5. C. 2,3,5. D. 2,4,5. E.1,3,5. • 5. Review of Test Items • Present the test items to others for review. HOD should review the items to spot grammatical errors, omissions and gender friendliness of the items. • Regulation on multiple examiners is useful here

28. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 6.Trial Run of the Test • Try out the test on a representative sample of the target population and score the test in readiness for item analysis. 7. Item Analysis Arrange candidates’ scripts according to their scores. Different authors divide the scores into three or four equal parts. (i) Item Difficulty/Facility Index (ii) Item Discrimination Power (iii) Effectiveness of the distractors (iv) Further Item Validity

29. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) (i) Item Difficulty Index (Contd.) If 15/20 of the upper group and 5/20 of the lower group got item 3 correct (B), therefore, 20 (15+5) persons out of 40 (20+20) got the item correct. Item 3 ITEM 3

30. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) 7. Item Analysis (Contd.) (i) Item Difficulty Index (Contd.) Item Difficulty = 100x (R/T) = 100x20/40 = 50% where R = the number of persons in the upper and lower thirds combined who got the item right, and T = total number of persons in the upper and lower thirds combined Therefore, item 3 is moderately difficult. Difficulty Index of < 10% =Too difficult; > 90% = Too easy

31. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 7. Item Analysis (Contd.) • (ii) Item Discrimination Power • Extent to which each item discriminates between those in the upper third and the lower third, respectively. • Item Discrimination Power = Ru – Rl/½ T, where: • Ru = no. of those in the upper third who got the item right • Rl = no. of those in the lower third who got the item right • T = total number of persons in the upper and lower thirds

32. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 7. Item Analysis (Contd.) • (ii) Item Discrimination Power (DP) (Contd.) • The DP for item 3 above is • Ru – Rl = 15-5 = 10 =0.50 • ½ T ½(20+20) 20 • Range of DP = -1.00 to +1.00 • Interpretation: 0.40 and above =Very good item 0.30 to 0.39 = Reasonably good but room for improvement 0.20 to 0.29 = Marginal, improve < 0.19 = Poor, reject or revise

33. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 7. Item Analysis (Contd.) • (iii) Effectiveness of Distractors • Every option of an item is chosen by at least one person in the upper or lower third of the class while each distractor is chosen more frequently by those in the lower third than in the upper third.

34. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 7. Item Analysis (Contd.) • (iii) Effectiveness of Distractors (Contd.) • Option D is the correct answer in item 5 • Options A, B, & C are all functional distractors with at least one candidate picking each of them • Options A & B discriminate rightly while option C discriminates wrongly • Item 5 • Item 5 • Item 5

35. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 7. Item Analysis (Contd.) • (iv) Coefficient of Item Validity (Yarroch, 1991) • This involves a post-test clinical interview of examinees to find out “the relationship between the examinees’ knowledge and the knowledge the examination item was intended to measure” (p. 621). Essay items equivalent to the multiple-choice items could also be administered to the examinees to further test their proper understanding.

36. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 7. Item Analysis (Contd.) • (iv) Coefficient of Item Validity (Yarroch, 1991) • Civ = (% correct for correct reason) + • (% incorrect for correct reason)

37. QUALITY ASSURANCE IN OBJECTIVE TESTING (CONTD.) • 8. Amend the items as necessary based on the results of item analysis and reject items that are not suitable • 9. Select and compile the final test items duly following rules guiding objective testing.

38. Ethical principles in evaluation • The teacher must display a sense of duty towards the stakeholders • The method of evaluation must protect the integrity of examinees and examiners • The examinees must be provided with adequate information about the test and the method for its preparation

39. Ethical principles in evaluation contd. • All tests items should be vetted by the Heads of Department and preferably scored twice by two independent scorers as stipulated in the university regulations • There should be sufficient check against errors in adding marks and preparation of mark sheets • There should be continuous research on improving the evaluation programme by individual lecturers and other stakeholders • The actual test items should never be revealed to any unauthorized persons

40. Ethical principles in evaluation contd. • Test material should never be revealed to the students in advance directly or indirectly • The test scores or trend in the test performance of a student should not be revealed to an unauthorized person. (Sidhu, 2005, p. 357)

41. CONCLUSION • Objectives tests, especially multiple-choice tests, are good for preparing global citizens since they are popular among many countries of the world • They can be used to test practically any subject matter, they only require some training • Objective tests are fun to prepare, with adequate practice, especially using continuous assessment tests as starting points. • They take off the drudgery of marking from lecturers • Students have access to their results in good time • THANK YOU

42. BIBLIOGRAPHY • Abimbola, I. O. (1996). Advances in the development and validation of instruments for assessing students’ science knowledge. In G. A. Badmus & P. I. Odor (Eds.), Challenges of managing educational assessment in Nigeria (pp. 33-40). Kaduna: JAMB, NABTEB, & NBEM. • Abiri, J. O. O. (2006). Elements of evaluation, measurement and statistical techniques in education. Ilorin: Library and Publications Committee, University of Ilorin, Ilorin, Nigeria • Gillette, F. (n.d.). Multiple choice tests: Composition, assembly and analysis. San Francisco, CA: San Francisco State University. http://www.sfsu.edu. Retrieved 21/9/14.

43. BIBLIOGRAPHY (CONTD.) • http://www.carnet.hr Retrieved, 21/9/14 • http://www.testing.byu.edu Retrieved, 21/9/14 • http://www.uleth.ca Retrieved, 21/9/14 • Sidhu, K. S. (2005). New approaches to measurement and evaluation. New Delhi: Stirling Publishers

44. BIBLIOGRAPHY (CONTD.) • Tests: Objective tests. San Luis Obispo, CA: Academic Skills Center, California Polytechnic State University • Yarroch, W. L. (1991). The implications of content versus item validity on science tests. Journal of Research in Science Teaching, 28 (7), 619-629